Backport of #3043 to release-1.4.
Under node CPU contention the linstor-controller JVM cannot answer its
GET /health liveness probe within the piraeus-operator default timing
(timeoutSeconds 1, failureThreshold 3). The kubelet then liveness-kills
the container, the controller crash-loops, linstor-csi gets EPERM and
RWX-Filesystem DataVolumes never bind. Add CPU/memory requests (requests
only, no limits) so the JVM keeps answering /health under contention, and
relax the liveness timing to timeoutSeconds 5 / failureThreshold 6.
(cherry picked from commit 7801f7748e)
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Deleting a Tenant whose etcd DataStore has 0 referencing TenantControlPlanes left the DataStore stuck Terminating (finalizer kamaji.clastix.io/TenantControlPlane), hanging the etcd helm uninstall and wedging the tenant namespace. Backports clastix/kamaji#1122 (fixes clastix/kamaji#1115) as a source patch on the pinned 26.3.6-edge build. Refs #3018.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 47638f8569)
A node group is sized either by instanceType or by explicit resources, but the
generated values schema required resources on every node group. A node group
with instanceType and no resources — or an empty resources dropped during
typed-CR to HelmRelease translation — failed schema validation with "missing
property 'resources'" even though sizing was fully specified.
Mark resources optional in the node group definition and regenerate the schema,
API types, README, and ApplicationDefinition. The chart template emits
domain.cpu/memory only when resources is present, so a node group sized purely
by instanceType now passes validation and installs.
When a node group sets both instanceType and explicit resources, explicit
resources take precedence: the instancetype is omitted from the rendered VM and
the per-group instancetype lookup is skipped, so KubeVirt accepts the explicit
domain.cpu/memory with no instancetype to conflict with. instanceType stays
required and defaulted (no schema change), so the console's required
instanceType dropdown keeps working — sending the default instanceType plus
explicit resources simply sizes the VM by resources. This replaces an earlier
render-time hard failure on the both-set case, which was unreachable to clear
from the dashboard (the required dropdown cannot be emptied).
Explicit resources are all-or-nothing alongside an instanceType: a partial spec
(only cpu, or only memory) on a group that carries a non-empty instanceType
would emit one domain override next to the instancetype — which KubeVirt rejects
— so it now fails at render time with a clear error. Partial resources on a
group without an instanceType (used to tune kubelet reservations) are unaffected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit a349c753ad)
CONSOLE_BRANCH was hardcoded to `main` on both the main and release-1.4
branches, so every patch build of cozystack 1.4.x cloned cozystack-ui@main —
picking up unreleased UI features and bypassing the patch release gate.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The vm-disk source image dropdown in the console lists PersistentVolumeClaims in cozy-public (the vm-default-images-* golden images). The browser issues that list as the tenant identity (e.g. system:serviceaccount:tenant-root:tenant-root), but the cozy:tenant:dashboard Role only granted read on Flux helmrepositories and helmcharts. The list returned 403 and the dropdown stayed empty even when the images existed.
Add get/list/watch on persistentvolumeclaims to the Role. cozy-public is a shared public namespace, so read on its PVCs is safe to expose to every tenant dashboard subject. A helm-unittest regression pins the grant.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit bb322138ec)
With publishing.proxyProtocol enabled the host emits a cozystack.ouroboros
Package via the no-override Package macro, so the proxy inherits the wrapper
chart default backend ingress-nginx-controller.cozy-ingress-nginx. That
layout describes a managed Kubernetes tenant cluster; on the host,
extra/ingress deploys ingress-nginx into the root tenant namespace as
root-ingress-controller in tenant-root. The composed backend FQDN never
resolves, so ouroboros-proxy never reaches Ready.
Emit the host ouroboros Package with a proxy.target override derived from
publishing.ingressName (namespace plus <short>-ingress-controller service
name), mirroring the trimPrefix transform extra/ingress applies. Add bundle
helm-unittest coverage for the default and a non-default ingress name, and
correct the misleading wrapper values.yaml comment.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit f90411da7a)
v0.8.0 makes the TCP proxy log an explicit reason when its backend
readiness check fails, instead of staying silently NotReady. That
silent failure mode is exactly what makes a mispointed proxy backend
hard to diagnose, so the upgrade pays off the moment the target is
wrong. The bump also carries the v0.7.3 migration of the kubectl
sidecar image off Docker Hub to mirror.gcr.io, sidestepping anonymous
pull rate-limits on shared CI/runner IPs.
Chart manifest and controller image are re-pinned by digest in lockstep
with the version, and the image-pin test fixture is updated to match.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit 3a5ff3d9c8)
The vendored COSI controller (v0.2.2) sets only BucketClaim.status.bucketName
in the dynamic-provisioning path and hardcodes bucketReady=false, never
re-reading the Bucket afterwards. The controller does not watch Buckets, so the
readiness transition the sidecar performs (false->true) is never reflected on
the BucketClaim: bucketReady stays false forever and BucketAccess is never
granted, leaving provisioned buckets without credentials.
Re-read the live Bucket after create and propagate its readiness, so the
periodic resync converges the BucketClaim to ready. This matches the behavior
the rewritten upstream controller already has on its main branch.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit ed20a4322a)
Ivan's original schema edits were correct in intent but lived in the
generated artifacts (values.schema.json and the embedded openAPISchema
in postgres-rd/cozyrds/postgres.yaml). `make generate` reverts them
because values.yaml's `@field {map[string]string}` annotation is the
source of truth -- same regeneration trap that caught #2466 (issue #2597).
cozyvalues-gen v1.4.0 (cozystack/cozyvalues-gen#25, merged 2026-05-21)
adds an `intOrString` type alias for map values that emits
`additionalProperties.anyOf: [integer, string]` plus
`x-kubernetes-int-or-string: true`. Move the change to source by
flipping the annotation, then regenerate:
- packages/apps/postgres/values.yaml: `@field {map[string]intOrString}`
with updated description noting integer values are accepted.
- packages/apps/postgres/values.schema.json: regenerated -- now carries
anyOf + x-kubernetes-int-or-string: true.
- packages/system/postgres-rd/cozyrds/postgres.yaml: embedded
openAPISchema regenerated to match.
- packages/apps/postgres/README.md: regenerated type column.
- api/apps/v1alpha1/postgresql/types.go: Parameters is now
`map[string]intstr.IntOrString`.
The result survives future regenerations. The template's
`{{ \$value | toString | quote }}` already coerces both inputs to
quoted strings, verified end-to-end with helm template against both
`max_connections: 100` (int) and `max_connections: "100"` (string) --
identical render output. Dangerous-param block list still rejects
archive_cleanup_command and recovery_end_command additions from the
prior commit.
Co-Authored-By: IvanHunters <xorokhotnikov@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 14e196cf22)
Add archive_cleanup_command and recovery_end_command to the dangerous
parameters blocklist in db.yaml. Both are PostgreSQL shell-exec hooks
identical in threat model to archive_command/restore_command and must
not be settable by tenants.
Also update the BLOCKED list in values.schema.json, values.yaml, and
the embedded openAPISchema in cozyrds/postgres.yaml to reflect the
two new blocked parameters.
Additionally update the embedded openAPISchema in cozyrds/postgres.yaml
to allow integer values for postgresql.parameters (anyOf string|integer),
matching the fix already applied to values.schema.json.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 388e823916)
The kamaji control-plane-provider override in templates/providers.yaml
targeted containers[name=manager], but the upstream container is named
"controller". The CAPI operator merges container overrides by name, so
the intended cpu: 1 / memory: 1024Mi limits were silently dropped and
the upstream default of memory: 128Mi was used, producing repeated
OOMKilled restarts of the kamaji controller (exit code 137).
The other three providers (core, bootstrap, infraprovider/kubevirt)
ship upstream with no resources block and have no cozystack override,
so they run BestEffort and are first to evict under memory pressure.
Set modest limits (500m / 256Mi) and requests (10m / 64Mi) matching
typical CAPI controller-manager footprint.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 6ae66f72a6)
Move the kube-rbac-proxy image override from a patch + vendored-chart
edit to a values.yaml override under kubeRbacProxy.image.repository.
This matches the project convention documented in docs/agents/overview.md
("Override configuration in root values.yaml") and the pattern used by
sibling packages such as kamaji.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit e5437240e7)
The gcr.io/kubebuilder/kube-rbac-proxy image is no longer available
since GCR was deprecated. Replace it with quay.io/brancz/kube-rbac-proxy
from the original upstream author.
Applied as a local patch in patches/ because upstream chart 2.8.1+ drops
kubeRbacProxy entirely but also bumps CRDs to controller-gen v0.19.0 with
significant schema changes — out of scope for this fix.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit d0a0b06afd)
## Summary
Replaces the old openapi-ui with the new cozystack-ui build from the
cozystack-ui repository.
## Key Changes
### New UI
- **Image**: `999669/cozystack-ui:latest@sha256:ea4e832c2...`
- **Architecture**: `linux/amd64`
- **Repository**: https://github.com/cozystack/cozystack-ui
(feat/ui-overhaul branch)
### Removed Components
- Removed BFF (Backend-For-Frontend) container - new UI communicates
directly with Kubernetes API
- Removed old openapi-ui and openapi-ui-k8s-bff build targets from
Makefile
- Removed old UI Dockerfiles and patch files
### RBAC Updates
- Added read permissions for `cozystack.io/applicationdefinitions` to
fix 403 errors
- New UI requires access to applicationdefinitions for marketplace and
catalog functionality
### Bug Fixes
- Fixed VNC WebSocket connections to use dynamic URLs instead of
hardcoded localhost:8001
- VNC now works properly in-cluster deployments
## Testing
Tested on dev3.infra.aenix.org cluster:
- ✅ Dashboard loads correctly
- ✅ Application definitions accessible
- ✅ VNC connections working
- ✅ Backup pages functional
- ✅ All TypeScript errors resolved
## Migration Notes
The new UI is a complete rewrite with:
- Modern React 19 + TypeScript stack
- Improved backup management pages
- Better type safety and error handling
- Direct Kubernetes API communication (no BFF needed)
… with others
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
align Altinity backup strategy field name for similarity with others
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
implement backup strategy for MariaDB
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added native MariaDB backup/restore strategy using the operator-native
BackupClass/Plan/RestoreJob flow with example manifests for end-to-end
workflows.
* **Documentation**
* Added MariaDB how‑to and examples; marked legacy restic/chart-side
backup parameters as DEPRECATED and documented migration guidance.
* **Tests**
* Added unit tests covering MariaDB backup/restore reconciliation,
templating, storage handling, idempotency, URIs, and deadlines.
* **Chores**
* Extended controller RBAC to allow observing MariaDB operator
resources.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2605)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds infrastructure for building a patched kilo image from upstream
`main` branch source, following the same pattern as the kamaji package.
The patch introduces a new `cross` mesh granularity mode — WireGuard
tunnels are created only between nodes in different locations, while
intra-location traffic stays on the CNI overlay.
Changes:
- `images/kilo/Dockerfile` — multi-stage build: fetches upstream
tarball, applies patches, builds `kg`/`kgctl` binaries with CNI plugins
and iptables-wrapper
- `images/kilo/patches/feat-add-cross-mesh-granularity.diff` — adds
`CrossGranularity` to the kilo binary
- `Makefile` — new `image:` target for building the patched image,
updated `update:` target to pin upstream commit SHA
### Screenshots
N/A — no UI changes.
### Release note
```release-note
feat(kilo): add patched image build with cross mesh granularity support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added "cross" mesh granularity mode for finer topology grouping,
adjusted routing/encryption behavior
* Added e2e tests covering cross-mode scenarios for improved reliability
* **Chores**
* Build automation now tracks latest main commit and derives image
digests from build metadata
* Introduced multi-stage container build and runtime packaging for kilo,
including required networking tooling
* **Configuration**
* Updated default image reference to a new repository and digest
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2604)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Two related quality-of-life additions to the Keycloak system component:
1. **`keycloak` chart now accepts `extraEnv`.** Operators can append
extra
`EnvVar` entries to the Keycloak StatefulSet container without forking
the
chart. Useful when a custom theme needs runtime configuration via
`kcContext.properties`, or when an extension expects environment
variables.
2. **`keycloak-configure` chart now exposes a top-level `userProfile`
value**
that is mirrored verbatim into
`ClusterKeycloakRealm.spec.userProfileConfig`.
This lets operators declare custom user-profile attributes (required
fields, validators, permissions, ...) declaratively, alongside the
existing
`login` and `smtp` blocks.
Both values default to empty, so existing deployments are unaffected.
### Release note
```release-note
feat(keycloak): support extraEnv values and userProfile customization
```
Replace deployment-specific examples in the new extraEnv and
userProfile docs with generic placeholders so the chart comments do
not lean on any particular downstream use case.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Operators can now declare custom user-profile attributes (required
fields, validators, permissions, ...) via a new top-level userProfile
value. The value is mirrored verbatim into the
ClusterKeycloakRealm.spec.userProfileConfig managed by the EDP
keycloak-operator.
The default remains empty, so realms that don't need custom user
profile configuration are unaffected.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Custom themes (Keycloakify, FTL) often need runtime values from the
deployment environment via kcContext.properties. Adding an extraEnv
list to the chart lets operators inject those values per cluster
without forking the chart.
The new field appends verbatim to the container env list and accepts
the full corev1.EnvVar schema, including valueFrom for secret or
configMap-backed values.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:
nano -> t1.nano micro -> t1.micro
small -> t1.small medium -> c1.small
large -> c1.medium xlarge -> c1.large
2xlarge -> c1.xlarge
Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.
Three hardcoded legacy literals in chart templates migrated:
- packages/apps/harbor/templates/harbor.yaml:121 (portal)
- packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
- packages/system/keycloak/templates/sts.yaml:115 (heap fallback)
packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.
Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Point values.yaml to ghcr.io/cozystack/cozystack/kilo with the
freshly built image digest, replacing the upstream ghcr.io/squat/kilo
reference.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Address review feedback from gemini-code-assist on Dockerfile:36:
Remove custom Fastly CDN repo pinned to v3.20 while base image is
alpine:3.23 — use default repos from the base image instead.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add Dockerfile for building kilo from upstream main branch source with
patch support, CNI plugins, and iptables-wrapper. Update Makefile with
`image:` target for building the patched image and an updated `update:`
target that pins the upstream commit SHA.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The v0.7.2 ouroboros release adds --log-level / OUROBOROS_CONTROLLER_LOG_LEVEL,
plants Debug logs at every informer event handler (AddFunc/UpdateFunc/
DeleteFunc with kind+namespace/name via describeObj) and at reconcile
entry/exit, wires SetWatchErrorHandler on Ingress / Gateway / HTTPRoute
informers, and surfaces cache-sync elapsed at Info. The diagnostic
surface targets the kamaji-tenant-via-konnectivity flake where bats
sees a fresh Ingress create succeed against the tenant apiserver but
the controller's coredns-custom rewrite lands roughly 7-10 minutes
later — without per-event visibility the failure mode is
indistinguishable between 'watch never fires', 'reconcile silently
errors', and 'API list returns empty until reflector reconnect'.
Bumps the chart-manifest digest, the controller-image digest, and the
cozystack-side image-pin assertion in lockstep. The bats deadline for
the coredns-custom rewrite check rises from 180s to 900s — even on
the broken-watch path the reflector eventually reconnects within
that budget, so the test stops failing on the symptom while we keep
the diagnostic on to learn what the symptom actually is.
cozystack tenants force controller.logLevel: debug via the
defaultOuroborosValues helper; operators that want quieter logs can
override through addons.ouroboros.valuesOverride. Two helm-unittest
fixtures pin both the default and the override path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds
[`lexfrei/ouroboros@v0.7.0`](https://github.com/lexfrei/ouroboros/releases/tag/v0.7.0)
— a Go reimplementation of
[`compumike/hairpin-proxy`](https://github.com/compumike/hairpin-proxy)
— and wires it into cozystack as a *conditional default*: a single
`publishing.proxyProtocol: true` switch turns PROXY-protocol on at the
host ingress-nginx **and** auto-deploys ouroboros to fix the resulting
hairpin-NAT problem (intra-cluster traffic to the cluster's own public
hostnames arriving at ingress-nginx without the PROXY header it now
requires). Default stays `false` — clusters that do not run
PROXY-protocol get zero new resources, zero RBAC delta, zero behaviour
change.
The chart is platform-agnostic at v0.7.0: the controller and proxy
binaries auto-detect the cluster's DNS domain from `/etc/resolv.conf`
inside the pod and compose the rewrite target FQDN and the backend host
at runtime as `<svc>.<ns>.svc.<cluster-domain>`. Cozystack tenants on
`cozy.local`, kubeadm-bootstrapped clusters on `cluster.local`,
federations on `k8s.example.com`, custom-domain RKE2/k3s — all work
without operator override. Verified end-to-end on a `cozy.local`
cluster: controller logs `cluster-domain=cozy.local` at startup, chart
emits `--proxy-service-name + --proxy-service-namespace` (no chart-baked
`--proxy-fqdn`), Corefile picks up the rewrite block between BEGIN/END
markers, in-pod `dig <ingress-host>` returns the ouroboros-proxy
ClusterIP. Operators that prefer a chart-baked FQDN can still set
`controller.clusterDomain` explicitly — the explicit-override path is
preserved for non-Service backends and for split-horizon DNS topologies.
The chart runs in `coredns` mode for the host (the host CoreDNS is
Talos-managed and does not carry an `import` directive, so the
alternative would silently no-op there) and in `coredns-import` mode for
tenant deployments (tenants run cozystack-coredns, which carries the
import directive and ships an empty `kube-system/coredns-custom`
ConfigMap; ouroboros writes plugin-only `rewrite name` snippets into the
`ouroboros.override` data key without ever touching the chart-rendered
Corefile, so it does not race the chart that re-templates
`data.Corefile` on every Flux reconcile).
### Layout
* `packages/system/ouroboros/` — vendored chart from
`oci://ghcr.io/lexfrei/charts/ouroboros@0.7.0`. Cozystack-side defaults
pinned in `values.yaml`: `controller.mode=coredns` (host scenario),
`proxy.target.namespace=cozy-ingress-nginx` (override the chart-default
`ingress-nginx` namespace; chart-default
`serviceName=ingress-nginx-controller` matches as-is), Gateway-API
watching left at upstream's default of `false`. `proxy.target.host` is
left empty so the proxy composes the backend host at runtime. Image is
digest-pinned at
`0.7.0@sha256:478665e05dd0ffc4c1f7764320ea24b52251781884ddfd668d93a03e39d9094c`
and pulled directly from `ghcr.io/lexfrei/ouroboros` — this package is
intentionally not mirrored under `ghcr.io/cozystack/*`. The chart
manifest itself is also digest-pinned via
`OUROBOROS_CHART_DIGEST=sha256:f28bfac9fd7070b1f3357983ff5aad28c16115e7de1cfc16ee568a3b4cfc9d7e`
in the package Makefile, with a Chart.yaml-version vs. Makefile-version
assert that aborts the update target on lockstep drift. Helm-unittest in
`tests/` covers controller args, RBAC narrowing per mode, Gateway-API
gating, runtime-composition flag emission in both `coredns` and
`coredns-import` modes, image pin drift.
* `packages/system/coredns/templates/coredns-custom.yaml` +
`values.yaml` — empty `coredns-custom` ConfigMap as the chart-managed
`import` prerequisite, plus the `import /etc/coredns/custom/*.override`
plugin in the `.:53` server block + `extraVolumes`/`extraVolumeMounts`
mount at `/etc/coredns/custom/`. Same package serves host and tenant
CoreDNS, so this covers both layers without duplication. Helm-unittest
pins the rendered Corefile via a snapshot so future `make update` does
not silently drop the import directive. The empty rendered ConfigMap
deliberately omits `data:` — Helm three-way merge has no template-side
keys to compare against, so apiserver-side runtime writes (the
`ouroboros.override` data key) survive every chart upgrade. The
`notExists: data` assertion in `tests/coredns_custom_test.yaml` pins the
invariant.
* `packages/core/platform/sources/ouroboros.yaml` — new
`cozystack.ouroboros` PackageSource for the host install (`dependsOn:
cozystack.networking, cozystack.ingress-application`).
* `packages/core/platform/sources/kubernetes-application.yaml` —
`kubernetes-ouroboros` component registered against `system/ouroboros`
so the tenant HelmRelease can reference it.
* `packages/apps/kubernetes/templates/helmreleases/ouroboros.yaml` —
tenant-side HelmRelease deployed via the tenant admin-kubeconfig,
mirroring the cert-manager pattern. A `cozystack.defaultOuroborosValues`
helper flips `mode` back to `coredns-import` for tenants and mirrors
per-tenant `addons.gatewayAPI.enabled` into the controller (forcing
`--gateway-api` on a tenant without Gateway-API CRDs CrashLoops the
controller on `WaitForCacheSync`). `controller.clusterDomain` is
intentionally NOT pre-set — chart 0.7.0 auto-detects per-pod from
`/etc/resolv.conf` and composes the rewrite target at runtime, so the
same wrapper renders correctly on `cluster.local`, `cozy.local`, or any
custom tenant cluster-domain. Operators on a non-default cluster-domain
can still override via
`valuesOverride.ouroboros.controller.clusterDomain`. `dependsOn` grows a
conditional `gateway-api-crds` entry on the same gate. If
`addons.ouroboros.enabled` is true while `addons.ingressNginx.enabled`
is false, the HelmRelease fails the render with a clear error rather
than silently skipping — the hairpin fix has nothing to fix against
without ingress-nginx, and a no-op addon that quietly does nothing is
harder to debug than an explicit render-time error. The valuesOverride
gateway-api guard accepts every truthy spelling Helm/YAML emit (bool
`true`, string `"true"`/`"True"`/`"TRUE"`, numeric `1`, YAML
`yes`/`Yes`/`YES`/`on`/`On`/`ON`/`y`/`Y`) via
`toString`-then-`lower`-then-string-compare; direct bool `eq` would
error on type mismatch when an operator passes the value as a string
from `--set`, and case-insensitive folding catches `--set ...=On` /
`--set ...=True` that pure-lowercase comparison would miss. Tenant
disable is single-step: flipping `addons.ouroboros.enabled: false`
deletes the rendered HelmRelease, helm-controller runs `helm uninstall`
on the tenant, and the chart's pre-delete cleanup hook
(`coredns-cleanup-hook.yaml`, vendored at 0.7.0) nulls the
`ouroboros.override` key on the tenant's `kube-system/coredns-custom`
ConfigMap automatically. No render-time guard is wired on the tenant
side — gating on a `lookup` for the leftover HelmRelease would deadlock
the routine disable (the parent render runs before helm-controller
applies the missing-child diff, so the lookup would always find the
leftover at the moment of the flag flip). If the chart's hook fails to
land (controller pod stuck CrashLooping, ConfigMap RBAC drift, Job
timeout), operators recover manually via the documented tenant cleanup
recipe.
* `packages/apps/kubernetes/values.yaml` — new `OuroborosAddon` typedef
with `enabled` + `valuesOverride`. `make generate` re-emitted README,
JSON schema, the Go struct + deepcopy, and the kubernetes-rd
ApplicationDefinition openAPISchema in lockstep.
* `packages/core/platform/values.yaml` — new `publishing.proxyProtocol`
flag with extensive comment covering the upstream-LB precondition (the
L4 LB in front of ingress-nginx must already be injecting PROXY-v1
headers before the flag flips on, otherwise external traffic breaks),
the host-vs-tenant fan-out (the flag does not cascade into tenants), the
Talos re-render flap window, and the asymmetric disable path. The host
disable path: flipping back to `false` reverts ingress-nginx
PROXY-protocol config (flip the LB off first), but the host ouroboros
itself stays installed because its Package CR carries
`helm.sh/resource-policy: keep` — an explicit `kubectl delete
package.cozystack.io cozystack.ouroboros` (or `bundles.disabledPackages`
entry) actually triggers helm uninstall, at which point the chart's
pre-delete cleanup hook fires and patches the host Corefile
automatically. The values comment ships a manual `sed`-based cleanup
recipe as a fallback for the rare case where the chart's hook fails to
land (controller stuck in CrashLoop, ConfigMap RBAC drift, hook
timeout); the `publishing.proxyProtocolAcknowledgeUnclean` gate keys off
a `lookup` against the leftover Package CR to catch operators who flip
the flag back without confirming the cleanup landed.
* `packages/core/platform/templates/bundles/system.yaml` — flips between
the default-form and explicit-form `cozystack.ingress-application`
Package emission so the host ingress-nginx gets `use-proxy-protocol` +
`real-ip-header=proxy_protocol` injected only when the flag is true
(`use-forwarded-headers` and `compute-full-forwarded-for` are
deliberately *not* on the list — they would let any upstream proxy spoof
`X-Forwarded-For`); the same conditional emits `cozystack.ouroboros`.
Host Gateway-API watching is left at the upstream chart's default of
`false` because cozystack does not install Gateway-API CRDs cluster-wide
on the host.
* `hack/e2e-apps/ouroboros.bats` — auto-discovered smoke test for the
tenant addon (HelmRelease ready, `coredns-custom` ConfigMap present in
tenant, controller pod running, and after creating a TLS Ingress the
rewrite snippet shows up in the import ConfigMap; an in-tenant `dig`
from a netshoot pod also confirms CoreDNS actually serves the rewrite —
pod-readiness alone passes vacuously since the upstream chart ships no
readiness probes).
### Why not KEP-1860 `ipMode=Proxy`
Cilium's `kubeProxyReplacement: true` (cozystack default) drops the LB
frontend entirely when `ipMode != VIP`, breaking Service ingress for
L2/BGP-announced IPs. KEP-1860's Proxy mode is reserved for clusters
fronted by an external CCM-managed proxy LB. ouroboros is the correct
fix for L2/BGP-announce topologies that cozystack ships by default.
### Coverage
Covered by helm-unittest (host chart, coredns wrapper, tenant addon —
including the `kubernetesProvider` mock that exercises the lookup-driven
`proxyProtocolAcknowledgeUnclean` gate, the lookup-returns-nil
CI/dry-run path, the gateway-api truthy-spelling acceptance across
`true`/`"true"`/`"True"`/`"TRUE"`/`1`/`yes`/`on`, the assertion that
`--proxy-service-name` + `--proxy-service-namespace` are emitted instead
of a chart-baked `--proxy-fqdn` in both `coredns` and `coredns-import`
modes, the assertion that `--target-service-name` +
`--target-service-namespace` are emitted for the proxy backend, and the
`_namespace.etcd`-empty render gate) and a tenant e2e bats test that
creates a real Ingress, asserts the rewrite snippet lands in the import
ConfigMap, and confirms in-tenant DNS serves it (with an in-pod retry
loop tolerating CoreDNS reload latency). The host-level
`publishing.proxyProtocol: true` path is not e2e-tested in this PR — it
requires a PROXY-protocol-aware upstream LB, which the cozystack e2e
harness does not stand up. That path is exercised manually and by docs
review.
End-to-end runtime auto-detect was verified manually on a `cozy.local`
cluster (dev17): chart deployed at v0.7.0, controller logged
`cluster-domain=cozy.local`, the rewrite block landed in
`kube-system/coredns` Corefile pointing at
`test-ouroboros-proxy.test-ouroboros.svc.cozy.local.`, an in-pod `dig
<ingress-host>` resolved to the ouroboros-proxy ClusterIP on the first
attempt, and the chart's pre-delete hook restored the Corefile cleanly
on `helm uninstall`.
### Supply-chain note
Image and chart are pulled from a contributor's personal namespace
(`ghcr.io/lexfrei/...`). Existing precedent
(`packages/system/etcd-operator/Makefile` pulls from
`ghcr.io/aenix-io/...`) is similar but `aenix-io` is the
cozystack-adjacent org while `lexfrei` is one human. Worth a maintainer
call.
Pairs with cozystack/website#527 (operator docs).
### Release note
```release-note
feat(networking): add `publishing.proxyProtocol: true` to enable PROXY-protocol on host ingress-nginx and auto-deploy ouroboros (a Go reimplementation of compumike/hairpin-proxy) to solve the resulting hairpin-NAT problem; tenants can opt in per-cluster via `addons.ouroboros.enabled` (requires `addons.ingressNginx.enabled`). Default behaviour unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Ouroboros addon to address hairpin‑NAT for ingress‑nginx with
configurable enablement and tenant overrides.
* Added a packaged Ouroboros chart providing proxy, controller, CoreDNS
integration, external‑DNS and etc‑hosts modes.
* Introduced PROXY‑protocol gating with an explicit acknowledgement
flag.
* **Documentation**
* Expanded addon reference and platform docs with detailed
Ouroboros/PROXY‑protocol configuration and defaults.
* **Tests**
* Added extensive unit and e2e tests covering rendering, RBAC, uninstall
cleanup, and DNS behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Implemented CNPG-strategy controller for postgres backups with supporting in-place and to-copy restores.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* CNPG-based backup strategy for PostgreSQL with S3-backed object store
and controller support
* Option to reference existing Kubernetes Secret for S3 credentials
(configurable key names)
* Controller supports scheduled/ad-hoc backups and both in-place and
to-copy restores; safer credential handling (no inline keys)
* **Examples**
* Complete manifests: bucket, credentials, CNPG strategy, BackupClass,
Plan, BackupJob, RestoreJob, Postgres source/target
* **Security/RBAC**
* Controller RBAC expanded for CNPG resources; tests enforce no
cluster-wide Secret read
* **Documentation**
* README, chart values/schema and UI schema updated to document
secret-reference workflow and restore guidance
* **Tests**
* New end-to-end and unit tests covering CNPG flows, restore semantics,
cleanup behavior and RBAC constraints
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozystack build pipeline tags chart versions as `0.0.0+<git-sha>`.
The new coredns-custom ConfigMap interpolated `.Chart.Version` directly
into the `helm.sh/chart` label, producing `cozy-coredns-0.0.0+<sha>`
which the apiserver rejects: label values follow RFC 1123 and `+` is
not a legal character. The chart never installed, kube-system/coredns
inside tenant clusters never came up, every dependent HelmRelease
stalled, and the e2e suite hit the 2h job timeout without reaching
the bulk of the bats tests.
Apply the standard `replace "+" "_" | trunc 63 | trimSuffix "-"`
sanitization, matching the pattern the vendored upstream coredns chart
already uses in its own `helm.sh/chart` helper a few directories down.
Add a helm-unittest case that overrides the chart version to a build-
metadata form (`0.0.0+abc1234567`) and asserts the rendered label both
matches the sanitized literal and conforms to the RFC 1123 label-value
regex. Without the version override the bug is invisible to the test
suite — Chart.yaml ships the bare `0.0.0` placeholder and helm-unittest
defaults to that, so any future template that interpolates
`.Chart.Version` into a label value cannot regress past CI silently.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Updates the NVIDIA GPU Operator from v25.3.0 to v26.3.1 and adds an
**experimental** `vgpu` variant for SR-IOV vGPU on Ada Lovelace and
newer GPUs.
Changes:
- Update vendored upstream chart to v26.3.1 (CDI enabled by default,
component version bumps).
- Pin chart version in Makefile for reproducibility.
- Add `values-vgpu.yaml` for the vGPU variant with `defaultWorkload:
vm-vgpu` so the chart activates the vGPU Manager DaemonSet without
per-node labelling.
- Add `vgpu` variant to PackageSource alongside the existing `default`
(passthrough).
- Rename `values-talos.yaml` → `values-passthrough.yaml` and pin
`defaultWorkload: vm-passthrough` for symmetry with the vgpu variant.
The overrides apply to any host OS using vfio-pci, not Talos
specifically.
- Refresh `hack/dcgm-default-counters.csv` to match the new
dcgm-exporter image tag (`4.5.1-4.8.0`) — adds `DCGM_FI_DEV_FB_RESERVED`
and other fields introduced upstream.
`hack/check-gpu-recording-rules.bats` passes against the new fixture.
- Add `hack/check-gpu-operator-variants.bats` that renders both variants
via `helm template` and asserts the pinned ClusterPolicy values, so the
next chart bump cannot silently un-pin them.
- Add internal documentation for the vGPU bring-up.
The `vgpu` variant assumes the user supplies a proprietary vGPU Manager
image via Package CR component values — the .run cannot be
redistributed.
### `vgpu` variant is experimental
NVIDIA's `vgpu-device-manager` walks `/sys/class/mdev_bus/`, which does
not exist on Ada Lovelace and Blackwell — the DaemonSet errors with "no
parent devices found" and is disabled by default. Until an SR-IOV-aware
controller is shipped, profile assignment is an out-of-band step (`echo
<id> > /sys/.../current_vgpu_type` per VF) that must be re-applied after
every node reboot. Without it, `permittedHostDevices.pciHostDevices`
reports zero allocatable resources. The variant ships so the chart bump
and the underlying skeleton (DaemonSet + image plumbing + docs) land
together; production deployments need an automated profile-assignment
mechanism in addition to this PR.
### KubeVirt requirement
KubeVirt advertises VFs via `permittedHostDevices.pciHostDevices` rather
than `mediatedDevices`. This requires kubevirt/kubevirt#16890 ("vGPU:
SRIOV support", merged 2026-04-10) which is `main`-only — released tags
`v1.6.x` / `v1.7.x` / `v1.8.x` do not include it. The patch is targeted
at the next minor KubeVirt release (v1.9.0); track the PR for the actual
release tag. Backports to released branches are not planned.
### Behavior changes inherited from upstream chart v25 → v26
The upstream chart flipped two defaults that are pinned back to `false`
here, so `default` (passthrough) deployments do not silently change
footprint on upgrade:
- `ccManager.enabled` was flipped to `true` upstream — would auto-enable
Confidential Computing on Hopper hardware (H100/H200/B100) after
upgrade.
- `vgpuDeviceManager.enabled` was flipped to `true` upstream — DaemonSet
would render even in passthrough mode.
Both stay opt-in via Package CR overrides. The new bats test guards the
pin.
Other inherited changes that are user-visible and worth pinning down
before upgrade:
- `driver.licensingConfig.configMapName` is deprecated upstream in
favour of `driver.licensingConfig.secretName`. The deprecation marker
lives in the CRD's `description` field (visible via `kubectl explain`) —
there is no helm-render-time warning, the chart template silently
prefers `secretName` and falls back to `configMapName`. Operators with
existing licensing overrides should migrate the key on the next chart
upgrade.
- `driver.version` upstream default jumped from `570.x` to `580.x`.
Invisible on the `default` variant (which sets `driver.enabled: false`),
user-visible for anyone running the upstream Ubuntu driver path.
- `vfioManager.image` switched from
`nvcr.io/nvidia/cuda:12.8.1-base-ubi9` to
`nvcr.io/nvidia/cloud-native/k8s-driver-manager:v0.10.0`. Affects every
`default` (passthrough) deployment on upgrade — different image,
different layer, different image-pull surface. Worth pre-mirroring for
air-gapped clusters.
- `dcgmExporter.image` switched to the `-distroless` flavour. Smaller
surface, fewer base-image CVEs, but no shell — operators who used to
`kubectl exec` into the exporter for ad-hoc debugging will need to
switch to `kubectl debug` or container ephemeral images.
- `CDI` (Container Device Interface) is enabled by default in v26.3.1;
this is upstream's recommended setting and is left as-is.
- `node-feature-discovery` subchart source switched from the GitHub
Pages Helm repo to `oci://registry.k8s.io/nfd/charts`. Air-gapped
installations need to mirror the new OCI URL.
### Talos
Talos is **not recommended** for the vGPU variant. NVIDIA does not
publicly distribute the vGPU guest driver — it requires NVIDIA
Enterprise Portal access — and Sidero closedsiderolabs/extensions#461
noting that they cannot support vGPU "unless NVIDIA changes their
licensing terms or provides us a way to obtain, test, and distribute the
software". Building a Talos system extension that bakes in the vGPU
driver is therefore not feasible without a private fork that violates
the EULA. Passthrough (`default` variant) on Talos is unaffected.
### Release note
```release-note
[gpu-operator] Update NVIDIA GPU Operator to v26.3.1 and add experimental `vgpu` variant for SR-IOV vGPU on Ada Lovelace / Blackwell (L4, L40, L40S, B100; Ubuntu host; requires KubeVirt v1.9.0+ for `pciHostDevices` advertising). The variant ships skeleton support — profile assignment (`current_vgpu_type` per VF) is currently out of band and must be automated by the operator before production use. On the existing `default` variant: `ccManager` and `vgpuDeviceManager` upstream defaults are pinned back to `false` to preserve prior behavior on chart upgrade; `defaultWorkload` is now pinned to `vm-passthrough` (was unset, falling back to `container`); `driver.licensingConfig.configMapName` is deprecated by upstream in favour of `secretName`; `driver.version` upstream default moves 570 → 580; `vfioManager.image` switches from CUDA UBI9 to k8s-driver-manager (relevant for air-gapped mirrors); `dcgmExporter` image switched to the `-distroless` flavour. NFD subchart source switched to `oci://registry.k8s.io/nfd/charts`.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced vGPU support variant with end-to-end configuration guide,
validation procedures, and troubleshooting documentation
* **Updates**
* Upgraded GPU Operator to v26.3.1 and Node Feature Discovery to v0.18.3
* Improved component health checks and added PodDisruptionBudget
availability controls
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Bumps the vendored cert-manager chart from v1.19.3 to v1.20.2. Pure
upstream bump
via `make update` — no cozystack-side template changes.
### Why
[Commit
`2b6e20cc`](https://github.com/cozystack/cozystack/commit/2b6e20cc)
migrated
the platform's ACME HTTP-01 setup to the modern `ingressClassName` API
on both
the ClusterIssuer's solver and the per-tenant override annotation
`acme.cert-manager.io/http01-ingress-ingressclassname`. The override
annotation,
however, was only added to upstream cert-manager in v1.20.0
([cert-manager#8244]) — v1.19.x silently ignores it. Result:
cert-manager always
falls back to the ClusterIssuer's solver class (`tenant-root`), so
HTTP-01 solver
Ingresses for non-`tenant-root` tenants land on the wrong controller and
Let's
Encrypt validation fails for tenant-scoped Ingresses (grafana, harbor,
alerta,
etc. inside child tenants).
Targeting v1.20.2 (not .0):
- [cert-manager#8655] (in v1.20.1) — restores
`clusterissuers/issuers/finalizers`
RBAC dropped in v1.20.0.
- [cert-manager#8665] (in v1.20.2) — webhook helm-template fix.
### Verified live on dev10
| Stage | `Challenge.spec.solver.http01.ingress.ingressClassName` |
|---|---|
| Pre-bump (v1.19.3) | `"tenant-root"` (bug) |
| Post-bump (v1.20.2) | `"<tenant>"` (fixed) |
Full propagation chain confirmed end-to-end: source Ingress annotation
-> ingress-shim
sets `http01-override-ingress-ingressclassname` on the Certificate ->
acmeorders sets
`spec.solver.http01.ingress.ingressClassName` on the Challenge -> http01
solver creates
the solver Ingress on the tenant's class.
### Upgrade considerations
- **Container UID/GID change**: 1000/0 -> 65532/65532 across controller,
webhook,
cainjector. PSP / SecurityContext rules pinning UID 1000 will need
updating.
- **Helm chart image structure** refactored from `image.repository` to
`imageRegistry` + `imageNamespace` + `image.name`. Cozystack's wrapper
`values.yaml` is empty, so rendered image references are unchanged
(`quay.io/jetstack/cert-manager-{controller,webhook,cainjector,acmesolver}:v1.20.2`).
- **NetworkPolicy templates restructured**: legacy
`networkpolicy-egress.yaml`
removed; per-component `networkpolicy-cainjector.yaml` and
`networkpolicy-cert-manager.yaml` added. All gated on
`*.networkPolicy.enabled: false` (default). No behavioral change.
- **CRDs**: `selectableFields` added (Kubernetes >=1.30) and Azure DNS01
`zoneType`
enum. Both pure additions.
- **RBAC**: gains `clusterissuers/issuers/finalizers` and
`gateway.networking.k8s.io.listenersets[/finalizers]`.
- `DefaultPrivateKeyRotationPolicyAlways` feature gate moves Beta -> GA,
can no
longer be disabled. Already default since v1.18; no behavior change.
### Release note
```release-note
fix(cert-manager): bump to v1.20.2 - fixes HTTP-01 solver IngressClass
propagation for tenant-scoped Ingresses (the
`acme.cert-manager.io/http01-ingress-ingressclassname` annotation was a no-op
in prior cert-manager versions). Default container UID/GID changes from
1000/0 to 65532/65532 - review your PSP/SecurityContext rules.
```
[cert-manager#8244]:
https://github.com/cert-manager/cert-manager/pull/8244
[cert-manager#8655]:
https://github.com/cert-manager/cert-manager/pull/8655
[cert-manager#8665]:
https://github.com/cert-manager/cert-manager/pull/8665
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added network policy support for controller, webhook, and CA injector
* Added Azure DNS zone type selection (public/private)
* Added webhook client verification configuration options
* Extended pod disruption budget configuration with unhealthy pod
eviction policy
* **Updates**
* Upgraded cert-manager to v1.20.2
* Enhanced global image registry and namespace configuration
* Expanded issuer reference field selectors
* Added support for extra containers in controller pod
* Updated issuer terminology and configuration
* **Documentation**
* Updated Helm chart documentation with new configuration options
* Updated post-installation guidance
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Adds publishing.proxyProtocol on the platform values: a single flag
that both turns PROXY-protocol on at the host ingress-nginx (via the
ingress-application Package CR's component values dict) and emits the
new cozystack.ouroboros Package CR for the host install. The two
pieces are coupled by design — turning PROXY-protocol on at
ingress-nginx without ouroboros breaks intra-cluster traffic to the
cluster's own public hostnames (kube-proxy / Cilium-KPR short-circuits
the LB IP, the connection arrives at ingress-nginx without the PROXY
header it now requires, and ingress-nginx closes the connection).
The new sources/ouroboros.yaml PackageSource declares the host chart
with dependsOn: cozystack.networking + cozystack.ingress-application
so install ordering is deterministic. The bundle template emits the
host ouroboros Package via the standard helper — the parent chart's
Gateway-API watching default of false stays in place, because cozystack
does not install Gateway-API CRDs cluster-wide on the host (system/
gateway-api-crds is referenced only from the kubernetes-application
PackageSource, tenant scope). Forcing --gateway-api on a host without
the CRDs would CrashLoop the controller on WaitForCacheSync against
missing GVRs.
The flag default is false. Clusters that do not run PROXY-protocol get
zero new resources, zero RBAC delta, zero behaviour change.
Disabling: flipping publishing.proxyProtocol back to false stops
emitting the cozystack.ouroboros Package CR, but every Package CR
carries helm.sh/resource-policy: keep — Helm leaves the existing
Package on the cluster so ouroboros stays installed and the live
Corefile rewrite block keeps pointing at the still-running
ouroboros-proxy Service. Flag-flip alone is idempotent and safe. To
actually uninstall ouroboros the operator removes the Package CR
explicitly (or adds the package name to bundles.disabledPackages) and
runs the documented Corefile cleanup recipe in the same window —
otherwise stale rewrites point at a deleted Service and in-cluster
DNS for hairpinned hostnames returns connection-refused. The values
comment ships the cleanup recipe (a JSON merge-patch on data.Corefile
that strips the BEGIN/END block while preserving labels, annotations,
and other data keys including any Talos-managed metadata) and
documents the upstream-LB precondition (the L4 LB in front of
ingress-nginx must already be injecting PROXY-v1 headers before the
flag flips on), the host-vs-tenant fan-out (the flag does not cascade
into tenant clusters; tenants opt in per-Kubernetes-CR via
addons.ouroboros.enabled), and the Talos re-render flap window
(machine-config pushes wipe the BEGIN/END markers; the controller
re-applies on next reconcile, ≤10 minutes by default).
Also drops the stale "distro-full" entry from the bundles.system.variant
options comment in core/platform/values.yaml — the variant was removed
in v1.0.1 and was never restored.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds the OuroborosAddon typedef, addons.ouroboros knob, and tenant
HelmRelease that deploys the cozy-ouroboros chart inside a tenant
Kubernetes cluster via the host's flux-helm-controller and the tenant
admin-kubeconfig — same shape as the cert-manager addon.
The HelmRelease overrides controller.mode back to coredns-import
because tenants run the cozystack-coredns wrapper (which carries the
`import /etc/coredns/custom/*.override` directive); the parent chart's
default of mode=coredns is the right shape only for the Talos-managed
host CoreDNS. controller.gatewayApi.enabled is mirrored from the
per-tenant addons.gatewayAPI.enabled — forcing --gateway-api on a
tenant without Gateway-API CRDs CrashLoops the controller on
WaitForCacheSync against missing GVRs. dependsOn grows a conditional
gateway-api-crds entry on the same gate.
controller.clusterDomain is intentionally NOT pre-set by the wrapper.
The chart auto-detects it inside the controller pod by parsing
/etc/resolv.conf — the search list of which is `*.<ns>.svc.<domain>`
and reflects exactly the zone the tenant CoreDNS actually serves
(governed by KamajiControlPlane.spec.networkProfile.clusterDomain
defaulting to `cluster.local`, and the cozystack-coredns wrapper
kubernetes-plugin zone, also `cluster.local`). Pre-setting from a
host-side fact like `_cluster.cluster-domain` would override the
auto-detect with a value that does not match what the tenant kube-dns
serves, silently emitting NXDOMAIN-bound rewrites. Operators that run
a tenant on a non-default cluster-domain can still override via
addons.ouroboros.valuesOverride.ouroboros.controller.clusterDomain.
If addons.ouroboros.enabled is true but ingressNginx is off, the
HelmRelease fails the render with a clear error rather than silently
skipping — the hairpin fix has nothing to fix against without
ingress-nginx, and a no-op addon that quietly does nothing is harder
to debug than an explicit render-time error.
The valuesOverride gateway-api guard accepts every truthy spelling
Helm/YAML emit (bool true, string "true"/"True"/"TRUE", numeric 1,
YAML "yes"/"Yes"/"YES"/"on"/"On"/"ON"/"y"/"Y") via
toString-then-lower-then-string-compare; direct bool eq would error
on type mismatch when an operator passes the value as a string from
--set, and case-insensitive folding catches `--set ...=On` /
`--set ...=True` that pure-lowercase comparison would miss.
The acknowledgeUncleanRemoval gate is defence-in-depth for the rare
edge case where helm uninstall fails to land cleanly (tenant
unreachable, hook timeout, manual kubectl delete bypassing helm) —
in the normal path the upstream chart's pre-delete cleanup hook
(coredns-cleanup-hook.yaml, vendored at 0.6.0) nulls the
ouroboros.override key automatically when the HelmRelease is
uninstalled. The gate only blocks render when a leftover HelmRelease
on the host kube-apiserver shows the previous install is still
sitting on the cluster.
The component is also registered in the kubernetes-application
PackageSource (as kubernetes-ouroboros at path system/ouroboros) so
the tenant HelmRelease's ExternalArtifact resolves.
make generate re-emitted README.md, values.schema.json, the Go struct
+ deepcopy, and the kubernetes-rd ApplicationDefinition openAPISchema
in lockstep.
helm-unittest covers: render gating on enabled flag, hard-fail when
ingress-nginx is off, namespaces and labels, chartRef, dependsOn
cilium/coredns/ingress-nginx + conditional gateway-api-crds, default
coredns-import mode, valuesOverride precedence, gateway-api flag
mirroring in both directions, gateway-api truthy-spelling acceptance
(including case-insensitive on/On/True/TRUE), the lookup-driven
acknowledge gate (kubernetesProvider mock asserts the failedTemplate
path when a prior HelmRelease still exists on the host), the
lookup-returns-nil CI/dry-run path (render proceeds when the cluster
mock has no objects), the _namespace.etcd-empty render gate, and
that controller.clusterDomain is NOT pre-set in the rendered
HelmRelease (so chart auto-detect runs).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Extends the cozystack-coredns wrapper with the standard CoreDNS
`import /etc/coredns/custom/*.override` directive in the .:53 server
block and renders an empty kube-system/coredns-custom ConfigMap as
a chart-managed prerequisite, mounted into the CoreDNS Deployment
at /etc/coredns/custom/ via extraVolumes / extraVolumeMounts. The
ConfigMap carries helm.sh/resource-policy: keep so the chart-managed
empty value cannot strip what later writers add into it.
This exposes a stable extension point on the cozystack CoreDNS for
operators (or other cozystack components) that need to add plugin-only
rewrite snippets without touching the chart-rendered Corefile —
otherwise the chart would re-template data.Corefile on every Flux
reconcile and overwrite the additions.
The same package serves both host and tenant CoreDNS deployments, so
this plumbing covers both layers without per-layer duplication. The
volume mount is `optional: true` for defence-in-depth: a future chart
upgrade that drops the templates/coredns-custom.yaml render still
leaves CoreDNS healthy with no override snippets in play.
helm-unittest fixtures pin the ConfigMap shape (name, namespace, keep
annotation, no data), the `import` directive in the rendered Corefile,
the presence of the `reload` plugin (required for live ConfigMap
updates), and the volume + volumeMount on the CoreDNS Deployment.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
New packages/system/ouroboros/ vendors lexfrei/ouroboros 0.6.0 from
oci://ghcr.io/lexfrei/charts/ouroboros and pins cozystack-side defaults
for a hairpin-NAT fix on top of ingress-nginx + PROXY-protocol.
The controller watches Ingress (and optionally Gateway / HTTPRoute)
for public hostnames and rewrites cluster-internal DNS so those
hostnames resolve to a small in-cluster TCP proxy that prepends the
PROXY-protocol v1 header before forwarding to ingress-nginx — without
that, intra-cluster traffic to the cluster's own public hostnames
arrives at ingress-nginx without the PROXY header it requires and
ingress-nginx closes the connection.
cozystack pins:
* controller.mode=coredns by default (mutates the live
kube-system/coredns Corefile in place between BEGIN/END markers).
The host CoreDNS on cozystack is Talos-managed and does not carry
an `import /etc/coredns/custom/*.override` directive, so the
alternative coredns-import mode would silently no-op there.
Tenant deployments override this back to coredns-import via the
apps/kubernetes wrapper (separate change).
* proxy.target.host=ingress-nginx-controller.cozy-ingress-nginx.svc.cluster.local —
cozystack deploys ingress-nginx into cozy-ingress-nginx in both
the host control plane and tenant clusters; the chart-default
`ingress-nginx` namespace exists nowhere in cozystack and would
silently dial a non-existent Service.
* controller.gatewayApi.enabled defaults to upstream's false at the
parent level. The bundle template (separate change) flips it on
for the host scope; the tenant wrapper mirrors per-tenant
addons.gatewayAPI.enabled. Forcing --gateway-api on a tenant
without Gateway-API CRDs CrashLoops the controller on
WaitForCacheSync.
* image is digest-pinned at
ghcr.io/lexfrei/ouroboros:0.6.0@sha256:b33c415cd2edf231eb62076297fee1b75657bef2bfcc0be916b84088aa49479d and pulled
directly from the upstream registry; this package is intentionally
not mirrored under ghcr.io/cozystack/*. Air-gapped operators
mirror both the chart and the image separately — bump together
with OUROBOROS_CHART_VERSION in the Makefile.
The vendored chart, helm-unittest fixtures (mode dispatch, RBAC
narrowing per mode, Gateway-API gating, target FQDN), and chart
layout follow the same shape as packages/system/cilium and
packages/system/metallb.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Changes the default `resourcesPreset` for both `kafka` and `zookeeper`
in the Kafka managed application from `small` (1 CPU / 512Mi) to
`medium` (1 CPU / 1Gi).
The `small` preset is no longer sufficient for the current Strimzi 0.45
/ Kafka 3.9 image baseline:
- Strimzi sets JVM heap to 50% of the container memory limit for Kafka
and 75% for ZooKeeper. With a 512Mi limit that leaves only 256Mi (Kafka)
or 128Mi (ZooKeeper) for the JVM non-heap.
- Actual non-heap usage in the Kafka 3.9 image (Metaspace, code cache,
direct buffers, JMX javaagent, embedded Jetty admin server for
`/v1/ready` and KRaft migration endpoints, thread stacks) systematically
exceeds those budgets at startup, triggering a cgroup OOM kill (exit
137) before the broker can connect to ZooKeeper.
Result on a fresh deployment with the previous default: every Kafka
broker and every ZooKeeper pod ends up in `CrashLoopBackOff`, with
`lastState.terminated.reason: OOMKilled`. ZooKeeper occasionally hits
`OOMKilled` once quorum traffic ramps up, breaking the quorum and
causing Kafka brokers to fail with `ZooKeeperClientTimeoutException`.
The `medium` preset (1Gi) is the smallest preset that gives both Kafka
(heap 512Mi + ~500Mi non-heap budget) and ZooKeeper (heap 768Mi + 256Mi
non-heap budget) enough headroom to start cleanly. Users who need more
can still pick `large`, `xlarge`, or `2xlarge`, or override `resources`
explicitly.
The `small` preset is intentionally kept in the enum so existing
deployments that pin `resourcesPreset: small` are not broken; only the
default changes.
### Release note
```release-note
fix(kafka): default `resourcesPreset` for both Kafka and ZooKeeper is now `medium` (1 CPU / 1Gi). The previous default (`small`, 512Mi) is no longer sufficient under the current Strimzi 0.45 / Kafka 3.9 image and caused OOMKilled CrashLoopBackOff on fresh deployments. Existing deployments that explicitly set `resourcesPreset` are unaffected.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated default resource allocation preset from "small" to "medium"
for Kafka and ZooKeeper components across CRD type definitions, Helm
chart values, and JSON schema configurations.
* **Documentation**
* Updated parameter reference tables to reflect the new default resource
preset values for both Kafka and ZooKeeper.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
IvanHunters
