mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-03 18:47:04 +09:00
ci/build-cache-mode-max
4098
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Myasnikov Daniil
|
e1781ac407 |
ci(build): route flux-plunger and kubeovn-plunger through $(BUILDER)
These two recipes spell out their buildx flags instead of using $(BUILDX_ARGS), so they never received --builder and ran on the default docker driver. After the mode=max migration that became a hard failure -- "Cache export is not supported for the docker driver" -- since the embedded docker driver cannot export a registry cache. Add the same conditional --builder flag $(BUILDX_ARGS) carries, so WRITE_CACHE=1 builds route to the docker-container builder; an empty BUILDER still emits no flag for local builds. Part of #2937. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
6d04ba5b3d |
ci(build): warm the main cache on an isolated docker-container builder
`--cache-to type=registry,mode=max` is unsupported on the host's default embedded docker driver unless the daemon runs the containerd image store, so on a classic store the warmer would fail to export any cache. Build the warmer on a docker-container buildx builder, which supports mode=max regardless of the host image store. It also gives the warmer its own buildkit (separate bbolt), so it never contends with PR builds on the shared embedded buildkit. Part of #2937. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
a576f33461 |
refactor(build): mode=max registry cache with a main-only warmer
`--cache-to type=inline` only exports the final stage. 25 of 38 Dockerfiles are multistage, so the expensive `builder` stages (Go/Node compiles) were never cached even with a warm tag. Switch every image to a shared mode=max registry cache. - New `cache-args` macro (hack/common-envs.mk) emits `--cache-from` always and `--cache-to … mode=max` only when WRITE_CACHE=1, so PR builds stay read-only and concurrent PRs never race on the cache ref (the 409 class #2711 fixed for tags). oci-mediatypes/image-manifest keep the cache manifest portable across registries. - Cache ref is CACHE_REGISTRY/<img>:buildcache, co-located with the build registry (CACHE_REGISTRY now defaults to $(REGISTRY)). - New build-main.yaml warms :buildcache on every push to main (WRITE_CACHE=1), serialized via concurrency. All 31 package image recipes converted from the inline cache-from/cache-to pair to `$(call cache-args,…)`; ubuntu-container-disk keeps a per-k8s-version cache tag. Verified with `make -n image` across all packages in both PR (read-only) and main (mode=max write) contexts. Part of #2937. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
b825c31eca |
test(e2e): self-heal Cilium orphaned-endpoint leak across install and apps
Run the Cilium endpoint-leak watchdog as an in-cluster Job instead of a background process started in the install bats setup_file. bats *_file hooks fire per file, so the old process was killed at the install file's teardown_file — before any app test ran — and the leak recurred unhealed in the app phase (e.g. the tenant-Kubernetes CVMI importer). The Job is bound to the cluster, not to one bats file, so it covers install and the whole app suite. The heal logic stays in hack/e2e-cilium-endpoint-leak-healer.sh (single source of truth) and is shipped to the pod via a ConfigMap built from it at apply time. The pod uses hostNetwork so it needs no Cilium endpoint and cannot be wedged by the very leak it repairs, with broad tolerations so it schedules before nodes are Ready. RBAC is scoped to read events/pods and exec into the cilium-agent. Image reuses the in-tree digest-pinned alpine/k8s already shipped by the installer. Interim CI band-aid for cilium/cilium#38313; remove with a fixed Cilium. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Ivan Okhotnikov
|
e69d2682a0 |
fix(seaweedfs): drop volumeSizeLimitMB so multiple collections fit
Cozystack overrides the upstream seaweedfs volumeSizeLimitMB from 1000 to 30000 (30 GB), but the e2e sandbox provisions only a 10 GiB PVC per volume server. With maxVolumes=0 (auto), seaweedfs computes maxVolumes as int(10240 / 30000) → 0, then rounds up to 1, so each volume server can host exactly 1 logical volume. With defaultReplication=001 (one replica on a different node within the same rack) two volume servers share that single volume, giving the cluster a hard limit of one writable collection at a time. Since #2716 added a default bucket-cozy-backups Bucket to every tenant-root, that bucket now grabs the only slot on install. Any second collection — the bucket E2E (bucket-test in tenant-test), a tenant app's own bucket, etc. — sits in 'assign volume: No writable volumes' forever, and mc cp hangs until the test times out. Master logs reflect this clearly: failed to find writable volumes for collection:bucket-... replication:001 ttl: error: No writable volumes and no free volumes left topo failed to pick 1 from 0 node candidates volume_growth.go:135 create 6 volume, created 0: Not enough data nodes found! Lower volumeSizeLimitMB back to the upstream chart default of 1000, which makes a 10 GiB PVC fit ~10 logical volumes per server and unblocks all per-collection bucket workloads on small sandboxes. Production clusters still get plenty of headroom — volume size is a soft cap, not a preallocation. Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
562239d90c |
fix(seaweedfs): restore s3 Service name to chart-prefix form
The seaweedfs 4.31 upstream bump (#2834) switched the s3 Service name from "$chartname-s3" (built with the seaweedfs.name helper, evaluates to "seaweedfs-s3") to the seaweedfs.componentName helper, which is release-name based and now evaluates to "$release-s3" — "seaweedfs- system-s3" for the tenant-root deployment. The bats E2E asset hack/e2e-apps/bucket.bats hard-codes "service/seaweedfs-s3" for the port-forward that sets up its S3 client; backup-controller and other in-cluster consumers also referenced the legacy name. After the bump, the legacy Service is gone and the port-forward immediately exits, the test then times out on the 30-second nc poll for localhost:8333. Restore the legacy Service name. Internal references inside the chart (s3-deployment -filer=, etc) keep using componentName for the filer endpoint, so this change only re-exposes the public-facing name without touching the internal topology. Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
f8c83bfd2f |
fix(seaweedfs): restore -lock BucketClass and -readonly BucketAccessClass
The seaweedfs 4.31 upstream bump (#2834) rewrote packages/system/seaweedfs/charts/seaweedfs/templates/cosi/cosi-bucket-class.yaml from scratch and silently dropped the supplementary COSI resources that cozystack ships on top of the upstream chart: - BucketClass {name}-lock (object lock, COMPLIANCE mode, 365 days retention) — used by apps that store immutable backups - BucketAccessClass {name} parameters.accessPolicy: readwrite — the explicit grant for the default access class - BucketAccessClass {name}-readonly (accessPolicy: readonly) — used by every Bucket app that declares readonly users apps/bucket still renders BucketAccess resources that reference {name}-readonly, but the class no longer exists, so the COSI controller never picks the BucketAccess up: no finalizer is added, status.accessGranted stays empty, and the bucket E2E test waits 300s and times out on the viewer access. The admin (readwrite) access class still works because it falls back to the default BAC. Restore the three resources so the system chart again matches the shape that packages/extra/seaweedfs/templates/client/cosi-bucket-class.yaml already ships for the client topology (where this never regressed). Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
d937a91a9a |
test(e2e): drop bash-only RETURN trap from bucket.bats port-forward
The previous attempt added a bash `trap '...' RETURN` to clean up kubectl port-forward when the test function returned. cozytest.sh sources its converted bats test bodies into the runner shell, which on the GitHub-hosted Linux runner is /bin/sh -> dash. dash does not implement the bash-specific RETURN pseudo-signal, so the trap command exits non-zero, the surrounding `set -eu` aborts the test body immediately, and the captured output stops two lines after `name=test` with no other diagnostic — the test never reaches the mc S3 operations at all. Drop the trap and rely on cozytest.sh's existing process-group cleanup: when the test function returns, the leftover port-forward job in the same group is terminated. This restores POSIX shell compatibility while keeping the lifecycle fix from the previous commit (no `bash -c '... &'` wrapper, port-forward stays alive through the mc operations because it is a child of the calling shell, not of a transient subshell that exits immediately). Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
862b19ffb6 |
test(e2e): keep bucket.bats port-forward alive across mc operations
The test used to wrap kubectl port-forward in a transient subshell: bash -c 'timeout 100s kubectl port-forward ... > /dev/null 2>&1 &' The wrapper subshell exits immediately after backgrounding the kubectl process, the kubectl child is reparented to PID 1, and on the GitHub-hosted runner it gets reaped between the `nc -z` readiness probe and the first `mc cp` call. The captured failure looks like + bash -c timeout 100s kubectl port-forward ... + timeout 30 sh -ec until nc -z localhost 8333; do sleep 1; done + mc alias set rw-user https://127.0.0.1:8333 ... --insecure Added `rw-user` successfully. + mc cp --insecure /tmp/rw-test.txt rw-user/$BUCKET/rw-test.txt mc: <ERROR> Put "https://127.0.0.1:8333/$BUCKET/...": dial tcp 127.0.0.1:8333: connect: connection refused even though every S3 resource (Bucket, BucketClaim, both BucketAccess) is Ready/Granted=true at that point and the seaweedfs-s3 Service is healthy in tenant-root. Run kubectl port-forward as a job of the calling shell, capture its PID, and tear it down via a bats RETURN trap. The trap is enough on its own: the test no longer needs the outer `timeout 100s` belt because the port-forward will be killed when the test function returns, even on failure. Also switch the readiness probe to 127.0.0.1 to match the mc alias and avoid any IPv6 resolution path on hosts where kubectl binds IPv4 only. Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
18f47f12e8 |
test(e2e): bind bucket.bats S3 client to 127.0.0.1 instead of localhost
`kubectl port-forward service/seaweedfs-s3 -n tenant-root 8333:8333` binds the listener to 127.0.0.1 only — it has no IPv6 listener. The runner's getaddrinfo resolves `localhost` to `::1` first, and `mc` keeps a single connection target across the whole alias, so the initial `mc alias set` (which only sets credentials, no network I/O) silently succeeds while the subsequent `mc cp` upload tries `https://[::1]:8333` and gets immediate `connection refused`. The test then exits with the misleading bucket-test failure even though the S3 backend, credentials, and bucket all reconciled correctly. Use a literal IPv4 address (`127.0.0.1`) for the mc aliases and the `nc -z` readiness probe so the test no longer depends on the runner's IPv4/IPv6 resolution policy. Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Myasnikov DaniilandClaude Sonnet 4.6
|
4b9b1b4f96 |
fix(capi): add startupProbe to capi-controller-manager to survive cert provisioning delay
The capi-controller-manager liveness probe has no initialDelaySeconds (defaults to 0), giving the container only 30s (3×10s) to serve /healthz. In controller-runtime, the health endpoint starts after the webhook server, which needs its TLS cert from cert-manager. During a loaded install, cert-manager is busy issuing many certs simultaneously and the CAPI webhook cert can take >30s — causing a kill loop (6+ restarts observed in CI). A startupProbe with failureThreshold=30 gives up to 300s for the initial startup, after which the regular livenessProbe takes over for ongoing health monitoring. This is the standard pattern for slow-starting containers with external dependencies. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
8c411d801f |
fix(deps): bump Go to 1.26.4 and x/net to v0.55.0 to close OSV advisories (#2852)
## What this PR does Clears the OpenSSF Scorecard **Vulnerabilities** check (was **0/10**). Scorecard queries OSV against the dependency graph, so reproducing it with `osv-scanner` over both `go.mod` files surfaced **12 advisories**: | Advisory | Package | Fixed in | |---|---|---| | GO-2026-4918, 4971, 4977, 4980, 4982, 4986 | stdlib (Go toolchain) | 1.26.3 | | GO-2026-5037, 5038, 5039 | stdlib (Go toolchain) | 1.26.4 | | GO-2026-4918, GO-2026-5026 | `golang.org/x/net` | 0.55.0 | **10 of 12 are stdlib advisories** against the Go toolchain pinned at `1.26.2` — so the dominant fix is raising the Go directive, not churning dependencies. ### Changes - **Go directive `1.26.2 → 1.26.4`** in the root module and the `api/apps/v1alpha1` submodule (closes all 10 stdlib advisories; 1.26.4 is required for GO-2026-5037/5038/5039). - **`golang.org/x/net 0.52.0 → 0.55.0`** (closes GO-2026-4918, GO-2026-5026). `go mod tidy` carried the matching minor bumps to `x/crypto`, `x/sys`, `x/term`, `x/text`, `x/tools` via MVS. All version changes were made with `go mod edit` / `go get` / `go mod tidy` — the generated `go.mod` was not hand-edited. ### Verification - `osv-scanner` after the bump: **Total 0 packages affected by 0 known vulnerabilities.** - `go build ./...` passes on both modules (toolchain auto-upgraded to go1.26.4). - `make generate` produces **no drift** — `codegen-drift` CI stays green. > **Maintenance note:** the Renovate config (separate PR) deliberately does **not** auto-bump the Go toolchain directive, so future stdlib advisories (1.26.5+) will need a manual bump like this one. If you'd prefer Renovate to keep the Go *patch* version current automatically, that rule can be enabled — flagged for a decision. ### Release note ```release-note fix(deps): bump Go to 1.26.4 and golang.org/x/net to v0.55.0, closing 12 known OSV advisories (10 stdlib, 2 in x/net) ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated the Go toolchain to the latest 1.26.x release for improved build reliability and runtime compatibility. * Bumped several indirect Go module dependencies (networking, crypto, system, terminal, text, and tooling libraries) to newer stable versions to enhance stability and security. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
39f3ac72c5 |
fix(kubernetes): add spec.timeout to tenant CSI HelmRelease (#2727)
## What this PR does
Adds `spec.timeout: 10m` to the `kubevirt-csi-node` HelmRelease template
in `packages/apps/kubernetes/templates/helmreleases/csi.yaml`, aligning
it with every other tenant addon HR in the same directory (cilium,
coredns, ingress-nginx, etc. — all set `timeout: 10m`). `csi.yaml` was
the only one missing it; commit
|
||
|
Aleksei Sviridkin
|
8a23f84464 |
fix(prometheus-operator-crds): ship full upstream CRD bundle (#2660)
## Summary - Drop the `rm -f` line in `packages/system/prometheus-operator-crds/Makefile` so the full upstream CRD bundle (10 CRDs) ships instead of the 4-CRD subset. - Pin `helm pull --version 25.0.0` to keep the operator CRD version at `v0.87.0` (operator version bump is intentionally out of scope here). - Re-run `make update` to repopulate `charts/crds/templates/` with the 6 previously-stripped CRDs: `Alertmanager`, `AlertmanagerConfig`, `Prometheus`, `PrometheusAgent`, `ScrapeConfig`, `ThanosRuler`. The 4 already-vendored CRDs are byte-identical (version pin holds). ## Why The bundle was originally trimmed to the 4 service-discovery CRDs that VictoriaMetrics operator needs (`ServiceMonitor`, `PodMonitor`, `PrometheusRule`, `Probe`). That was correct when the chart was vendored inside `system/victoria-metrics-operator` (`ba5c785cc`, Dec 2023), but the prune was carried over verbatim when the chart was extracted to its own package, and again when the CRD release was propagated into tenant Kubernetes clusters (`1e2b66131`, Dec 2025). In tenant clusters the partial set blocks third-party apps that ship their own `prometheus-operator` (e.g. `kube-prometheus-stack`): they cannot deploy `Prometheus`/`Alertmanager` instances because the platform-managed CRD set is incomplete, and `--server-side --force-conflicts` workarounds get reverted by Flux as it re-reconciles the narrower release manifest. With the full bundle shipping, tenant apps can set `crds.enabled: false` on their `kube-prometheus-stack` release and consume the platform CRDs directly — the standard pattern for platform-level CRDs shared between multiple stacks. ## Test plan - [x] `helm template ./charts/prometheus-operator-crds` renders 10 `CustomResourceDefinition` resources, all in `monitoring.coreos.com` API group. - [x] `helm template packages/apps/kubernetes -f tests/values/common.yaml` renders cleanly (75 kinds). - [x] `helm unittest packages/apps/kubernetes` — 6 suites, 123 tests, all pass. - [x] No in-repo consumer hardcodes the 4-CRD list (verified by grep). - [ ] Smoke-test on a tenant cluster: confirm `kube-prometheus-stack` with `crds.enabled: false` installs cleanly and the operator can manage `Prometheus`/`Alertmanager` instances against the platform CRDs. ## Out of scope - Bumping the operator CRD version past `v0.87.0` — separate concern. - Switching from `HelmRelease` to a Flux `Kustomization` for CRD application — bigger refactor; current Helm ownership is fine now that the bundle is complete and downstream apps can opt out of installing CRDs. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Pinned the prometheus-operator-crds Helm chart pull to version `25.0.0` for consistent regeneration. * Streamlined the update workflow by removing the existing generated charts directory before pulling the chart, improving build cleanliness. * Removed the extra post-pull cleanup step that previously deleted selected generated CRD templates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
97cf5857a3 |
fix(prometheus-operator-crds): clean chart dir before helm pull
Add `rm -rf charts/prometheus-operator-crds` before `helm pull` in the update target so stale files from previous chart versions don't linger across updates. This matches the convention used by cilium, external-secrets-operator, external-dns, and cert-manager Makefiles. Address review feedback from gemini-code-assist on packages/system/prometheus-operator-crds/Makefile:9. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
dbbaca16eb |
fix(prometheus-operator-crds): ship full upstream CRD bundle
The chart previously stripped the bundle to 4 CRDs (ServiceMonitor, PodMonitor, PrometheusRule, Probe) — the set VictoriaMetrics operator needs for service discovery. This was correct when the chart was vendored inside system/victoria-metrics-operator (commit |
||
|
Myasnikov Daniil
|
d8ba954e90 |
fix(kubernetes): add spec.timeout to tenant CSI HelmRelease
Aligns kubevirt-csi-node HR with every other tenant addon HR in packages/apps/kubernetes/templates/helmreleases/ (cilium, coredns, ingress-nginx, etc.) — all set timeout: 10m, csi.yaml was the only exception. |
||
|
Aleksei Sviridkin
|
4996b9d8ef |
fix(api): publish open spec as x-kubernetes-preserve-unknown-fields, not additionalProperties:true (#2867)
## What this PR does `cozystack-api` published the free-form `.spec` of aggregated `apps.cozystack.io` resources (Tenant, Application, …) as the boolean `additionalProperties: true`. The Kubernetes ValidatingAdmissionPolicy **status type-checker** run by `kube-controller-manager` nil-dereferences when it recurses into that node (`k8s.io/apiserver/pkg/cel/openapi.isExtension`). As soon as a VAP matches one of these resources — `cozystack-tenant-host-policy` matches `tenants` — KCM `CrashLoopBackOff`s on every control-plane node, stalls reconciliation, and times out the install (`kubectl wait hr --all` never completes). With the 3× install retry removed, a single occurrence fails E2E outright. This emits `x-kubernetes-preserve-unknown-fields: true` instead, in `pkg/cmd/server/openapi.go::patchSpec` (both the schemaless and custom-schema branches, covering the OpenAPI v2 and v3 post-processors). It is semantically equivalent — "arbitrary fields allowed" — and the type-checker handles it safely. All managed-app custom schemas already use the safe **object-form** `additionalProperties`, so `patchSpec` was the only source of the boolean form. ### Regression test `TestPatchedSpecDoesNotPanicVAPTypeChecker` reproduces the exact crash through `k8s.io/apiserver/pkg/cel/openapi.SchemaDeclType` — the entry point KCM's `validatingadmissionpolicystatus` controller uses. It **panics on the pre-fix code and passes after the fix**. Companion structural tests assert the published spec carries `x-kubernetes-preserve-unknown-fields: true` and never the boolean `additionalProperties: true`, and that genuine object-form maps are left untouched. Verified against the pristine vendored `k8s.io/apiserver@v0.34.1` (the version cozystack-api builds against), which reproduces the panic, and previously confirmed end-to-end on `kindest/node:v1.33.1`. ### Upstream The underlying nil-dereference is a Kubernetes KCM bug — the VAP status type-checker should treat `additionalProperties: true` as dynamic rather than panic. A separate upstream report is being prepared; cozystack fixes the symptom immediately with the schema change above. Fixes #2863 ### Release note ```release-note fix(api): publish the free-form resource `.spec` as `x-kubernetes-preserve-unknown-fields` instead of `additionalProperties: true`, fixing a cluster-wide `kube-controller-manager` CrashLoopBackOff (nil-pointer panic in the ValidatingAdmissionPolicy status type-checker) that stalled installs. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Ensured published OpenAPI schemas no longer contain boolean-form additionalProperties anywhere; top-level and nested schemas now use preserve-unknown-fields for free-form specs, preventing validation/runtime issues. * **Tests** * Added regression tests that feed varied untrusted schemas and verify boolean additionalProperties are removed and published schemas remain valid for tooling. <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.5.0-rc.3 api/apps/v1alpha1/v1.5.0-rc.3 |
||
|
Aleksei Sviridkin
|
547500ffcd |
fix(seaweedfs): restore S3 TLS and COSI provisioner SA after 4.31 bump (#2916)
## What this PR does The SeaweedFS 4.31 chart bump (#2834) broke E2E for **every** PR on `main`: in `tenant-root` the `seaweedfs-system` release never becomes ready, so `install-cozystack` times out waiting on its HelmRelease. Two regressions came in with the new chart; both reproduce with a plain `helm template` of `packages/system/seaweedfs` (no cluster needed) and are fixed here as vendored patches re-applied by `make update`, with helm-unittest coverage. **1. S3 served plaintext while everything expected HTTPS.** Until chart 4.05 the `-cert.file`/`-key.file` args were rendered whenever `enableSecurity` was set, so `weed s3` served TLS on the main port (8333). Chart 4.31 nested those args inside `{{- if .Values.s3.httpsPort }}`, and we ship `httpsPort: 0` — so no TLS args rendered, s3 spoke HTTP, and the HTTPS readiness/liveness probes (plus the ingress `backend-protocol: HTTPS` and the COSI sidecar's `https://` endpoint) all failed with `server gave HTTP response to HTTPS client`. `patches/s3-tls-main-port.patch` moves the cert/key args back out of the `httpsPort` gate, restoring TLS-on-main-port; `-port.https` is still only added when `httpsPort` is set. **2. COSI provisioner ServiceAccount name mismatch.** `cosi-service-account.yaml` and the `ClusterRoleBinding` name the SA `<global.seaweedfs.serviceAccountName>-objectstorage-provisioner`, but the Deployment referenced `componentName` (`<release>-objectstorage-provisioner`). Whenever `serviceAccountName` != release name — our case, `tenant-root-seaweedfs` vs `seaweedfs-system` — the Deployment could not create pods (`serviceaccount "seaweedfs-system-objectstorage-provisioner" not found`). `patches/cosi-provisioner-sa-name.patch` aligns the Deployment with the SA and the binding. (Upstream installs only avoid this because their release name and serviceAccountName usually coincide.) This is `main`-only: the 4.31 chart landed only on `main`, every release branch is still on 4.05, so no `backport` is needed. ### Release note ```release-note fix(seaweedfs): restore S3 TLS on the main port and fix the COSI provisioner ServiceAccount name after the 4.31 chart bump, so tenant SeaweedFS becomes ready again ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed S3 TLS certificate arguments to render correctly when HTTPS port is disabled. * Updated COSI provisioner to use configurable service account naming. * **Tests** * Added regression tests for S3 TLS configuration and COSI service account handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
baab38a4f0 | Merge branch 'main' into fix/seaweedfs-e2e-tls-cosi | ||
|
Aleksei Sviridkin
|
9d43e42b0f |
test(e2e): pin Kubernetes to v1.33.12 to avoid KCM VAP type-checker panic (#2868)
## What this PR does The e2e cluster bootstrap (`hack/e2e-prepare-cluster.bats`) let `talosctl gen config` pick its bundled default Kubernetes version — **v1.33.1** for Talos v1.13.0. v1.33.1 predates the fix for the kube-controller-manager `ValidatingAdmissionPolicy` status type-checker nil-pointer panic on `additionalProperties: true` schemas ([kubernetes/kubernetes#135155](https://github.com/kubernetes/kubernetes/pull/135155), backported to `release-1.33` in [#136958](https://github.com/kubernetes/kubernetes/pull/136958), first released in **v1.33.10**). Because `cozystack-api` publishes `additionalProperties: true` on the aggregated Tenant schema, KCM crash-loops cluster-wide during install on the old default and fails E2E (#2863). This pins the e2e **management** cluster to **v1.33.12** (latest 1.33 patch), which contains the fix. It complements — and is independent of — the schema-side fix in #2867; either alone avoids the panic. Refs: #2863 ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated end-to-end test cluster configuration to explicitly pin the Kubernetes version for more consistent test execution. * Added explanatory comments clarifying the version choice and kept existing secret/configuration wiring intact. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei KvapilandClaude
|
484d73bf78 |
fix(seaweedfs): restore S3 TLS and COSI provisioner SA after 4.31 bump
The 4.31 chart bump (#2834) broke E2E for every PR: tenant-root seaweedfs-system never becomes ready. 1. S3 served plaintext. Until 4.05 the -cert.file/-key.file args were rendered whenever enableSecurity was set, so weed s3 served TLS on the main port (8333); 4.31 nested them under 'if s3.httpsPort' and we ship httpsPort=0, so s3 spoke HTTP while the HTTPS probes, the ingress backend-protocol and the COSI sidecar endpoint all expected HTTPS ('server gave HTTP response to HTTPS client'). Restore the 4.05 behaviour via patch. 2. COSI provisioner ServiceAccount mismatch. cosi-service-account.yaml and the ClusterRoleBinding name the SA <serviceAccountName>-objectstorage-provisioner, but the Deployment referenced componentName (<release>-objectstorage-provisioner). When serviceAccountName != release name (tenant-root-seaweedfs vs seaweedfs-system) the Deployment could not create pods ('serviceaccount not found'). Align the Deployment with the SA + binding. Both fixes are vendored as patches re-applied by 'make update', and covered by helm-unittest. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
42ca3e6bd3 | test(api): add read-path authorization tests for TenantNamespace registry (#2864) | ||
|
myasnikovdaniil
|
25a6b56fab |
docs(release): add changelog for v1.4.3 (#2841)
This PR adds the changelog for release `v1.4.3`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.4.3.md`.
|
||
|
cozystack-ci[bot]
|
1b7e74fb87 |
docs: add changelog for v1.4.3
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com> |
||
|
Myasnikov Daniil
|
379bc5843b |
refactor(api): initialize schema extensions with spec.Extensions{}
Address review feedback from gemini-code-assist on pkg/cmd/server/openapi.go:
markOpenObject and sanitizeForV2 initialized s.Extensions with the generic
map[string]any{}; use the named spec.Extensions{} type instead, matching the
kube-openapi types and the generated definitions in zz_generated.openapi.go.
No behavioral change — spec.Extensions is defined as map[string]interface{}.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
b1b84bc03c |
test(api): cover nested-in-additionalProperties and anyOf/oneOf/not in sanitizer regression
Address review feedback from lexfrei and coderabbitai on pkg/cmd/server/openapi_test.go: TestPatchSpecSanitizesUserSuppliedBooleanAdditionalProperties only exercised the top-level, properties, items, and allOf positions, leaving the one remaining type-checker-reachable sanitizer branch untested: a boolean additionalProperties nested inside an object-form additionalProperties. Deleting that recursion line kept the whole suite green while the input panicked SchemaDeclType again (#2863). Add the nested-under-additionalproperties case (the reachable blocker, verified red->green by mutation) plus anyOf/oneOf/not cases (defense-in-depth: those branches are recursed by the sanitizer but never traversed by SchemaDeclType, so the hasBooleanAdditionalProperties assertion guards them). Rewrite the test docstring to distinguish the type-checker-reachable cases from the defense-in-depth ones, replacing the inaccurate "each case panics SchemaDeclType" note. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
d739663298 |
fix(deps): bump golang.org/x/net to v0.55.0 in api/apps/v1alpha1 submodule
The submodule still pulled golang.org/x/net v0.53.0, affected by GO-2026-5026 (failure to reject ASCII-only Punycode-encoded labels in golang.org/x/net/idna, which the submodule imports). The root module was already on v0.55.0; align the submodule so the advisory is closed across both go.mod files. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
8497fc6f6f |
fix(deps): bump Go to 1.26.4 and x/net to v0.55.0 to close OSV advisories
Clears the OpenSSF Scorecard Vulnerabilities check (was 0/10). An osv-scanner pass over both go.mod files reported 12 advisories: - 10 stdlib advisories against the Go toolchain pinned at 1.26.2 (GO-2026-4918/4971/4977/4980/4982/4986 fixed in 1.26.3; GO-2026-5037/5038/5039 fixed in 1.26.4) — closed by raising the go directive to 1.26.4 in both the root module and api/apps/v1alpha1. - golang.org/x/net 0.52.0 (GO-2026-4918, GO-2026-5026) — bumped to v0.55.0; go mod tidy carried the matching x/crypto, x/sys, x/term, x/text and x/tools minor bumps via MVS. After the bump osv-scanner reports 0 advisories. Both modules build, and 'make generate' produces no drift (codegen-drift stays green). Versions were changed with 'go mod edit'/'go get'/'go mod tidy', not by hand-editing the generated go.mod. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-By: Claude <noreply@anthropic.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
96b9f5eac6 |
test(e2e): pin Kubernetes to v1.33.12 to avoid KCM VAP type-checker panic
The e2e cluster bootstrap let `talosctl gen config` pick its bundled default Kubernetes version (v1.33.1 for Talos v1.13.0). That version predates the fix for the kube-controller-manager ValidatingAdmissionPolicy status type-checker nil-pointer panic on `additionalProperties: true` schemas (kubernetes/kubernetes#135155, backported to release-1.33 in #136958, first released in v1.33.10). cozystack-api's aggregated Tenant schema publishes `additionalProperties: true`, so on the older default KCM crash-loops cluster-wide during install and fails E2E. Pin the e2e management cluster to v1.33.12 (latest 1.33 patch) so it runs a Kubernetes that contains the fix. This complements the schema-side fix in #2867 and is independent of it. Refs: #2863 Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
244de9d855 |
fix(api): recursively sanitize boolean additionalProperties in published spec
Addresses review feedback on #2863. The previous fix only converted the open spec wrapper cozystack-api injects, so a user-supplied ApplicationDefinition openAPISchema carrying a boolean additionalProperties — declared explicitly at the top level, or nested under properties/items/allOf/anyOf/oneOf — was republished verbatim and re-triggered the kube-controller-manager VAP type-checker crash. openAPISchema is untrusted input (external-apps and operator-authored definitions flow through unvalidated), so close the whole class rather than only the schemas cozystack ships. Recursively rewrite every boolean-form additionalProperties (nil inner schema): additionalProperties:true becomes x-kubernetes-preserve-unknown-fields:true, additionalProperties:false is dropped. Both JSON forms carry a nil inner schema and crash the type-checker identically; object-form maps are recursed into and left intact. Also publish the schemaless spec from a fresh schema so no inherited $ref or properties leak into the open object. Regression tests cover top-level true/false and nested (properties, items, allOf) boolean forms, asserting the published schema carries no boolean additionalProperties node and does not panic SchemaDeclType. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
8348f96b14 |
chore(seaweedfs): bump SeaweedFS to 4.31 (#2834)
## What this PR does Re-vendors the SeaweedFS Helm chart from 4.05 (chart 4.0.405) to 4.31 (chart 4.31.0) via `make update`, and adapts the package layer to the breaking changes that accumulated across the gap. What lands with the bump: the PostgreSQL filer store (the backend used here) moves to pgx v5 with PgBouncer support and an `ON CONFLICT` upsert default; HTTPS certificates hot-reload without a pod restart on cert-manager rotation; more S3 and volume Prometheus metrics are exported, and the S3 Service now exposes a metrics port by default. Package-layer adaptations: - **Dropped `s3-traffic-distribution.patch`.** Native `s3.trafficDistribution` support landed upstream in chart 4.12. The native field is gated to Kubernetes >= 1.31 and auto-converts `PreferClose` to `PreferSameZone` on >= 1.35, which the hardcoded patch did not do. The patch is replaced with `s3.trafficDistribution: PreferClose` in the package values, plus helm-unittest coverage pinning all three version cases. - **Migrated `global.*` to `global.seaweedfs.*`.** Chart 4.18 namespaced the app-specific globals to avoid polluting the shared global namespace when the chart is used as a subchart. The package values move `enableSecurity`, `serviceAccountName`, `extraEnvironmentVars` and `monitoring` to the new path instead of relying on the chart's backward-compat shim. Verified with `helm template` and `helm unittest`: the cluster env vars reach the pods, the gRPC mTLS `security.toml` renders intact (including the new admin/worker sections from the 4.24/4.26 security tightening), and both retained patches still apply. ### Screenshots N/A — no UI changes. ### Release note ```release-note chore(seaweedfs): bump SeaweedFS to 4.31 ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * OpenShift support and dedicated values for secure deployments * S3 Iceberg gateway with optional ingress and service settings * Volume UI ingress support and RocksDB variant guidance * Grafana: new “Upload Errors” panel * **Bug Fixes** * Improved S3 traffic distribution handling across Kubernetes versions * **Documentation** * Expanded README: DB init, admin secret options, worker job types, OpenShift notes * **Chores** * Helm chart bumped to 4.31.0; values restructured for SeaweedFS and COSI bucket class simplified <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
109987f2f0 |
fix(api): publish open spec as x-kubernetes-preserve-unknown-fields, not additionalProperties:true
cozystack-api published the free-form ".spec" of aggregated apps.cozystack.io
resources (Tenant, Application, ...) as the boolean `additionalProperties: true`.
The Kubernetes ValidatingAdmissionPolicy status type-checker run by
kube-controller-manager nil-dereferences when it recurses into that node, so as
soon as a VAP matches one of these resources (cozystack-tenant-host-policy on
tenants) KCM CrashLoopBackOffs cluster-wide and stalls reconciliation, failing
cold installs once the 3x install retry was removed.
Emit `x-kubernetes-preserve-unknown-fields: true` instead. It is semantically
equivalent ("arbitrary fields allowed") and the type-checker handles it safely.
All app custom schemas already use the safe object-form additionalProperties,
so patchSpec was the only source of the boolean form.
Adds a regression test that reproduces the exact crash through
k8s.io/apiserver/pkg/cel/openapi.SchemaDeclType (the entry point KCM uses): it
panics on the pre-fix code and passes after the fix.
Fixes #2863
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Aleksei Sviridkin
|
0c1b438ef3 |
fix(api): close TenantNamespace watch result channel on termination
The Watch goroutine never closed the events channel it owns, and ProxyWatcher.Stop does not close the channel it proxies. When the backing watcher ended, consumers ranging over ResultChan blocked until their own timeout instead of seeing end-of-stream. Closing the channel on goroutine exit matches the watch.Interface contract and the pattern already used by the tenantmodule and application registries. Goroutine-termination tests now assert the result channel closes. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f546696dfb |
test(api): extract fake-watcher stop polling into a helper
Four goroutine-termination tests duplicated the same IsStopped polling loop; waitForFakeWatcherStop keeps the deadline logic in one place. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
9c1c23bdaf |
test(api): share one scheme helper across TenantNamespace registry tests
Manual runtime.NewScheme setups in the List/Get error-path tests silently discarded AddToScheme errors; the shared helper fails the test instead. Renamed from newWatchTestScheme since it now serves both test files. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
13e310fe71 | Merge branch 'main' into chore/seaweedfs-4.31 | ||
|
Aleksei Sviridkin
|
6fc2606111 |
test(api): add read-path authorization tests for TenantNamespace registry
Cover the per-event filtering paths of REST.Watch directly instead of only transitively through the bookmark test: authorization filtering via RoleBindings, privileged groups (system:masters, cozystack-cluster-admin), non-tenant namespace and foreign object drops, backing bookmark forwarding, field and label selector filtering, missing user rejection, access-check error resilience, ADDED deduplication by resourceVersion, terminating bookmark flush on backing watcher close, and goroutine shutdown on Stop and context cancellation. Also close the remaining gaps on the sibling read paths sharing the same authorization logic: List/filterAccessible (per-user filtering, privileged groups, missing user, RoleBindings outside the tenant set, backing list errors) and the Get error branches. Branches the controller-runtime fake client cannot drive (backing bookmarks, non-Namespace objects, chosen resourceVersions, backing watch errors) are exercised through a stub client.WithWatch backed by a test-owned FakeWatcher. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
c6f6fee617 |
docs(gpu-operator): document passthrough variant host-driver prerequisite (#2767)
## What this PR does Documents an undocumented prerequisite of the `cozystack.gpu-operator` `default` (passthrough) variant: it assumes the GPU is not already claimed by a host-installed NVIDIA driver. The chart's `vfio-manager` DaemonSet runs the upstream NVIDIA `k8s-driver-manager` init container, which is hard-coded to exit non-zero and label the node `nvidia.com/gpu.deploy.driver=pre-installed` the moment `chroot /host nvidia-smi` returns a driver version (a `PATH` lookup, not a fixed file path). There is no env-var override in `k8s-driver-manager` v0.10.0, so today the only working path is a clean host. The new "Passthrough variant: host preparation" section in `docs/gpu-vgpu.md` covers: - The exact symptom (init log + sandbox-validator crashloop + `lspci` evidence) so operators can recognise it without spelunking. - A clean-host workaround: a build-from-installed purge list (`dpkg-query`, so a zero-match pattern can't silently abort the whole `apt` transaction the way a hardcoded `'cuda-*'` pattern does on a host without the CUDA repo), `/etc/modprobe.d/blacklist-nvidia.conf` (incl. `nvidia_peermem`), `update-initramfs -k all`, reboot. Warns that `autoremove` is dangerous on converged GPU + NVIDIA DOCA / Mellanox hosts because of the shared `nvidia-` prefix. - Post-reboot verification (no `nvidia` modules loaded, no `nvidia-smi` on `PATH`, no leftover NVIDIA DKMS module) and a label-cleanup step — the DaemonSet pods crash-loop and retry on their own, but the operator's success path never resets the `pre-installed` label, so it must be cleared manually. - A "Known limitation" subsection that names the upstream constraint and the planned mitigation path (upstream env-var override + a future `container` variant of the package). Linked to #2763. ### Release note ```release-note docs(gpu-operator): document that the passthrough (`default`) variant requires a clean host and explain how to purge a pre-installed NVIDIA host driver before enabling the variant. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Renamed GPU guide to cover passthrough and vGPU scenarios; updated verification/last-verified info. * Added passthrough host-preparation and troubleshooting, including a clean-host workaround with driver removal, kernel-module blacklist, initramfs update, reboot, and steps to force a fresh device bind. * Clarified FORCE_REINSTALL does not bypass early host-driver detection and noted pending mitigation paths. * Updated OS support table: passthrough requires clean-host on Ubuntu 20.04/22.04/24.04; Ubuntu 26.04 includes usr-merge notes and vGPU marked as caution (⚠️). <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.5.0-rc.2 api/apps/v1alpha1/v1.5.0-rc.2 |
||
|
Aleksei Sviridkin
|
cd36b0d3e3 |
docs(gpu-operator): document passthrough variant host-driver prerequisite
The vfio-manager DaemonSet's k8s-driver-manager init container exits non-zero and labels the node pre-installed whenever it finds a host NVIDIA driver, leaving vfio-pci unbound and the sandbox validator in CrashLoopBackOff. Upstream offers no override knob, so the only working path today is a clean host. Document the symptom (including the Init:CrashLoopBackOff pod state), a build-from-installed purge plus module-blacklist workaround (with the autoremove-vs-DOCA warning and the apt zero-match-abort caveat), the post-reboot verification, and clearing the stale pre-installed label that the operator's success path never resets. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1d747b3d8d |
feat(platform): auto-wire KubeVirt permittedHostDevices and HostDevices feature gate when gpu-operator is enabled (#2768)
Closes #2765. ## What this PR does Makes the passthrough and vGPU variants of `cozystack.gpu-operator` produce a KubeVirt CR that is immediately usable, without an out-of-band `kubectl patch`. Today the gpu-operator stack reports green, `nvidia.com/<model>` shows up under node `.status.allocatable`, but every `VirtualMachine` referencing the resource via `domain.devices.hostDevices` is rejected by the admission webhook because `HostDevices` is not in `developerConfiguration.featureGates` and the resource is not listed under `permittedHostDevices`. That manual reconciliation is now driven by the platform. The change has three parts. `packages/system/kubevirt` now parameterizes the KubeVirt CR template. `developerConfiguration.featureGates` keeps its existing static defaults and appends `.Values.extraFeatureGates`; `spec.configuration.permittedHostDevices` renders verbatim from `.Values.permittedHostDevices` when set; `spec.configuration.mediatedDevicesConfiguration` does the same. Defaults stay empty so the rendered CR is byte-identical to the previous template when no overrides are supplied. The chart's `make update` target re-applies the four sed transforms after fetching the upstream CR, so refreshing the KubeVirt release version stays a single command. `packages/core/platform` adds two defaults tables under `files/`. `gpu-passthrough-defaults.yaml` covers Hopper (H100/H200), Ada Lovelace (L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10), Turing (T4) and Volta (V100/V100S); PCI vendor:device pairs are stable, `resourceName` slugs follow the `<arch>_<model>_<form>_<mem>` convention `nvidia-sandbox-device-plugin` generates as of v25.x, and `externalResourceProvider: true` is set everywhere because those resources are advertised by the sandbox plugin rather than KubeVirt's in-tree device plugin. `gpu-vgpu-defaults.yaml` covers the Pascal–Ampere mdev profile range (A100-40C/80C, A40-24Q/48Q, A30-24C, A10-24Q, V100D-32C, T4-16Q) — the same family span the upstream `vgpu-device-manager` walks `/sys/class/mdev_bus/` for. Ada Lovelace and Blackwell SR-IOV vGPU is intentionally out of scope for the default table and is expected to be advertised via the user-override hook described below. `templates/bundles/iaas.yaml` now selects between the two tables via a new `bundles.iaas.gpuOperatorVariant` knob (`default` for passthrough, `vgpu` for mediated; default is passthrough). The same variant flows through to the gpu-operator Package CR, replacing the previous always-default call. When the package is enabled, the platform injects `HostDevices` into `extraFeatureGates` plus the chosen defaults into `permittedHostDevices` for the kubevirt component's values. Operators extend or replace via top-level platform values: ```yaml gpu: replaceDefaults: false # append (default) vs. wipe-and-replace permittedHostDevices: pciHostDevices: [] # cluster-specific PCI passthrough entries mediatedDevices: [] # cluster-specific vGPU entries mediatedDevicesConfiguration: {} # extra mediated-device types / per-node mappings ``` `docs/gpu-vgpu.md` is updated to drop the manual `kubectl patch` step and describe the auto-wiring contract, the per-variant defaults, the escape hatches, and the sandbox-device-plugin `resourceName` slug convention so an operator whose node publishes a different name knows where to override. ## Test coverage `helm-unittest` covers eight branches in `packages/core/platform/tests/bundles_gpu_kubevirt_wiring_test.yaml`: gpu-operator disabled emits no extras; gpu-operator enabled at default variant emits the HostDevices gate plus `pciHostDevices`; vgpu variant switches to `mediatedDevices` plus `mediatedDevicesConfiguration`; user overrides append by default; `replaceDefaults: true` wipes the platform table; an unknown variant fails fast with a descriptive error. `packages/system/kubevirt/tests/kubevirt_cr_test.yaml` pins the empty-values render and the three populated shapes at the CR template level. `make helm-unit-tests` passes locally on the full repo. ### Release note ```release-note feat(platform): the KubeVirt CR is now auto-configured for GPU host-device passthrough (HostDevices feature gate plus permittedHostDevices populated from NVIDIA defaults) whenever cozystack.gpu-operator is in bundles.enabledPackages. Operators no longer need a manual kubectl patch to make VMs schedule against a GPU. The variant is selected via bundles.iaas.gpuOperatorVariant (default: passthrough; vgpu: mediated). Extend or replace the NVIDIA defaults via .gpu.permittedHostDevices and .gpu.replaceDefaults. **Upgrade note:** the bundle now OWNS spec.configuration.permittedHostDevices on the KubeVirt CR. If you previously hand-edited it (kubectl edit kubevirt), the first reconcile after upgrade overwrites your manual entries with the rendered NVIDIA default table. Before upgrading, move any custom entries into .gpu.permittedHostDevices (set .gpu.replaceDefaults: true to keep only your list) and verify each resourceName against what your nodes advertise (kubectl describe node <node> | grep nvidia.com/) — the default table now carries the exact slug the sandbox device plugin generates, which upper-cases the card's PCI-IDs name and keeps the Tesla/GL tokens (e.g. a Tesla T4 is nvidia.com/TU104GL_TESLA_T4), but a different plugin build can publish a different string; a mismatch makes the admission webhook reject GPU VMs on their next restart or migration. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Bundle-managed GPU Operator with selectable passthrough vs vGPU variants; platform now injects GPU settings into KubeVirt and forwards vGPU manager overrides when enabled. * Default platform mappings for PCI passthrough and vGPU mediated-device resources added. * **Documentation** * Expanded GPU/vGPU deployment guide with bundle-managed vs manual guidance, wiring/precedence rules, resource-name conventions, and verification steps. * **Tests** * New and expanded tests for bundle→KubeVirt wiring, variant behaviors, default/override merging, and idempotent template parameterization. * **Chores** * Added Helm test target and hardened KubeVirt template update/idempotency checks. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
b6108039b7 |
feat(gpu-operator): add container variant for preinstalled host driver (#2766)
## What this PR does Add a third architectural variant to `cozystack.gpu-operator` for hosts where the NVIDIA driver and `nvidia-container-toolkit` are already installed by the distro package manager (the typical Ubuntu/Debian GPU node shape: apt-installed driver + apt-installed CT). The existing variants do not produce a working setup in that shape. `default` (passthrough) runs `vfioManager`, which unbinds the host driver to bind `vfio-pci` — with the host driver active, `k8s-driver-manager` early-exits and the bind never happens, so the GPU never reaches a pod. `vgpu` requires the proprietary NVIDIA vGPU host driver and a license server; the user-installed open or proprietary driver does not satisfy it. The new `container` variant maps onto upstream gpu-operator's third mode (`sandboxWorkloads.enabled=false` + `devicePlugin.enabled=true`) with every component that would conflict with the host install switched off: - `driver.enabled=false` — use the host driver. - `toolkit.enabled=false` — host already has `nvidia-container-toolkit`. Enabling the operator's toolkit DaemonSet would overwrite the `/etc/containerd/config.toml` the host package configured (via `nvidia-ctk runtime configure`), breaking the host runtime wiring. Matches NVIDIA's documented guidance and the existing `examples/values-native-talos.yaml` reference, which pins the same flag for the same reason. - `devicePlugin.enabled=true` — publish `nvidia.com/gpu` to the kubelet. - `vfioManager.enabled=false` — do not unbind the host driver. Upstream renders this `true` unconditionally; it only stays at zero pods because no node carries a `vm-passthrough` sandbox assignment when `sandboxWorkloads` is off — pinned off so the rendered policy matches intent. - `cdi.enabled=false` — upstream defaults this `true`, but CDI spec generation is serviced by the operator-managed toolkit DaemonSet this variant disables. An apt host doesn't auto-generate CDI specs (`nvidia-ctk cdi generate` is a manual step) and uses the classic `nvidia-container-runtime` hook, so leaving CDI on would point the device plugin at CDI injection with no specs on the host and silently break allocation. - `sandboxWorkloads.enabled=false` — regular pod workloads, not VMs. With this off, upstream `defaultWorkload` is already `container`, so no override is needed. Host prerequisite: because the operator's toolkit component is disabled, the node's containerd must already have the `nvidia` runtime registered (`nvidia-ctk runtime configure --runtime=containerd`); `apt install nvidia-container-toolkit` installs binaries only and does not wire containerd. With `driver.enabled=false` the operator uses the pre-installed host driver at its standard location, so a stock apt install needs no `hostPaths.driverInstallDir` override (Talos lands the driver under a non-standard prefix — see `examples/values-native-talos.yaml`). The bundle wiring (`packages/core/platform/templates/bundles/iaas.yaml`) already enables `cozystack.gpu-operator` as an optional package — no bundle changes required. Users opt into this variant by setting `variant: container` in their `Package` CR. Verified locally with `helm template` against `values.yaml` + `values-container.yaml`: ClusterPolicy renders with `driver.enabled=false`, `toolkit.enabled=false`, `devicePlugin.enabled=true`, `vfioManager.enabled=false`, `cdi.enabled=false`, `sandboxWorkloads.enabled=false`. Platform `helm unittest` and the `gpu-operator` variant bats suite are green. Not yet smoke-tested on a real Ubuntu GPU node — the config matches the documented apt-host wiring (legacy `nvidia-container-runtime` hook, CDI off), but an end-to-end CUDA-pod run on hardware is still owed. Closes #2764 ### Release note ```release-note feat(gpu-operator): add `container` variant for hosts with a pre-installed NVIDIA driver and nvidia-container-toolkit, exposing containerized GPU workloads via the standard NVIDIA device plugin without VFIO passthrough or vGPU. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a "container" GPU operator variant to support containerized GPU workloads on hosts with the NVIDIA driver and nvidia-container-toolkit. * **Documentation** * Updated variants documentation and OS support table to include the container variant and per-host requirements (including Ubuntu and Talos notes). * Clarified Talos native pod example wording. * **Tests** * Added regression tests to verify container-variant configuration and component enablement/disablement. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
fb070ffa21 |
feat(hack): rewrite check-readiness as a Go command with test suite (#2755)
## What this PR does Replaces the `hack/check-readiness.sh` diagnostic with a Go command at `cmd/check-readiness`, integrated into the build the same way as `cozypkg`: - `make check-readiness` builds `_out/bin/check-readiness`, with `Version` injected via `-ldflags`. - The tool shells out to `kubectl` and reports only **non-ready** cozystack / Flux / Kubernetes resources, parsed from status conditions (not column heuristics). Flags: `--wait`/`--timeout`, `--watch`, `--core`, `--parallel`, `--verbose`, `-n`/`-l` scoping, `--no-color`, `--version`. Adds a black-box golden test suite under `test/check-readiness/`: - A mock `kubectl` serves canned fixtures; the suite diffs stdout/stderr/exit against golden files across 30 scenarios (arg parsing, condition/phase/suspend handling, CRD-not-installed, kubectl arg propagation, wait/timeout, edge cases) plus a PTY test for color output. - `make test-check-readiness` runs it; wired into `make unit-tests`. Supersedes #2740, which added the same feature set to the bash script. ### Release note ```release-note feat(hack): replace check-readiness.sh with a `check-readiness` Go command (build with `make check-readiness`); reports non-ready cozystack/Flux/Kubernetes resources with --wait/--watch/--core/--parallel/--verbose and namespace/label scoping. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added `check-readiness` command for verifying Kubernetes resource readiness status across namespaces * Supports multiple execution modes: one-shot, wait with timeout, and watch with configurable intervals * Includes namespace filtering, label selectors, parallel resource fetching, and color output control * **Tests** * Added comprehensive test suite validating command behavior across various scenarios <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2755?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
8b4d31866e |
ci: add Go-scoped Renovate configuration (#2850)
## What this PR does Raises the OpenSSF Scorecard **Dependency-Update-Tool** check (was **0/10**) by adding a Renovate configuration at `.github/renovate.json`. Scorecard scores this check on the presence of a recognized config file, so the check turns green as soon as this merges. ### Why Renovate (not Dependabot) Renovate is already the cozystack org standard — it actively raises and merges PRs on `cozystack/ansible-cozystack`, and `cozystack/talm` + `cozystack/boot-to-talos` already ship `renovate.json`. The Renovate GitHub App is authorized on the org, so this config is picked up **without creating a new credential**. There is no Dependabot usage anywhere in the org. > If the app installation is scoped to "selected repositories", `cozystack/cozystack` needs to be toggled into the existing installation by an org admin — a one-click change, not a new auth artifact. The Scorecard check does not depend on that toggle. ### Scoped to Go dependencies only (per the 2026-05-27 decision) - **`enabledManagers: ["gomod"]`** — Renovate only ever touches Go modules. It will **not** open PRs for Dockerfiles, Helm charts, shipped component/operator versions, or GitHub Actions. Components and images stay hand-managed. - **`gomodTidy` + `gomodUpdateImportPaths`** run after each bump. The `gomod` manager auto-discovers both the root module and the `api/apps/v1alpha1` submodule. - **Kubernetes libraries grouped** (`sigs.k8s.io/controller-runtime` + `k8s.io/**`) so `client-go`/`apimachinery` never bump out of lockstep with controller-runtime (a split bump fails to compile). - **Go toolchain directive pinned** (`matchDepTypes: ["golang"]` disabled) — `go.mod` is generated and a toolchain bump needs a matching `setup-go` + codegen run, out of scope for an automated PR. ### "do not merge" on every bot PR (requirement) - `automerge` and `platformAutomerge` are **off**. - Every Renovate PR is labeled **`do-not-merge/hold`** + **`automated`** on creation. Both labels already exist in `.github/labels.yml`; `stale.yaml` already exempts `do-not-merge/hold` PRs from auto-close, so held bumps won't rot. - Bumps therefore stay parked until tests pass and a maintainer removes the label after a hands-on check. > Note: `do-not-merge/hold` is a **signal/marker** (cozystack uses native GitHub, not Prow Tide). Combined with automerge being off, nothing can auto-merge a held PR. Hard merge-blocking would need a status check tied to the label, which would require branch-protection changes that were explicitly excluded. ### Verification - `renovate-config-validator` (Renovate v42): **Config validated successfully.** - Valid JSON. ### Release note ```release-note ci: add a Go-scoped Renovate configuration (gomod only, no automerge, bot PRs held with do-not-merge/hold) to automate Go dependency updates ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Configured automated dependency management for Go modules, including synchronized updates for Kubernetes-related dependencies, rate limiting on concurrent updates, and manual approval requirements to ensure controlled rollouts. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
28dfb40579 |
fix(dashboard): grant tenant dashboard read on cozy-public PVCs (#2843)
## What this PR does The vm-disk **source image** dropdown in the console lists `PersistentVolumeClaims` in `cozy-public` (the `vm-default-images-*` golden images) to offer them when creating a VM disk. That list is issued by the browser as the **tenant identity** (e.g. `system:serviceaccount:tenant-root:tenant-root`), but the `cozy:tenant:dashboard` Role in `cozy-public` only granted read on Flux `helmrepositories`/`helmcharts`. The list returned `403 Forbidden` and the dropdown stayed empty even when images had been enabled (e.g. via `bundles.enabledPackages`). This is identity-dependent, which is why it can look intermittent: a cluster-admin login bypasses tenant RBAC and sees the images, while any real tenant — including `tenant-root-super-admin` — hits the 403. This PR adds `get`/`list`/`watch` on `persistentvolumeclaims` to the Role. `cozy-public` is a shared public namespace, so read on its PVCs is safe to expose to every tenant dashboard subject (the Role is already bound to the tenant groups and the tenant ServiceAccount). A helm-unittest regression pins the grant. Verified with `kubectl auth can-i list pvc -n cozy-public`: - `--as=system:serviceaccount:tenant-root:tenant-root` → no (before this PR) - `--as-group=tenant-root-super-admin` → no (before this PR) - cluster-admin → yes (why it "worked" for admins) No console code changed — this is an RBAC-only fix. ### Release note ```release-note fix(dashboard): allow tenant dashboards to list VM images in cozy-public, fixing the empty source-image dropdown when creating a vm-disk ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Dashboard access permissions have been expanded to include read-level access to persistent volume claims, enabling users to manage and select images through the dashboard more effectively. * **Tests** * Added test suite to verify dashboard role permissions are correctly configured and functional. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d037ff3c71 |
fix(platform): order cozystack-basics after the APIs its admission policies type-check (#2842)
## What this PR does `cozystack-basics` ships cluster-wide ValidatingAdmissionPolicies — the gateway / route / tenant host policies — that match `tenants.apps.cozystack.io` (served by `cozystack-api`) and the Gateway API types (`gateways`, `httproutes`, `tlsroutes`). kube-controller-manager's ValidatingAdmissionPolicy *status* type-checker panics with a nil-pointer dereference in `k8s.io/apiserver/pkg/cel/openapi` when it type-checks a policy against a matched type whose schema it cannot resolve yet. During a cold install this is a deadlock. `cozystack-basics` can install before `cozystack-api` is Ready; the tenant-host policy is created; KCM panics type-checking it against the not-yet-served `tenants` schema and CrashLoops; that stalls cert/token issuance, so `cozystack-api` never starts and the schema never resolves — KCM keeps crashing and every HelmRelease times out on `DependencyNotReady`. This adds `dependsOn` edges so `cozystack-basics` installs only after `cozystack-engine` (which serves `tenants.apps.cozystack.io` via `cozystack-api`) and `gateway-api-crds` are Ready. The policies are then never created before the APIs they type-check are served. A template capability gate (`.Capabilities.APIVersions.Has`) was considered and rejected: operator-generated HelmReleases run without drift detection, so helm-controller is checksum-driven — it would render the policy out at first install and never re-add it once the API appears, silently dropping the protection. Ordering via `dependsOn` gates the first install on Ready, which is reliable. No cycle risk: nothing `dependsOn` `cozystack-basics`, and no `cozystack-engine` component consumes a `cozystack-basics` output, so engine becomes Ready independently. Includes a helm-unittest regression test asserting the `dependsOn` edges stay. The underlying nil-pointer panic is an upstream Kubernetes bug (the best-effort status type-checker should never crash kube-controller-manager); it is unguarded as of the current release and will be reported separately. This change removes the trigger on the cozystack side. ### Release note ```release-note fix(platform): order cozystack-basics after the APIs its admission policies type-check, fixing a cold-install deadlock where kube-controller-manager could crash type-checking policies against APIs (tenants, Gateway API) not yet served ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Bug Fixes** * Improved cluster stability by implementing proper dependency ordering for validation policies during initialization, preventing resource configuration conflicts. * **Tests** * Added integration tests to validate cluster-wide policy dependencies and ordering, ensuring continuous system stability verification. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
5982018629 |
feat(monitoring): add tenant overview dashboard (#2809)
## What Adds a cross-tenant **Tenant Overview** dashboard to the root/infra Grafana. ## Why Platform admins currently have no single view of which tenants consume what. This shows resource utilization, app counts, and health per tenant at a glance. ## Panels - **Fleet summary** — tenants, apps, running pods, CPU/memory/storage totals - **Tenant leaderboard** — per-tenant table: CPU & memory (usage vs requests), storage, pods, PVCs, app count, not-ready apps - **Top consumers** — Top-N bar gauges by CPU, memory, storage, app count - **Trends** — per-tenant CPU & memory usage over time - **Health** — pod restart rate and not-ready HelmReleases per tenant ## Details - Registered in `dashboards-infra.list` only, so it deploys to the root/`cozy-monitoring` Grafana and never exposes cross-tenant data on per-tenant instances (same pattern as `gpu/gpu-tenants`). - Apps are counted from `gotk_resource_info` HelmReleases, excluding the paired `*-system`/`cozy-*` charts. - Uses only standard cadvisor / kube-state-metrics / Flux metrics. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## New Features * Added a Tenant Overview dashboard for comprehensive tenant infrastructure monitoring, including: * Fleet summary with tenant counts, app deployments, and aggregate resource usage * Tenant performance leaderboard with CPU, memory, and storage metrics * Top consumers visualization by resource type and app count * Historical usage trends and pod health metrics <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
c74964bf25 |
docs(release): expand release.md into release-engineer playbook (#2732)
## What this PR does Expands `docs/release.md` from a ~165-line process-design doc into a ~573-line single source covering both the process design (kept verbatim — mermaid diagrams and the RC/regular/patch numbered walkthroughs) and the operational playbook for cutting a release. New sections added under the existing diagrams: - **What CI does on a tag push** — phase-by-phase walk through `tags.yaml` (`prepare-release`, `generate-changelog`, `update-website-docs`), `pull-requests-release.yaml` (finalize), and `update-releasenotes.yaml` (sync release body). - **Automated patch tag** — `auto-release.yaml`'s nightly behavior. - **Backports** — bot mechanism, recursive-backport regression history (fix #2584), manual fallback, cherry-pick triage include/skip rule, monotonic-counter trap for migration backports. - **Changelogs** — pointer to `agents/changelog.md` plus the recurring failure modes that have shipped in real changelogs, with a verification snippet (chart-bump coverage, range-vs-changelog PR sweep, URL liveness, website PR-author check). - **Release-time fires** — three customer-impacting regressions (v1.3.0 cert-manager ingressClassName, v1.4.0 kubevirt-instancetypes upgrade, v1.4.0 Flux 2.7 → 2.8 readiness deadlock) with "what would have caught this" notes. - **Pre-release verification checklist**, **CI failures release engineers commonly hit**, **Force-retagging** (with open RFC #2677 context), **Splitting a release-blocking bundle PR**, **Cleanup after release**. Example version numbers across the walkthrough switched from the upstream `v0.42.x` legacy placeholders to simple semver-style (`v1.2.0`, `v1.2.1`, `release-1.2`) consistent with semver.org examples and applicable at any point in v1.x. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Enhanced release process documentation with clarified CI workflows, comprehensive verification checklists, expanded troubleshooting guidance, and case studies to ensure more reliable releases. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2732?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
c961ffe286 |
feat(ci): test-impact analysis E2E (default-on) + release E2E workflow (#2559)
## What this PR does
Adds **test-impact analysis (TIA)** to the PR E2E workflow so that only
bats files affected by the diff are exercised, and a **release-time E2E
workflow** that runs the full suite on every release tag to keep
coverage on the shipped code.
### 1. Test-impact analysis selector — `hack/select-e2e.sh`
Reads `packages/core/platform/sources/*.yaml` (the PackageSource
dependency graph) and emits the bats files affected by a PR diff. The
walk is the same one `cozypkg dot` renders.
| Input | Output |
|---|---|
| `packages/apps/postgres/values.yaml` | `postgres` |
| `packages/system/postgres-operator/...` | `postgres harbor`
(transitive) |
| `packages/system/cilium/...` | full suite (cilium has many transitive
`*-application` dependents) |
| `packages/library/cozy-lib/...` | full suite (library affects
everything) |
| `hack/e2e-install-cozystack.bats` | full suite (shared install affects
all apps) |
| `hack/e2e-apps/redis.bats` | `redis` (per-app edit, never escalates) |
| `docs/agents/contributing.md` | (empty — no E2E impact) |
| `dashboards/gpu/gpu-fleet.json` | (empty) |
Conservative fallbacks:
- An unrecognised `packages/*` path or a system source with no
`*-application` descendants escalates to the full suite, so a path
inside the graph is never silently dropped.
- Per-app bats edits are matched **before** the full-suite trigger, so
editing one bats file selects only that app rather than escalating.
### 2. Default-on TIA + `full-e2e` opt-out — `pull-requests.yaml`
- The `Select E2E tests` step runs by default and produces `apps` +
`skip` outputs.
- `Run E2E tests` runs unless `skip=true` (docs / dashboards / `*.md`
only).
- Adding the `full-e2e` label skips the selector step (output empty) and
falls through to the unchanged full-bats-list path — the safety hatch
when reviewers want the full suite on a PR.
### 3. `release-e2e.yaml` — full E2E on every release tag
Triggered on `v*.*.*`, `v*.*.*-rc.*`, `v*.*.*-beta.*`, `v*.*.*-alpha.*`.
Builds images + Talos image, then runs the full suite. Independent of
`tags.yaml`'s release publish flow — failure here is an alarm, not a
release blocker.
This closes the coverage gap left by making PR runs default to TIA: the
full suite is now exercised at tag-cut time against the shipped code.
## Tests
`hack/select-e2e_test.bats` covers 11 cases — single app, transitive
operator, networking/library full-suite, docs/dashboards skip,
`kubernetes-application` two-bats mapping, shared E2E helper, install
bats, per-app bats no-escalation, release-e2e workflow change.
YAML parse verified for both new workflow files.
## Workflow injection safety
All `${{ }}` contexts that could carry untrusted data
(`github.base_ref`, label arrays) flow through env vars, never directly
into shell text. `github.workspace` is also indirected through
`WORKSPACE` env in the release workflow.
### Release note
```
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* CI now runs a Test Impact Analysis to pick only affected apps,
reducing E2E runtime.
* Added a release-level E2E workflow that runs full acceptance tests for
version tags with deterministic sandboxing, artifact capture,
conditional retries, teardown, and report upload.
* E2E jobs accept selected-apps metadata and support conditional
full-suite overrides.
* **Tests**
* New test suite validates selection logic, operator expansion,
full-suite escalation, and docs-only/no-op behaviors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
931f286b9f |
fix(ingress,platform): deliver publishing.proxyProtocol to host ingress-nginx (#2799)
## What this PR does `publishing.proxyProtocol: true` is documented to enable PROXY-protocol on the host ingress-nginx, but the three nginx config keys (`use-proxy-protocol`, `real-ip-header`, `enable-real-ip`) never reached the controller. They were written into the `ingress-system` component values of the `cozystack.ingress-application` Package, but that component has no `install:` block, so the package controller never builds a HelmRelease for it and the values are dropped. External traffic behind a PROXY-protocol L4 load balancer kept breaking. This routes the switch through `_cluster.proxy-protocol` in the `cozystack-values` Secret — the same path `expose-external-ips` already travels to reach the host ingress release — and the `extra/ingress` chart applies it to the external-facing ingress (the namespace named by `expose-ingress`). The dead component-values injection is removed. The `extra/ingress` chart also gains a first-class `proxyProtocol` value, so PROXY-protocol can be enabled per Ingress application (a per-app boolean knob like `cloudflareProxy`). Two `fail` guards keep the new knobs safe: the per-app `proxyProtocol` value is rejected on the external-facing (host) ingress — that path must go through `publishing.proxyProtocol`, which also installs ouroboros (hairpin handling) and arms the disable-hazard guard — and `cloudflareProxy` + `proxyProtocol` together are refused, since Cloudflare's HTTP proxy does not speak PROXY-protocol to origins and its `real_ip_header` server-snippet would override `real-ip-header: proxy_protocol`. `use-forwarded-headers` / `compute-full-forwarded-for` are deliberately not set: without a paired `proxy-real-ip-cidr` they would let clients spoof `X-Forwarded-For`, and the PROXY-protocol frame is already the source of truth for the client address. helm-unittest coverage pins runtime delivery on the host path, the per-app path, and that non-host tenants stay untouched; the previous test that only checked template emission into the (non-consumed) Package component is replaced. Affected versions: v1.4.0–v1.4.2 and current main (introduced in #2561). **Upgrade note (action required).** Because the flag was a silent no-op before this fix, a v1.4.x cluster that set `publishing.proxyProtocol: true` aspirationally — without its L4 load balancer actually injecting PROXY frames — kept working by accident. After this fix the flag really enables `use-proxy-protocol`, which makes ingress-nginx require a PROXY frame on every connection; if the LB does not send one, all external ingress breaks. Before upgrading, confirm the upstream LB injects PROXY-protocol v1 frames, or unset `publishing.proxyProtocol`. Closes #2796 ### Release note ```release-note fix(ingress): publishing.proxyProtocol now enables PROXY-protocol on the host ingress-nginx (the config was silently dropped before reaching the controller); the Ingress application also gains a proxyProtocol value to enable PROXY-protocol per-app. ACTION REQUIRED on upgrade: the flag was previously a no-op, so a cluster that set it without the upstream LB injecting PROXY-protocol v1 frames will lose all external ingress once it takes effect — verify the LB, or unset publishing.proxyProtocol before upgrading. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tenant ingress support for PROXY-protocol via a new proxyProtocol option (default: false). * Platform-level publishing.proxyProtocol now controls external/host ingress behavior. * **Bug Fixes / Validation** * Rejects enabling per-app proxyProtocol on external-facing host ingresses. * Rejects using proxyProtocol together with cloudflareProxy to prevent conflicting real‑IP handling. * **Documentation** * Updated docs describing proxyProtocol usage, defaults, and safety requirements. * **Tests** * Added tests covering proxyProtocol wiring, config emission, and failure cases. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
