mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
02a3fda471ea756bbfd0d79d015665ebae669bc2
1920
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
c1071dab9b |
feat(kubernetes): let a node group log the guest serial console (#3637)
## What this PR does Both tenant Kubernetes e2e suites keep missing the node-Ready budget with zero Nodes registered, and the time goes inside the guest, between the worker VMI reaching Running and Talos asking for its apid certificate. Nothing collected today reaches that window. The in-guest capture the harness already has drives `talosctl`, so it needs the apid the worker never got to, and the management side has neither a Node nor a certificate request to read. Four attempts at this failure have been designed from outside the guest; the most recent raised the budget from 12m to 18m and the same fingerprint came back at the new deadline. KubeVirt can stream a VM's guest serial console into a `guest-console-log` container beside virt-launcher, readable with `kubectl logs` and needing nothing from inside the guest — no talosctl, no client certificate, no helper Pod, no reachable apid. That is the one property this failure class demands, so this turns it on for the suite's worker node group and collects it into the diagnostic bundle when node-join fails, ahead of the `talosctl` capture that cannot describe the same runs. Reads start at the beginning of the stream rather than tailing it: a guest that repaints its console would push the boot output out of any `--tail` window, while `--limit-bytes` truncates from the far end instead. Every read is bounded, and the walk stops at a cap that records both counts, so the collector cannot become an unbounded term in a failure path that already runs several others under one deadline. A read that returns no console is not filed as a quiet guest. Three outcomes get three labels, because only one of them says anything about the guest: the read failed and returned nothing, the read succeeded and returned nothing, or it broke off part way. Each read's stderr is kept beside the capture rather than inside it — kubectl's own `container guest-console-log is not valid for pod` would otherwise fill the file, and a console that never started would read as one that had spoken. That case is the headline one here, so the Pod state and events that explain it are collected alongside, once for the whole selector. The switch is a new opt-in `logSerialConsole` on a node group, off by default. It renders nothing unless set to true, including when set to false, because the `KubevirtMachineTemplate` is content-hash named and a byte emitted for a group that did not ask for it renames the template and rolls every live worker VM on upgrade. I verified that by rendering the base tree and this branch: absent and explicit-false produce a byte-identical template name on both charts. Worker VMs come from two charts, `kubernetes` and `kubernetes-nodes`; both carry the field, and the golden-parity check between them gains a case that keeps their output identical. The passing path checks separately that the container actually attached, and runs that check last, after everything the suite exists to prove — a debugging aid should not preempt the assertions that cover the cluster. Without it the suite would first learn the setting was inert in the one run that needed the console, when it can no longer be recovered. Relates to #3513. ### The risk this carries, stated up front The platform turns serial console logging off cluster-wide in the KubeVirt CR. That setting is a mitigation added in February 2026 for kubevirt/kubevirt#15989, where the `guest-console-log` container kept virt-launcher from starting and left VMs in `PodInitializing`. A per-VM `logSerialConsole` takes precedence over it — `isSerialConsoleLogEnabled` reads the VMI field first and only then falls back to the cluster setting — which is what makes this work, and also makes this the first thing on the platform to go through that mitigation. The mitigation was written against KubeVirt v1.6.x; the platform now ships v1.8.4, and the upstream issue is still open with no reproduction reported either way on the newer version. So the first run of these suites is also the experiment, and it doubles as a revalidation of that February mitigation on the version actually shipping, which nobody has done. The two outcomes are distinguishable rather than ambiguous: kubevirt/kubevirt#15989 shows up as virt-launcher wedged in `PodInitializing` with the VMI never reaching Running, while the failure this instruments shows the VMI Running and zero certificate requests. A red run with the first signature is an answer, not noise — and the collector is built so that it says which one happened instead of reporting silence. The blast radius is small: the product field is absent by default and only the e2e node group sets it, so backing this out is one line. The revert criterion, stated before the run rather than after it. If worker virt-launcher Pods sit in `Init` on `guest-console-log` and no Node registers, kubevirt/kubevirt#15989 still reproduces on v1.8.4: drop the `logSerialConsole: true` line from the e2e node group, keep the chart field, and the upstream issue gains the reproduction on a current version that its thread does not have. If the Pods reach Running and the console logs carry boot output, the mitigation is a candidate for removal platform-wide, which is its own change. Either way the run answers something that was open before it. Dropping `virtualMachineOptions.disableSerialConsoleLog` outright was the alternative. It is a smaller diff and needs no values key, but it exposes every VM on the platform to a race nobody has re-tested, instead of one node group in one test. That is its own revalidation exercise and belongs in its own change, on evidence this one produces. ### Verification `make unit-tests` passes. The new bats suite runs green under both `bash` and a `dash` `/bin/sh`, which matters because `hack/cozytest.sh` is `#!/bin/sh` and an earlier revision of this branch carried a bash-only construct that failed silently there. Every test in it is killed by a specific mutation of the code it covers. Both charts' helm-unittest suites pass, the golden-parity script reports byte-identical output across all four cases including the new one, and `make generate` is idempotent at the repo root. Reviewed by Claude across several rounds and by an independent second reader; the review loop found and fixed real defects in this branch, including a green-path check that had become unable to report the one thing it existed for. ### Screenshots Not a UI change. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/26 - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(kubernetes): worker node groups accept `logSerialConsole`, which streams a worker VM's guest serial console into a `guest-console-log` container readable with `kubectl logs`. Off by default. It is the only way to observe a worker that stalls during boot before Talos apid answers, when no Node and no certificate request exist to diagnose from. Enabling it rolls the node group and overrides the platform's cluster-wide `disableSerialConsoleLog`. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an optional `logSerialConsole` setting for Kubernetes worker node groups. * Streams worker VM guest serial-console output to a dedicated log container when enabled. * Configuration is disabled by default and available through Kubernetes application settings. * **Bug Fixes** * Improved worker failure diagnostics with bounded serial-console output and pod-state information. * Validates that enabled worker pools attach the expected console-log container. * **Documentation** * Documented configuration behavior, rollout effects, defaults, and timeout considerations. * **Tests** * Added coverage for rendering, attachment validation, capture limits, and failure scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
4740daa7be |
test(tenant, harbor, seaweedfs, api): anchor guards that pass on a missing document (#3639)
## What this PR does Five helm-unittest suites asserted things that stayed green when the thing they guard disappeared. No chart or template logic changes here — only the tests, and each fix is measured in both directions. **tenant `no_upgrade_force_test.yaml`.** Every test asserted `notExists: spec.upgrade.force` and nothing else. A `notExists` passes on a template that renders no document at all, so renaming the value that gates one of these releases leaves its test green while the release it guards is gone. Measured on `etcd.yaml`: with the gate renamed, the suite reported 6 passed and a sibling suite's `equal` was the only thing in the chart that noticed. Each test now pins the HelmRelease name alongside the absence. The same file also covers `computeplane.yaml` now. Seven templates in this chart render a HelmRelease and the list held six; the module landed after the suite was written. The invariant holds there today, so this closes an unguarded template rather than a live regression. That is the gap that produced this suite in the first place, when the first sweep missed `gateway.yaml`. Two rationales in that header were wrong and are corrected. The template list was said to be there because `namespace.yaml`'s `lookup` errors under helm-unittest; it does not, it returns empty offline and the template renders, which is what `namespace.yaml`'s own header says and what `namespace_ancestor_labels_test.yaml` relies on. Removing the whole list leaves the chart's suites passing, so the list stays for the reason that actually applies. And helm-controller does not deprecate `upgrade.force`: the CRD vendored at `internal/fluxinstall/manifests/fluxcd.yaml`, which is what the pinned Flux 2.8.x ships, describes it as a replacement strategy with no deprecation marker. `recreate` is the neighbouring field that carries one. **harbor, both suites.** Both addressed their subject by document index. An index is a position, so a template that gains a document ahead of the one under test moves every later assertion onto a different object. Measured by inserting one document at the head of each template: the cleanup suite reddens outright, while two of the four jobservice tests keep passing: the two `notExists` ones, satisfied against an object that never had the path. Every document these suites assert on is unique by kind within its template, so each test now names its subject. A selector that matches nothing fails loudly, including on the `notExists` tests. Indices are also easy to get wrong by hand: helm-unittest numbers documents in template source order while `helm template` prints them sorted by kind, so the rendered output is not the order being counted. **seaweedfs and cozystack-api.** Both forbade a rule by naming it in full, and helm-unittest compares every field it is handed by value equality, so one respelling walks past. For seaweedfs that is not theoretical: ungating the cnpg block and adding a verb to the rule is the same grant and more, and the guard stayed green while a Client instance held delete on another app's Postgres cluster. The Role is now bounded by shape. A Client renders exactly two rules, both in the core group, and the cnpg grant is gated between them, so ungating it adds a third whatever its spelling. For cozystack-api the count bound sits beside the existing named-rule assert rather than replacing it. Neither covers the other: `templates/rbac.yaml` carries no template directives, so the list is pinned at exactly sixteen rules from both sides, while a grant swapped in for an existing one leaves the count where it was and only the named-rule assert sees it. Measured on all three: append, delete, and swap. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff. It touches five files, all under `packages/*/*/tests/`, and no chart, template, values file or tooling. Nothing in the map keys on test files. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Tests** - Improved validation of rendered Kubernetes resources across Harbor, tenant, SeaweedFS, and system API configurations. - Tests now identify resources by kind and relevant properties, making checks more reliable when rendered document ordering changes. - Expanded coverage for tenant upgrade settings, compute plane resources, storage class propagation, cleanup permissions, and network policy access. - Added stricter assertions for expected resource counts, names, rules, and omitted settings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
7dca642894 |
fix(ci): restore e2e for fork PRs and close the merge-gate bypass (#3262)
## What this PR does Fixes #3257 — fork PRs never ran e2e, and the skipped required check let them merge anyway. Fork PRs get no secrets on `pull_request`, so `make image` pushed anonymously and died with `denied`. That failed every build, collapsed the `build → finalize → e2e` chain, and left the required **E2E Tests** job *skipped* — which branch protection treats as satisfied, so an approved fork PR could merge with e2e never having run. This implements **Option A** from the issue (privileged `workflow_run` split), pushing fork e2e images to the existing CI registry (OCIR): **Build primitive (`hack/common-envs.mk`)** - New `OCI_EXPORT_DIR` mode: build each image to a per-image OCI archive instead of pushing, and force `PUSH`/`LOAD` off. The digest is captured via `--metadata-file` regardless of output type, so the refs baked into `pr.patch` match what the privileged run later pushes. Threaded through the `image-tags` macro every package already uses — no per-package Makefile change. **`pull-requests.yaml` (unprivileged, fork branch)** - `build` / `build-talos` / `finalize` export images to OCI archives and upload them as artifacts instead of pushing → the red `denied` build wall disappears and the chain no longer collapses. - The in-tree e2e job is renamed off `"E2E Tests"` and guarded to same-repo PRs. A new `e2e-report` job concludes the required `"E2E Tests"` context as an **explicit check-run** (success on a docs-only PR or when e2e passed, failure otherwise). A skipped job can no longer satisfy the required check — this also closes the same hole for same-repo PRs whose build fails. **`e2e-fork.yaml` (new, privileged `workflow_run` from the default branch)** - Resolves the fork PR, opens a pending `"E2E Tests"` check-run, and fails-closed if the fork build failed. - Pushes the pre-built image archives to OCIR **by digest** and flux-pushes the digest-pinned packages tree. The OCIR token is live only for trusted binaries (`git`/`skopeo`/`flux`/`yq`) over data — never for `make` or any fork-authored script. - Runs the e2e suite against the pushed closure with **no push token** and concludes the check-run on the PR head SHA. Fork build code never receives registry credentials; the fork's images and test scripts run only in the credential-less, ephemeral e2e job — a stronger posture than a `pull_request_target` run that would hand fork code the secrets directly. ### Validation Local: buildx OCI-archive output + digest capture, macro expansion for single- and multi-image packages, non-fork push path unchanged, `actionlint` clean. Needs live validation (why this is a **draft**): `workflow_run` artifact download across runs; that the `"E2E Tests"` commit status satisfies the required context and keeps a fork PR non-mergeable until the privileged run concludes it; `refs/pull/<N>/merge` fetch + `git apply --3way`; `skopeo --preserve-digests` end-to-end. `workflow_run` only runs from the default branch, so the privileged half is exercised by mirroring the workflow onto a personal fork's default branch and opening a throwaway fork PR into it — not by merging first. _Since review_: the fork `e2e` job ran the pre-#2826 BATS app loop that the merged Chainsaw migration deleted (already broken); it now runs the same Chainsaw + TIA path as the in-tree job, with suite selection and the `full-e2e` override sourced from `resolve`'s trusted base-repo file list / labels (not a merge-ref diff). Publish job hardened per the non-blocking follow-ups: base-tree image allowlist, `skopeo copy --all`, pinned flux CLI. The required `E2E Tests` gate is now published as a commit **status** (default token + `statuses:write`), not a COZYSTACK_CI app check-run — the app lacked `checks:write` (`checks.create` 403'd) and a default-token check-run floated under a labeler suite; a commit status has neither problem, and forks can't post one (their `pull_request` token is read-only). Two review-bot findings fixed too: fork `OCI_EXPORT_DIR` is now absolute (a relative path misplaced archives under `make -C <pkg>` and every fork build would fail at upload), and the "≥1 archive" guard no longer no-ops under `nullglob`. ### Repo-admin prerequisites One repository setting lives outside this diff: 1. **Branch protection: keep `E2E Tests` required.** The gate is now a commit **status** named `E2E Tests` (posted with the default `GITHUB_TOKEN` — see `docs/agents/e2e-testing.md` §10), not a job or an app check-run. Branch protection already requires `E2E Tests` (currently the e2e job's name, renamed here to `E2E (in-tree)`), so the required context now resolves to the status — **no GitHub App, no `checks:write` grant, no app-id pin needed**. Just don't pin the context to a specific app. 2. **Actions → "Require approval for all outside collaborators".** Each fork run must be a deliberate maintainer click before it executes, since the fork's unprivileged run produces the artifacts the privileged `e2e-fork.yaml` consumes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff (7 files: two CI workflows, `docs/agents/e2e-testing.md`, `hack/common-envs.mk` + its bats suite, and two package `Makefile`s). The one trigger worth naming is "change developer tooling (`cozyvalues-gen`, `cozypkg`, the package `Makefile`s) → `development.md`" on the website. It does not fire here: `OCI_EXPORT_DIR` is a CI-only build mode that defaults off, and the local developer workflow is unchanged — `make image` still builds and pushes exactly as before, which the new default-path case in `hack/common-envs_test.bats` now pins. No chart values, no API surface, no app list, no platform variant, no Talos version bump and no release asset rename are touched. - [x] No downstream repository is affected by this change ### Release note ```release-note fix(ci): fork pull requests now run e2e via a privileged workflow, and a fork PR can no longer merge on a skipped (never-run) required E2E Tests check ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a dedicated fork E2E workflow that runs after the base PR and reports results via the required **E2E Tests** commit status. - Fork PRs now export build images as OCI archive artifacts for later publishing. - Added a same-repo E2E reporting job and renamed the in-tree E2E flow to **E2E (in-tree)**. - **Bug Fixes** - Added tighter, fail-closed gating to ensure merge blocking when **E2E Tests** isn’t correctly finalized. - **Documentation** - Documented fork vs same-repo E2E split, OCI export behavior, and required fork security settings. - **Tests** - Added Bats coverage validating `OCI_EXPORT_DIR` and preventing unintended registry pushes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
9d1d6808db |
test(seaweedfs,api): bound both Roles against a respelled rule
Each suite forbade a rule by naming it in full, and helm-unittest compares every field it is handed by value equality, so one respelling walks past. For seaweedfs that is not theoretical. Ungating the cnpg block and adding a verb to the rule (get,list,delete rather than get,delete) is the same grant and more, and the guard stayed green while a Client instance held delete on another app's Postgres cluster. Bound the Role by shape: a Client renders exactly two rules, both in the core group, and the cnpg grant is gated between them, so ungating it adds a third whatever its spelling. The rules themselves stay unpinned because the secret list is long and changes on its own schedule. For cozystack-api, add a count bound beside the existing spelling assert rather than in place of it. The two catch different things and neither covers the other: templates/rbac.yaml carries no template directives, so the list is pinned at exactly sixteen rules, while a grant swapped in for an existing one leaves the count where it was and only the named-rule assert sees it. The count is pinned from both sides — a dropped grant is the failure this suite is about, since the apiserver then returns forbidden and List silently drops the dropdown. Measured on all three: append, delete, and swap. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
c20d47a245 |
feat(kubernetes): let a node group log the guest serial console
A worker VM that stalls during boot before Talos apid answers is invisible to every diagnostic the platform has: no Node registers, no certificate request arrives, and talosctl needs the apid that never came up. The guest serial console does show that window, but KubeVirt attaches its guest-console-log container only when the VMI asks for it, and the platform turns console logging off cluster-wide. Add an opt-in per-node-group logSerialConsole that sets the KubeVirt field, which takes precedence over the cluster-wide setting. The key renders nothing unless true -- including when set to false -- because the KubevirtMachineTemplate is content-hash named, so a byte emitted for a group that did not ask for it renames the template and rolls every live worker VM. The description carries the two costs the field name does not show. Changing the value rolls the group it is set on, so the stuck VM whose console was wanted is replaced along with the rest and the setting buys visibility into the next boot rather than the current one. And the cluster-wide default it overrides exists because that same container has been seen holding virt-launcher in PodInitializing, which would take a group down rather than explain it. Worker VMs are emitted by two charts. Both carry the field, and the golden-parity check gains a case that keeps their output identical. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
605030b5a7 |
test(backupstrategy-controller): anchor the metrics-absence guard to its container (#3605)
## What this PR does The `metrics.enabled=false` case in `deployment_test.yaml` guards against the `metrics.enable` typo regression by asserting that the metrics arg and port are absent. Both assertions address the controller by index, and `notContains` passes whenever its path resolves to something that lacks the forbidden content. A different container at that index qualifies, so a sidecar placed ahead of the controller leaves the guard green while the flag it exists to forbid comes back on the real container. This adds an `equal` on `containers[0].name` ahead of the two negatives. It fails when the index stops pointing at the controller, which is the case the negatives structurally cannot see. Verified rather than argued. Rendering a sidecar ahead of the controller against the current tests leaves the case green and reddens only the two neighbouring tests, which use positive `contains` and defend themselves; the guard that exists for this scenario is not among them. With the anchor, the case fails on the anchor. This is not a new convention. `packages/system/opensearch-operator/tests/leader_election_test.yaml` and `packages/system/dashboard/tests/gatekeeper_test.yaml` already anchor index-addressed negatives the same way, the second with the reasoning spelled out in a comment. This brings one more package in line with what the tree already does. ## What this PR does not do It does not switch the assertions to a name filter. helm-unittest resolves `containers[?(@.name=="...")]`, but `notContains` against a filter that matches nothing passes just as quietly, so that trade swaps "wrong container" for "no container" inside the same family of silent green. It does not touch the other two cases in the file. Both are positive `contains`, both went red under the same injection, so they anchor themselves. ## Tests The change is itself a test fix. Package suite is green: 11 tests, 3 suites. ### Screenshots None. This changes a test file. ### Downstream repositories Walked the trigger map against the diff. One helm-unittest file, nine added lines, no chart output changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved deployment validation by confirming the correct container is targeted when metrics are disabled. * Added checks that metrics-related arguments and ports are absent in this configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
cab44d56ac |
test(tests): pin the keep annotation on cert-manager and etcd-operator CRDs (#3590)
## What this PR does
`cert-manager-crds` and `etcd-operator-crds` are the two packages this
repository points at as the in-tree precedent for protecting CRDs with
`helm.sh/resource-policy: keep`. Both carry the annotation, neither had
a test, so it could be lost with nothing reporting the loss. This adds a
suite to each, plus the `test:` target that `hack/helm-unit-tests.sh`
needs before it will run one. No annotation is added and no rendered
output changes: the point is to close the way of losing what is already
there.
The two are exposed differently, so the suites say so instead of sharing
one explanation. In `etcd-operator-crds` the annotation is ours. `make
update` stamps it with an awk step anchored on the
`controller-gen.kubebuilder.io/version:` line that the upstream CRDs
carry. If a future controller-gen stops emitting that line, the step
matches nothing, writes the CRDs out unstamped and exits zero.
In `cert-manager-crds` the annotation comes from the upstream template,
emitted under `{{- if .Values.crds.keep }}`, and the value that switches
it on is ours. That is two independent ways to lose it. The templates
are also not maintained in that package:
`packages/system/cert-manager/Makefile` deletes and refills its
`templates/` directory during its own `make update`, so a cert-manager
bump rewrites a neighbouring package that whoever runs the bump has no
reason to open.
Each suite is scoped with the wildcard `templates/*.yaml` and asserts
across every document rendered under it, rather than listing CRD names
or pinning a document count, so a CRD that upstream starts shipping is
covered without editing the test. Each also asserts the kind, because
the annotation assertion on its own would accept a non-CRD document that
happened to carry it. Both charts render only CRDs today, checked with
`helm template`.
The `test:` target is load-bearing rather than boilerplate.
`hack/helm-unit-tests.sh` runs a package's suite only when its Makefile
defines that target. With the target present, breaking a chart makes the
whole run exit 1 and name the directory. With the fixtures present but
the target removed, the same broken chart gives exit 0 and "All Helm
unit tests passed", having executed the suite zero times.
### Screenshots
Not a UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
I walked the trigger map in `docs/agents/contributing.md` against the
file list of this diff. It adds two test files and two `test:` targets
under `packages/system/`, and touches nothing else: no
`hack/package.mk`, no app `values.schema.json`, no platform values, no
`ApplicationDefinition`, no release asset names. The website's
"developer tooling (the package Makefiles)" trigger is the one worth
checking, so I read `content/en/docs/next/development.md` there: it
documents `make test` in general terms and does not go stale from a
package gaining that target.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added automated validation for cert-manager and etcd operator CRD
charts.
* Confirmed rendered resources are correctly identified as
CustomResourceDefinitions.
* Verified CRDs retain the required `keep` resource policy during Helm
operations.
* **Chores**
* Added convenient test commands for running Helm chart validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
177da7843d |
fix(kubernetes): keep tenant addon CRDs when a release is uninstalled (#3586)
## What this PR does Four tenant addon HelmReleases render their CRDs from a chart's `templates/` directory and remediate a failed install by uninstalling the release. An uninstall deletes what the chart rendered, and deleting a CRD makes the API server garbage-collect every custom resource of that kind, so a failed reinstall takes every Gateway, VerticalPodAutoscaler, ServiceMonitor or VolumeSnapshot in the tenant cluster with it. This annotates the CRDs of the four packages behind them, `gateway-api-crds`, `vertical-pod-autoscaler-crds`, `prometheus-operator-crds` and `vsnap-crd` (that last one is declared by `helmreleases/volumesnapshot-crd.yaml`), with `helm.sh/resource-policy: keep`, which is what `cert-manager-crds` and `etcd-operator-crds` already carry in this tree. Three of the four regenerate `templates/` from upstream on `make update`: two delete the directory outright and the third overwrites its file. So the annotation is written into the vendored files and reapplied by a step in each `update:` target. Without that second half the next regeneration drops it silently, with nothing reporting the loss. I ran `make update` for real to check the pair holds: `gateway-api-crds` and `vsnap-crd` come back byte-identical to what is committed here, and `vertical-pod-autoscaler-crds` reproduces the annotation on both of its CRDs. The rest of the VPA diff is upstream drift, because that package's `update:` tracks a branch rather than a pinned ref; the bump itself is not in this PR. The step is idempotent, so running it twice does not stack a second annotation into the same mapping. `prometheus-operator-crds` ships its CRDs in a vendored subchart that `make update` re-pulls, so nothing under `charts/` is edited. The upstream template already renders `.Values.annotations` onto every CRD it emits, and the annotation is set through the package's `values.yaml`. The Gateway API kustomization also emits a `ValidatingAdmissionPolicy` and its binding. Those hold no user data, so the stamping step selects on `kind: CustomResourceDefinition` and leaves them alone. Each package gains a helm-unittest suite and the `test:` target that `hack/helm-unit-tests.sh` requires before it will run a suite at all. Every suite asserts the annotation across all CRDs its chart renders, not only the ones listed by name, so a CRD arriving in a later upstream bump is covered without editing the test. The annotation has a cost worth stating plainly: uninstalling one of these releases on purpose now leaves the CRDs behind, to be removed by hand. That is the same trade-off `cert-manager-crds` already carries. This covers the first of the two directions in #3555. The second one, setting `install.strategy.name: RetryOnFailure` on these releases, is a separate decision with a different blast radius and is not part of this PR. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: I walked the trigger map in `docs/agents/contributing.md` against the file list of this diff. It touches four packages under `packages/system/` and nothing else: no `hack/package.mk`, no app `values.schema.json`, no platform values, no `ApplicationDefinition`, no release asset names. The one line that looked close is the website's "developer tooling (the package Makefiles)" trigger, so I read `content/en/docs/next/development.md` in the website repo: it documents `make update` and `make test` in general terms, and neither goes stale from this change. ### Release note ```release-note fix(kubernetes): tenant addon CRDs for Gateway API, VerticalPodAutoscaler, Prometheus Operator and VolumeSnapshot now carry `helm.sh/resource-policy: keep`. A failed install that remediates by uninstalling the release no longer deletes those CRDs and, with them, every custom resource of those kinds in the tenant cluster. In exchange, uninstalling one of these releases on purpose now leaves the CRDs in the cluster to be removed by hand. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Custom Resource Definitions for Gateway API, Prometheus Operator, Vertical Pod Autoscaler, and VolumeSnapshot are now retained when their Helm charts are uninstalled. * Prevents loss of custom resources and related configuration during chart lifecycle operations. * **Tests** * Added automated validation confirming all included CRDs receive the retention policy. * Added chart test commands to simplify verification of CRD updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
8aadbd5731 |
test(backupstrategy-controller): anchor the metrics-absence guard to its container
The two notContains assertions in the metrics.enabled=false case address the controller by index. notContains passes whenever its path resolves to something that lacks the forbidden content, and a different container at that index qualifies, so a sidecar inserted ahead of the controller leaves both assertions green while the flag they forbid comes back on the real container. Assert the container's name at the same index first. That fails when the index stops pointing at the controller, which is the case the negatives structurally cannot see. Verified by rendering a sidecar ahead of the controller: before this change the case stayed green and only unrelated tests in the same file went red, by accident rather than by design; after it, the anchor fails. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d90a6a5f46 |
fix(prometheus-operator-crds): keep the CRDs on uninstall
The tenant addon HelmRelease remediates a failed install by uninstalling the release, and an uninstall deletes whatever the chart rendered. Deleting a CRD makes the API server garbage-collect every custom resource of that kind, so a failed reinstall takes every ServiceMonitor, PodMonitor, PrometheusRule and ScrapeConfig in the tenant cluster with it. The CRDs ship inside a vendored subchart that `make update` re-pulls, so the annotation is set through the chart values the subchart already renders rather than by editing templates that would be overwritten. Removing the release on purpose now leaves the definitions behind to be deleted by hand. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
82f5255f5a |
fix(vsnap-crd): keep the CRDs when the release is uninstalled
The tenant addon HelmRelease remediates a failed install by uninstalling the release, and an uninstall deletes whatever the chart rendered. Deleting a CRD makes the API server garbage-collect every custom resource of that kind, so a failed reinstall takes every VolumeSnapshot and VolumeSnapshotContent in the tenant cluster with it — the objects that carry the only reference to a snapshot on the storage backend. Stamp helm.sh/resource-policy: keep on each CustomResourceDefinition and teach `make update` to reapply it, since that target rebuilds templates/ from upstream and would otherwise drop the annotation silently. Removing the release on purpose now leaves the definitions behind to be deleted by hand. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5d5d8f58aa |
fix(vertical-pod-autoscaler-crds): keep the CRDs on uninstall
The tenant addon HelmRelease remediates a failed install by uninstalling the release, and an uninstall deletes whatever the chart rendered. Deleting a CRD makes the API server garbage-collect every custom resource of that kind, so a failed reinstall takes every VerticalPodAutoscaler in the tenant cluster with it, along with the recommendation history held in the checkpoints. Stamp helm.sh/resource-policy: keep on each CustomResourceDefinition and teach `make update` to reapply it, since that target refetches the manifest from upstream and would otherwise drop the annotation silently. Removing the release on purpose now leaves the definitions behind to be deleted by hand. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a822ad10ff |
fix(gateway-api-crds): keep the CRDs when the release is uninstalled
The tenant addon HelmRelease remediates a failed install by uninstalling the release, and an uninstall deletes whatever the chart rendered. Deleting a CRD makes the API server garbage-collect every custom resource of that kind, so a failed reinstall takes every Gateway, HTTPRoute and ReferenceGrant in the tenant cluster with it. Stamp helm.sh/resource-policy: keep on each CustomResourceDefinition and teach `make update` to reapply it, since that target rebuilds templates/ from upstream and would otherwise drop the annotation silently. The ValidatingAdmissionPolicy pair the experimental kustomization also emits holds no user data and stays unannotated. Removing the release on purpose now leaves the definitions behind to be deleted by hand. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ac2f882829 |
test(etcd-operator-crds): pin the keep annotation on the CRDs
The CRDs carry helm.sh/resource-policy: keep, which is what stops a release uninstall from deleting them and taking every EtcdCluster, EtcdMember and EtcdSnapshot in the cluster with it. Nothing asserted that. The annotation is stamped by the awk step in `make update`, anchored on the controller-gen version line the upstream CRDs carry. If that line stops being emitted, the step matches nothing and exits zero. Scope by wildcard rather than a list of filenames, so a CRD added to the vendoring list is covered without editing the test, and assert the kind alongside the annotation, since the annotation assertion on its own accepts a non-CRD document that carries it. Add the test: target too. hack/helm-unit-tests.sh runs a package's suite only when its Makefile defines that target, so without it the fixtures sit unread and the run reports success on a chart that has already lost the annotation. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ead3ab4ed6 |
test(cert-manager-crds): pin the keep annotation on the CRDs
The CRDs carry helm.sh/resource-policy: keep, which is what stops a release uninstall from deleting them and taking every Certificate, Issuer and Order in the cluster with it. Nothing asserted that. The annotation is not local to this package on either side: it comes from the upstream template under a conditional, the value that switches it on is ours, and the templates are refilled from the vendored upstream chart by packages/system/cert-manager on its own update. The assertion reads the rendered output, so it does not depend on which of those moves. Scope by wildcard rather than a list of filenames, so a CRD that upstream starts shipping is covered without editing the test, and assert the kind alongside the annotation, since the annotation assertion on its own accepts a non-CRD document that carries it. Add the test: target too. hack/helm-unit-tests.sh runs a package's suite only when its Makefile defines that target, so without it the fixtures sit unread and the run reports success on a chart that has already lost the annotation. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ed6135d77d |
fix(etcd): fail the CRD vendoring when the keep stamp misses
`make update` stamps helm.sh/resource-policy: keep onto each vendored CRD with an awk step anchored on the controller-gen version line, which upstream owns. If upstream stops emitting that line the step matches nothing, writes the CRD out unstamped and exits zero, so the release quietly loses the protection that keeps a chart uninstall from deleting the CRDs and every EtcdCluster, EtcdMember and EtcdSnapshot under them. Count the annotations the step produced and require exactly one per CRD. Zero means the anchor is gone; more than one means it matched somewhere unintended. Stage every CRD in a temporary directory and replace templates/ only once all of them are counted, so a failure on any one leaves the vendored files exactly as they were instead of wiping the directory and refilling part of the set. Verified by substituting the upstream response: a CRD with no controller-gen line fails with "stamped 0 keep annotations, expected 1", one carrying the line twice fails with 2, a failure on the second CRD leaves all three committed files untouched, and the unmodified upstream still vendors byte-identical output. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
cd57ee6678 |
test(multus): anchor the master-CNI assertion to the top-level key
The check that multus is pointed at the cilium conflist matched its JSON fragment anywhere in the rendered daemon config. The same text nested inside any other key satisfies it, so a config whose effective multusMasterCNI names a different conflist passes. Multus uses that value as written when it is non-empty -- it delegates to whatever is named, so a wrong-but-existing name silently routes through the other CNI instead of cilium, which is the state this assertion exists to prevent. Auto-discovery happens only when the value is empty, and a name that does not exist fails manager startup rather than falling back. Anchor it to the top level, where the template writes its keys. That pins the position rather than parsing the document; deciding which key is effective needs a JSON parser, which helm-unittest does not have. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
7739d1e294 |
feat(multus)!: stage reference CNI plugins from the multus image
Multus-backed NetworkAttachmentDefinitions delegate to the upstream containernetworking reference plugins (bridge, macvlan, ipvlan, ...), and nothing guarantees a node has them in the directory Multus searches. On k3s they live under /var/lib/rancher/k3s/data/cni while Multus and Cilium install into /opt/cni/bin, so a bridge-type NAD fails there with `failed to find plugin "bridge" in path [/opt/cni/bin]`. Bundle 14 of the release's plugins into the multus image and add an install-cni-plugins init container that installs them into the host /opt/cni/bin, mirroring the existing install-multus-binary container. loopback, dhcp, tap and dummy are left out; the Dockerfile records why. Gate it on a new chart value, stageCniPlugins, resolved per bundle and on for both. Talos bundles only bridge, firewall, flannel, host-local, loopback and portmap, so the other ten are missing there exactly as on generic Linux; the four that overlap are rewritten with the same upstream version, v1.9.1 on both sides, rather than an older one. A marker in the image Dockerfile records the Talos release that pin was compared against, and a test holds it to the release the platform ships. multus therefore moves out of system.common-packages into the two variant branches, the way linstor already does for talos.enabled. networking.stageCniPlugins overrides the variant default in either direction. Where staging is on the install is unconditional: a plugin is replaced on every pod recreation, not only when the destination is missing. A destination that is already a directory is reported and left alone, since replacing it is not something this can do safely. Turning the value off afterwards does not undo it; nothing here removes or restores a plugin. The container is not privileged and takes no mount propagation. It runs in the spc_t SELinux domain, without which an enforcing node refuses the write. Each plugin is installed under a temporary name and swapped in with a rename: /opt/cni/bin is shared with cilium and kube-ovn. The plugin tarball is checksummed before extraction. A plugin that cannot be installed is reported on stderr and to /dev/termination-log, and the container still exits 0: Multus stays the node's primary CNI and nothing removes its config, so an init container that never completes would leave the shim with no daemon behind it. The manifest change lives in customize-deployment.patch so it survives make update. BREAKING CHANGE: enabling stageCniPlugins replaces reference CNI plugins in /opt/cni/bin on existing nodes, Talos included -- four of the names it writes are ones the Talos node image also carries. Opt out before upgrading: declining afterwards does not restore what was replaced. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
myasnikovdaniil
|
ee26a50f0e |
fix(postgres-operator): align CNPG operator and CRDs to 1.28.2 for PVC resize-deadlock fix (#3510)
## What this PR does Raises the CloudNativePG operator image **and its CRDs together** to 1.28.2, which carries the PVC resize-deadlock fix (cloudnative-pg#9980 / cloudnative-pg#9981). Symptom: after a simultaneous `resources` + `size` change on a single-instance Postgres, the operator deletes the sole primary Pod, classifies the PVC as `resizing`, and never recreates the Pod — the cluster wedges with zero instances and the disk FS resize never completes. Upstream publishes a CloudNativePG chart only per minor (`.0`/`.1`), so there is no 1.28.2 chart on the 1.28 line. Following the pattern of #3526 / #3528, `patches/cloudnative-pg-1.28.2.patch` (applied by `make update`) raises the vendored chart's `appVersion` and CRDs from 1.28.1 to 1.28.2 in lockstep. The operator image follows `appVersion`, so there is no `image.tag` pin (keeping operator and CRDs aligned and avoiding #3479). Verified #3479-safe: there are no CRD `status` field changes between 1.28.1 and 1.28.2 — only the `extensions` spec grows. Backportable to release-1.6 as-is (same 0.27.1 base chart). ### Release note ```release-note fix(postgres-operator): align CloudNativePG operator and CRDs to 1.28.2, fixing the PVC resize-deadlock that wedged single-instance PostgreSQL clusters after a simultaneous resources+size change ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added support for PostgreSQL extension configuration in catalog and cluster schemas. - Made extension image settings optional where appropriate. - **Bug Fixes** - Updated CloudNativePG to version 1.28.2, including the PVC resize deadlock fix. - Kept the operator image and CRDs aligned at the same version. - **Documentation** - Clarified versioning and chart update guidance. - **Tests** - Added checks confirming operator image and CRD version alignment. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Alexey Artamonov
|
bcb36262ef |
fix(postgres-operator): align CNPG operator and CRDs to 1.28.2 for PVC resize-deadlock fix
Raises the CloudNativePG operator image and its CRDs together to 1.28.2, which carries the PVC resize-deadlock fix (cloudnative-pg#9980 / #9981): after a simultaneous resources+size change the operator deletes the sole primary Pod, leaves the PVC in the resizing class, and never recreates the Pod, wedging the cluster. Upstream publishes a chart only per minor .0/.1, so there is no 1.28.2 chart on the 1.28 line. Following #3526/#3528, patches/cloudnative-pg-1.28.2.patch (applied by make update) raises the vendored chart's appVersion and CRDs from 1.28.1 to 1.28.2 in lockstep; the operator image follows appVersion, so no image.tag pin. Verified #3479-safe: no CRD status field changes between 1.28.1 and 1.28.2 (only the extensions spec grows). Backportable to release-1.6. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Andrey Kolkov
|
22c1df8b94 |
test(etcd-operator): pin deployment unittest to v0.5.4
The manager image tag derives from .Chart.AppVersion, so bumping appVersion to v0.5.4 without updating the two image literals in the deployment unittest broke the suite (helm-unittest gates the PR via `make unit-tests`). Update both asserts (manager container image and the agent OPERATOR_IMAGE env) to ghcr.io/cozystack/etcd-operator:v0.5.4. Suite is green again (18/18). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
19f969b2f8 |
chore(etcd-operator): bump etcd-operator to v0.5.4
Bump the cozystack etcd-operator packages from v0.5.3 to v0.5.4. The release is a controller bug-fix set with no API, RBAC or values changes: - fix(controllers): derive --initial-cluster-state from phase, not from the seed (cozystack/etcd-operator#355) - fix(controllers): stop exempting the bootstrap seed from self-heal (cozystack/etcd-operator#354) - fix(controllers): extend crash-loop self-heal to memory members (cozystack/etcd-operator#352) - fix(controllers): switch the PDB from maxUnavailable to minAvailable (cozystack/etcd-operator#351) Adaptations: - etcd-operator/Chart.yaml: appVersion v0.5.3 -> v0.5.4 (the manager image tag defaults to .Chart.AppVersion, so this reimages the controller). - etcd-operator/Makefile, etcd-operator-crds/Makefile: ETCD_OPERATOR_REF v0.5.3 -> v0.5.4. - etcd-operator-crds/templates/etcdmembers.yaml: re-vendored at v0.5.4 via `make update`; description-only change to the /scale replicas/selector fields tracking the PDB minAvailable fix. etcdclusters/etcdsnapshots unchanged. - templates/rbac.yaml left as-is: manager-role-rules.yaml is byte-identical between v0.5.3 and v0.5.4. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Timofei Larkin
|
e5d4940338 |
chore(codeowners): don't gate per-app generated files on architect review
api/apps/v1alpha1/*/types.go and packages/system/*/cozyrds/*.yaml are entirely produced by `make generate` from each app's own values.yaml, but the /api/ codeowner rule pulled every app's regenerated types.go into an architect-only review, regardless of which area actually changed. Correctness there is already enforced by the pre-commit make-generate CI hooks, not by human review, so list no owner for these generated paths and let that check carry the weight instead. Also fixes a stale api/.gitattributes pattern that never matched any real file (zz_generated_deepcopy.go vs the actual zz_generated.deepcopy.go), and adds a matching linguist-generated attribute for the cozyrds ApplicationDefinition manifests, so both collapse in the PR diff view like the Go files already do. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
myasnikovdaniil
|
10554f9dce |
[cozystack-scheduler] Publish SchedulingClass demo (#2252)
## What this PR does Publish the scripts used for running the Cozystack scheduler demo at the community meetup on 19.03.2026. ### Release note ```release-note [cozystack-scheduler] Add demo scripts to the cozystack-scheduler system packages. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a complete set of interactive Bash demo scripts covering scheduling classes: creating scheduling classes and tenants, colocating pods on a single node, spreading replicas evenly, enforcing one-pod-per-node anti-affinity, and pinning workloads to a specific node. * Included guided scaling steps plus cleanup and teardown scripts to remove demo resources. * **Documentation** * Introduced a shared demo helper with colored, step-by-step terminal logging and Kubernetes polling to wait for pods and Redis readiness, improving clarity and consistency across demos. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timofei Larkin
|
337bb4933e |
[cozystack-scheduler] Publish SchedulingClass demo
## What this PR does Publish the scripts used for running the Cozystack scheduler demo at the community meetup on 19.03.2026. ### Release note ```release-note [cozystack-scheduler] Add demo scripts to the cozystack-scheduler system packages. ``` Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Andrei Kvapil
|
3dc0f56477 |
feat(kube-ovn): build the chart and image in-tree, retiring the wrapper repo (#3460)
## What this PR does Brings the `kubeovn-chart` wrapper into the tree, so `cozystack/kubeovn-chart` can be archived. Until now `packages/system/kubeovn` pulled a chart tarball from the latest `cozystack/kubeovn-chart` tag at `make update` time, and that repository also built and pushed the `kubeovn` image. The wrapper existed only to cache the upstream chart; with the current caching approach that reason is gone, and the split cost a second repository, a second release step, and a package whose provenance a reader cannot follow without leaving the tree. The package is now shaped like `packages/system/cilium`: - `update` pulls the upstream `kubeovn/kube-ovn` chart directly and applies `patches/mtu.diff`, then syncs the version into the Dockerfile. The MTU patch already carried monorepo-relative paths, so it applies unchanged. - `image` builds `images/kubeovn` (carrying the existing `fix-keepvmip-portgroup` patch) and pins the digest into `values.yaml`, using the shared `image-tags` / `cache-args` helpers. It also carries `cozystack/kubeovn-chart#5`, which never got a tag: `OVN_IPSEC_KEY_DIR` is pinned under `/var/lib`. The kube-ovn default sits in `/etc/origin`, which a Talos node cannot create, so enabling IPsec left the node without ESP security associations. ### The nightly host rewrite had to be fixed first Moving the image build here activates a gap recorded in `docs/agents/image-refs.md`. The nightly host rewrite is a literal `<src-registry>/` substring replace, and kubeovn keeps its host in `global.registry.address` with the bare repository in a sibling key — no trailing slash, no match. While the image came from a separate repository the baked reference carried the public host and the ownership filter skipped it; once CI builds the image here it stamps the build-registry host into that key, and the result is an image mirrored to the public registry while the published tree keeps pointing at the private build registry. The second commit adds an end-of-line-anchored expression that fires only when the source registry is the complete scalar value, so a contiguous reference — which continues past the host — is still handled by the first expression and the two can never both fire on one reference. `keycloak-operator` splits at a different boundary and stays unfixed; that is documented, and dormant while no CI path rebuilds it. Verified: `hack/nightly-mirror_test.bats` passes with the new case and the case fails without the fix; `promote-retag` and `promote-rewrite-tags` suites unaffected; `helm template` and `helm unittest` on the package pass. The pre-existing `helm lint` complaint about a missing `kube-ovn` dependency reproduces on `main` and is left alone. ### Follow-ups this PR does not do - `cozystack/kubeovn-chart` should be archived once this merges, with a pointer here. Its open PR #5 is carried by this change and needs no separate landing. - `cozystack/website` links the Kube-OVN license at `cozystack/kubeovn-chart` in `operations/configuration/licenses.md` and `guides/platform-stack/_index.md` (all versions, including `next`). Archived repositories stay readable so nothing breaks, but the link should move upstream or in-tree. **No follow-up has been opened yet** — flagging rather than ticking the box. ### Screenshots N/A — build and packaging change, no user-facing or UI impact. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff. The website row matches — not through the app-list or platform-component triggers, but because two of its pages link the repository being archived. Left unticked because no follow-up exists there yet; a human should decide whether the license link moves to `kubeovn/kube-ovn` or to this package. No other row matches: the change touches no `packages/apps/` or `packages/extra/` package, no `values.schema.json`, no `ApplicationDefinition`, no platform variant, and no node prerequisite. ### Release note ```release-note feat(kube-ovn): the Kube-OVN chart and image are now built in this repository instead of cozystack/kubeovn-chart. OVN_IPSEC_KEY_DIR now defaults to /var/lib/ovs_ipsec_keys, so enabling IPsec on Talos nodes no longer leaves them without ESP security associations. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Updated Kube-OVN image build to use upstream chart metadata and keep registry/version alignment during `update`. - Added conditional MTU support for the OVN CNI DS via chart values. - Introduced `OVN_IPSEC_KEY_DIR` to standardize the OVN IPsec keys directory. - **Bug Fixes** - Preserved VM networking port-group assignments during live migration when an alive virt-launcher sibling pod exists. - Improved nightly mirror host rewriting for split and standalone registry host formats. - **Tests** - Added coverage for the end-of-line “whole scalar value” host rewrite case. - **Documentation** - Refreshed “Known gaps” guidance for nightly mirror host rewriting behavior and limitations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
68cdeb8541 |
fix(ci): emit a single tag for capi-providers under OCI export (#3257)
capi-providers-cpprovider passes two --tag flags; under --output type=oci buildx writes both refs into the archive index, and `skopeo copy oci-archive:<f> docker://…` in the privileged push loop then fails with "more than one image in oci, choose an image". Gate the secondary :$(IMAGE_TAG)-$(TAG) tag off when OCI_EXPORT_DIR is set so the exported archive holds a single manifest. Refs: #3257 Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
76c8f890fa |
fix(ci): export OCI archives for buildx calls that bypass image-tags (#3257)
Fork PRs build images to per-image OCI archives instead of pushing, gated on OCI_EXPORT_DIR. The `--output type=oci` was appended only inside the `image-tags` macro, so `packages/system/capi-providers-cpprovider`, whose `image:` recipe uses a hardcoded `--tag` and never calls `image-tags`, ran buildx with no exporter under export mode (PUSH/LOAD forced to 0). No archive was written, and the fork build's `if-no-files-found: error` upload then failed the whole build. Its `--cache-to type=inline` also needs an image exporter. - hack/common-envs.mk: extract the OCI-output snippet from `image-tags` into a reusable `oci-output <image-name>` macro so any buildx call can opt in; `image-tags` now ends with `$(call oci-output,$(1))` (no-op for the ~33 conforming packages). Move `comma :=` above its first user. - packages/system/capi-providers-cpprovider/Makefile: add `$(call oci-output,cluster-api-control-plane-provider-kamaji)` (archive base name equals the OCIR repo the privileged push keys on) and drop the inline cache under export (`--cache-to type=inline` gated off). - .github/workflows/pull-requests.yaml: wrap the `E2E Tests` check-run creation in try/catch. On error, `core.setFailed` so the required merge gate fails loudly and fail-closed instead of silently succeeding. Repo-wide audit of every `docker buildx build` reachable from the root `build:` matrix confirms capi was the only recipe bypassing both `image-tags` and `oci-output`. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrei Kvapil
|
8fc8d23dfd |
fix(computeplane): address review round three
Gate the Proxied ingress Service on _namespace.etcd like the addon HelmRelease it fronts, so an etcd-less tenant with the addon pinned on does not get a selector-less Service (+ test). Correct the secrets framing: the tenant-visible surface is an allowlist (include: [] withholds everything), so drop the never-load-bearing exclude entry, the bats cross-check and the rename-coupling comments, and state the real mechanism. Fix the etcd prerequisite docs (an ancestor's etcd satisfies the wrapped chart; the awaiting-etcd beacon is the diagnostic), reword the tenant toggle to stop advertising the not-yet-shipped placement routing, require maxReplicas >= 1 on the ingress-role node group, and fix comment drift (Service name after the release rename, placement-hook label, package-guard lookup semantics). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
6dcf5373e7 |
fix(computeplane): let Flux derive both release names
The aggregated API cannot represent spec.releaseName: the conversion rebuilds HelmRelease specs without it, so the first tenant edit through the API dropped the module's explicit computeplane-module release name, let it default to the object name computeplane, and collided with the Helm storage record the cluster release owned — with the tenant chart writing the old name back every interval. Drop releaseName from both HelmReleases and derive names from object names, like every other tenant module. Kamaji now writes the admin kubeconfig to computeplane-cluster-admin-kubeconfig; the computeplane-rd secrets.exclude entry follows in the same change, and a bats test pins the exclusion to the name the cluster release actually produces so a future rename cannot silently re-expose the credential. Also add the check-release-name guard the other extra modules carry, and pin the fluxcd addon's distribution artifact to the flux-operator version vendored in system/fluxcd-operator instead of :latest. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
96842d4a94 |
fix(computeplane): validate nodeGroups and require an ingress-nginx role
nodeGroups was a bare object: typos surfaced as a stuck release deep in the wrapped chart instead of an admission error, and groups without the ingress-nginx role silently left the pinned ingress-nginx controller (and with it the ComputePlane ingress) Pending forever, since supplied groups are not merged with the wrapped chart's default md0. Reuse the kubernetes app's nodeGroups typedefs for the schema and refuse rendering when no group carries the role. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
b2232904b6 |
feat(tenant): add the computeplane module toggle
Wire the computeplane module into the tenant chart as a bool toggle, matching the other tenant modules. The module's own Helm release takes the -module suffix so the canonical release name computeplane (and with it the computeplane-admin-kubeconfig Secret) stays reserved for the cluster HelmRelease the module renders. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
4c3262a4ee |
feat(computeplane): deliver ComputePlane as a tenant module wrapping the kubernetes app
Add the operator-owned computeplane tenant module per the revised design (cozystack/community design-proposals/compute-plane): an extra/computeplane chart that provisions a single-tenant, Cozystack-managed cluster for untrusted-code apps by deploying the ordinary apps/kubernetes chart with operator-fixed values (fluxcd, cert-manager, ingress-nginx Proxied). The package re-declares apps/kubernetes as its own source-only component, so the module is self-contained; the cluster release is named computeplane so Kamaji writes the admin kubeconfig to computeplane-admin-kubeconfig — the contract consumed by placement: ComputePlane applications. The tenant holds no admin kubeconfig and can set cluster shape (nodeGroups) only. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
|
7ebce2e942 |
feat(kubeovn): build the chart and image in-tree
kubeovn was the last package whose chart and image were produced by a separate repository: packages/system/kubeovn pulled a tarball from the latest cozystack/kubeovn-chart tag at update time, and that repository built and pushed the image. The wrapper existed only to cache the upstream chart; with the current caching that reason is gone, and the split cost a second repository, a second release step, and a package whose provenance a reader could not follow without leaving the tree. Move the wrapper in, shaped like packages/system/cilium: update pulls the upstream kube-ovn chart directly and applies patches/mtu.diff, and image builds images/kubeovn and pins the digest into values.yaml. The MTU patch already carried monorepo-relative paths, so it applies here unchanged. Also carries cozystack/kubeovn-chart#5, which never got a tag: pin OVN_IPSEC_KEY_DIR under /var/lib. The kube-ovn default sits in /etc/origin, which a Talos node cannot create, so enabling IPsec left the node without ESP security associations. Co-authored-by: Timofei Larkin <lllamnyp@gmail.com> Co-authored-by: Kirill Ilin <stitch14@yandex.ru> Co-authored-by: Aleksei Sviridkin <f@lex.la> Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
8b461f0c0b |
feat(monitoring): drop the Grafana rebuild for the upstream image + catalog plugins (#3378)
## What this PR does Cozystack builds and publishes `ghcr.io/cozystack/cozystack/grafana` — a rebuilt Grafana image — and uses it by default in the monitoring stack. Grafana is AGPL-3.0, so publishing a rebuilt binary makes the project a redistributor of an AGPL Grafana build. This is a governance concern for CNCF Incubation (there is no precedent for a CNCF project redistributing an AGPL binary). The only reason the image was ever rebuilt was to bundle the VictoriaLogs datasource plugin, which was not in the official Grafana plugin catalog at the time. It is now published there as a signed plugin (`victoriametrics-logs-datasource`), so the rebuild is no longer necessary. This PR moves the deployment to the upstream `grafana/grafana` image (digest-pinned, routed through `cozy-lib.image` so mirrored-registry installs keep working) and installs the plugins at container startup via `GF_INSTALL_PLUGINS`, with versions pinned: - `victoriametrics-logs-datasource 0.29.0` — the VictoriaLogs datasource (signed; the reason the image was rebuilt) - `natel-discrete-panel 0.1.1`, `grafana-worldmap-panel 1.0.6`, `marcusolsson-dynamictext-panel 6.2.0` — reproduce the panel set the removed Dockerfile vendored **Air-gapped installs keep working.** The plugin zips are not downloaded from the internet at pod startup: they are mirrored from the official catalog into the existing `grafana-dashboards` image at build time and served by that in-cluster static file server, next to the dashboards it already serves. `GF_INSTALL_PLUGINS` points at `http://grafana-dashboards.cozy-grafana-operator.svc/plugins/…`, so the runtime dependency stays inside the cluster and the artifact is mirrored into private registries together with every other first-party image. A new `allow-to-grafana-dashboards` CiliumNetworkPolicy (mirroring `allow-to-keycloak`) opens tenant egress to that server, since tenant egress is default-deny. As a result, `ghcr.io/cozystack/cozystack/grafana` is no longer built or published: the Dockerfile, the pinned image tag, the `image` targets in both monitoring Makefiles, and the build step in the root Makefile are removed. Notes: - The AGPL-3.0 Grafana binary is no longer redistributed; the mirrored plugin zips are Apache-2.0 licensed and are served unmodified from the official catalog builds (signatures intact). - `allow_loading_unsigned_plugins` and the custom `/var/lib/grafana-plugins` path are dropped: the catalog datasource is signed, and plugins install into the writable default `GF_PATHS_PLUGINS` (`/var/lib/grafana/plugins`). - The VictoriaLogs datasource provisioning (the `GrafanaDatasource` CRs pointing at `vlselect`) is unchanged. - Plugins are re-installed on every pod start from the in-cluster server (no PVC involved, ~77 MB per start per replica over the cluster network); a failed download fails the pod loudly instead of silently dropping the datasource. - `natel-discrete-panel` and `grafana-worldmap-panel` are Angular plugins; Angular is disabled by default in Grafana 11, so they do not load — this is unchanged from the current rebuilt image (same Grafana 11.6.15, same config). They are kept for parity and can be dropped as a follow-up. ### How this was verified - `helm unittest` across the repo passes, including new regression tests: the monitoring chart asserts the exact upstream image reference (with and without `images-registry` set) and the exact `GF_INSTALL_PLUGINS` value; the tenant chart asserts the new egress policy. - Local end-to-end of the full chain in Docker: built the modified `grafana-dashboards` image, served it on a shared network, and started upstream `grafana/grafana:11.6.15` with the rendered `GF_INSTALL_PLUGINS` value. All four plugins download from the in-cluster-style URL (no internet egress from Grafana), and `GET /api/plugins` reports `victoriametrics-logs-datasource` v0.29.0 as `enabled: true` with `signature: valid`. ### Screenshots Not applicable — no UI change. Grafana renders the same datasources and panels; only the image source and plugin delivery mechanism change. ### Downstream repositories Walked the trigger map against the diff. This is an internal change to how the monitoring stack sources the Grafana image and its plugins (`packages/system/monitoring`, `packages/system/grafana-operator`, `packages/apps/tenant` network policy). No package is added, renamed or removed; no `values.schema.json`, `ApplicationDefinition`, platform values, release asset, shared `hack/` tooling or `package.mk` contract changes; the added `grafana.image` key is an internal system-chart value, not a user-facing app field. No downstream repository is affected. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(monitoring): Grafana now runs on the upstream `grafana/grafana` image; the VictoriaLogs datasource and panel plugins are mirrored from the official Grafana catalog into the in-cluster `grafana-dashboards` server and installed at startup, so air-gapped installs keep working. Cozystack no longer builds or publishes `ghcr.io/cozystack/cozystack/grafana`. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Grafana in monitoring is now based on a digest-pinned upstream image for more consistent deployments. * Required Grafana plugins are installed automatically at startup via `GF_INSTALL_PLUGINS`; unsigned-plugin configuration was removed. * Tenant networking now includes a new egress rule specifically for Grafana dashboards. * Grafana dashboards image now includes required plugin assets prepared during image build. * **Tests** * Added/updated checks covering digest pinning, mirrored-registry behavior, plugin-installation behavior, and the new Grafana dashboards network policy. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
d598e672f2 |
fix(reloader): add startupProbe to survive slow first start under load (#3470)
## What this PR does Adds a startup probe to the reloader deployment, rendered from package values through a patch on the vendored chart. Reloader went into CrashLoopBackOff on a loaded install. Events show `Liveness probe failed: ... connect: connection refused`, repeating on the liveness cadence until the kubelet gave up. A refusal rather than a timeout closes two questions at once. It is an RST from the target netns, so the packet arrived and the pod network is fine. It also rules out the mode in [stakater/Reloader#252](https://github.com/stakater/Reloader/issues/252), where the listener is already up and `/metrics` just answers slowly; that one produces probe timeouts. Nothing was listening on `:9090` yet. `http.ListenAndServe` is the last statement of `startReloader`, so the whole prologue runs before the socket opens. The slow part is not the informers, those are lazy and do their List in a goroutine. It is `IsOpenshift = isOpenshift()` in `pkg/kube`: a package-level var, so it runs at init before `main()`, and it does a `GET /apis/project.openshift.io` on a `context.TODO()` carrying no deadline. The latency is the API server's, and the container waits it out under a 100m CPU limit. Without a startup probe the only grace period is the liveness budget itself, `initialDelaySeconds 10 + periodSeconds 10 * failureThreshold 5`. A start that overruns it gets killed mid-prologue, and the restart re-runs the prologue from the top against an API server that is no less busy, so the loop feeds itself. This reaches past reloader. `packages/core/platform/sources/linstor.yaml` lists `cozystack.reloader` in the LINSTOR stack's `dependsOn`, and `dependsOn` fans out to every component of that stack, so a reloader stuck in this loop blocks that whole install. The probe is `/live` at `periodSeconds 5` and `failureThreshold 45`, about 225s, well inside the 10m install timeout that cozystack-operator stamps on generated HelmReleases. Liveness is suspended while it runs and gets its tight budget back as soon as the listener binds. `timeoutSeconds` is 5 to match the other two probes instead of the kubelet default of 1s, since a throttled process at 100m can hold the socket open and still need more than a second to answer. Both liveness knobs the chart already exposes would work without any patch. Widening `failureThreshold` buys the grace permanently and loosens hang detection for the life of the container. `initialDelaySeconds` is start-up-only and would genuinely do, but it is a fixed blind window on every start, including one that binds in three seconds, and nothing restarts a container that wedges inside it. Adaptive against fixed is the trade. The win is smaller than it sounds while `reloader.enableHA` is unset, since `/live` is a constant 200 then and liveness only catches the listener dying outright. A chart bump does not fix this. 2.2.14 still renders no `startupProbe` anywhere, and v1.4.19 added a second eager package-level call next to `IsOpenshift`, so the prologue got longer instead of shorter. `packages/system/velero` already carries a startupProbe patch on its vendored chart for the same reason. `make update` now drives the shared `%-update` rule with an exact `CHART_VERSION`. It has to be exact. With a stale local repo cache `helm pull --version 2.0.0` fails with version-not-found, while `--version "^2.0.0"` quietly resolves to 2.2.14, whose `deployment.yaml` has the ports block at line 179 instead of 161, so the hunk would fuzz or land somewhere unintended. The patch runs with `--forward` because `patch(1)` defaults diverge between implementations over a tree that already carries the change. GNU 2.7.6 refuses and exits 1. Apple 2.0-12u11 takes its `Assume -R? [y]` default, strips the hunk back out and exits 0. GNU under `--batch` does the same, which is the trap, since `--batch` is what you reach for to make a script non-interactive. `--forward` refuses in all three and leaves a `.rej`. Two things keep the probe from disappearing quietly, and they fire at different times. `--forward` catches it at the moment someone re-runs the patch by hand. The test only catches it at the next run of the suite, late enough for a commit to travel in between. `make test` also checks that the vendored `Chart.yaml` still records the pinned version, so bumping `CHART_VERSION` without re-running `make update` fails instead of leaving the pin and the tree disagreeing. Checked by doing the violation and watching it go red. ### Screenshots Not applicable, no UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. All five files sit inside one `packages/system` package. No package is added, renamed or removed, nothing under `hack/` is touched, and no CRD, namespace, `packages/core` values key, version enum or release asset changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(reloader): add a startup probe so a slow first start is not killed by the liveness probe and left in CrashLoopBackOff, which also unblocks the LINSTOR stack that depends on reloader ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a configurable startup health probe for the Reloader service. * The probe allows additional time for the service to start before health checks begin. * Documented the probe’s timing and behavior in the chart configuration. * **Bug Fixes** * Preserved existing liveness and readiness probe behavior. * **Tests** * Added coverage for enabled, disabled, and unaffected health probes. * Added validation to ensure the chart version matches the pinned release. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
9fe6ae2ec1 |
fix(keycloak-configure): patch HelmRelease in release namespace on teardown (#3372)
## What this PR does
The `keycloak-configure` pre-delete teardown Job clears the Flux
`HelmRelease` finalizers as its final step so the release can be
uninstalled. That final `kubectl patch` targeted a hardcoded namespace
that does not match where the release actually installs. The release
installs into `cozy-keycloak`, and the teardown Job's own RBAC — a
`Role` + `RoleBinding` granting `patch` on the `HelmRelease` by release
name — is created in the release namespace, but the patch was aimed at
`cozy-system`.
The result is a deadlock: the Job's `ServiceAccount` is `Forbidden` to
patch the `HelmRelease` in the mismatched namespace, so the Job errors
and retries forever, the `HelmRelease` sticks in `Terminating`, and the
Helm release wedges in status `uninstalling`. This blocks any teardown
or reinstall of the component.
The fix templates both the release name and the namespace from `{{
.Release.Name }}` / `{{ .Release.Namespace }}`, so the patch targets the
namespace where the `HelmRelease` lives and where the teardown RBAC
grants access. I also added a helm-unittest that pins the rendered
target, guards against the hardcoded namespace regressing, and asserts
the patch tracks the release name and namespace.
### Screenshots
N/A — no UI changes.
### Release note
```release-note
fix(keycloak-configure): fix the pre-delete teardown Job so uninstalling the release no longer wedges — it now patches the HelmRelease in the release namespace instead of a hardcoded one
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved Helm chart deletion cleanup to patch the correct HelmRelease
using the chart’s release name and namespace instead of fixed values.
- Reduced risk of teardown targeting resources in an unintended
namespace.
- **Tests**
- Added a new delete/teardown YAML test suite to verify correct
parameterization and that only the intended namespace is modified.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
7f67d02fb7 |
fix(reloader): add startupProbe to survive slow first start under load
Reloader binds its :9090 listener as the last statement of startReloader, so /live and /metrics answer connection refused until everything before it has finished. The blocking part of that prologue is the package-level IsOpenshift = isOpenshift() in pkg/kube, which issues a GET /apis/project.openshift.io against the API server on a context carrying no deadline, before main() is even entered. That latency belongs to the API server, and the container waits it out at a 100m CPU limit. The chart renders no startupProbe, so the only grace period is the liveness budget: initialDelaySeconds 10 + periodSeconds 10 * failureThreshold 5, about 60 seconds. A first start that needs longer is killed mid-prologue, and the restart does not resume it, it runs it again from the beginning, so the loop sustains itself. The blast radius reaches past reloader: the LINSTOR stack lists cozystack.reloader in dependsOn, which fans out to every component of that stack, so a reloader stuck in this loop blocks the whole install. Render a values-driven startupProbe on /live so liveness only begins evaluating once the listener is up. Budget = periodSeconds 5 times failureThreshold 45, so about 225 seconds. timeoutSeconds matches the 5s the chart already gives liveness rather than the kubelet default of 1s, which a throttled process can exceed while holding the socket open. Both liveness knobs the chart already exposes could buy the same grace with no patch to carry. Widening failureThreshold buys it permanently, loosening hang detection for the life of the container. initialDelaySeconds is start-up-only and would genuinely do, but it is a fixed blind window on every start, whereas a startup probe returns liveness to its tight budget the moment the listener binds. The chart is vendored, so the template change is carried as patches/deployment-startup-probe.diff and reapplied by `make update`, which now drives the shared %-update rule with an exact chart version pin instead of pulling whatever is latest. The patch is applied with --forward because patch(1) defaults diverge by implementation over a tree that already carries the change: GNU 2.7.6 refuses and exits 1, Apple 2.0-12u11 takes its "Assume -R? [y]" default and strips the hunk back out with exit 0, and GNU under --batch does the same. --forward refuses in all three. A helm unittest pins the probe, its budget, the no-op path the template promises for an unset value, and the liveness numbers the budget is sized against, so the documented arithmetic cannot drift away from the chart without a red test. The same target asserts that the vendored Chart.yaml still records the pinned version, so raising the pin without re-running `make update` fails instead of leaving the tree and the pin quietly disagreeing. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
fd568dee87 |
fix(keycloak-configure): patch HelmRelease in release namespace on teardown
The pre-delete teardown Job clears the HelmRelease finalizers as its final step, but targeted a hardcoded namespace that does not match the namespace where the release and its RBAC live. The Job's Role and RoleBinding are created in the release namespace and grant patch on the HelmRelease by release name, so the ServiceAccount was Forbidden to patch the HelmRelease in the hardcoded namespace. The Job errored and retried forever, the HelmRelease stuck in Terminating, and the Helm release wedged in "uninstalling", blocking any teardown or reinstall. Target the release name and namespace so the patch matches the RBAC that grants it. Add a helm-unittest pinning the rendered namespace. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
82e10878be |
fix(harbor): guarantee CPU for redis sentinels
The inline RedisFailover ran its three sentinels at 50m CPU request and a 100m limit. Sentinels only become Ready after the redis-operator elects a master and rewrites their monitored-master config, and on a CPU-contended node CFS shares cycles proportionally to the request, so underprovisioned sentinels converge slowly and delay harbor readiness behind them. Raise the sentinels to Guaranteed QoS (requests equal limits, 250m CPU / 128Mi) so election is never starved or throttled under load. The redis data container is unchanged. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Timofei Larkin
|
66c96b4329 |
fix(dashboard): stop requesting offline_access so logout can revoke sessions (#3447)
## What this PR does The dashboard and linstor-gui auth-proxies (oauth2-proxy) requested the `offline_access` scope, so Keycloak issued an **offline** refresh token. oauth2-proxy has no server-side session store: it keeps the refresh token in the `kc-access` cookie and silently refreshes it (`--cookie-refresh`). An offline token is decoupled from the SSO session by design, so logout (`end_session`) does not revoke it. As a result a copied `kc-access` cookie stayed usable for the whole offline-session lifetime (Keycloak default ~30 days) even after the user logged out — logout did not invalidate it. This drops `offline_access` from the proxy `--scope` in both packages, so Keycloak issues an **online** refresh token bound to the SSO session, which logout revokes. The `offline_access` optional client scope is also removed for cleanliness (with a note on why). Scope of the fix: - `packages/system/dashboard` — proxy `--scope` + optional client scope - `packages/system/linstor-gui` — proxy `--scope` + optional client scope + ingress auth unittest The public `kubernetes` OIDC clients (kubectl `oidc-login`) are intentionally left untouched: `offline_access` is legitimate there and those clients do not sit behind oauth2-proxy. Notes: - The keycloak-operator only *adds* client scopes, so removing the optional scope does not strip the realm-default `offline_access` from an already-created client. The proxy `--scope` change is what actually prevents issuance. - This prevents *new* offline tokens. Offline sessions already issued to existing users remain valid until the realm's offline-session idle timeout (~30d) or explicit revocation via the Keycloak Admin REST API / Account Console. - After this change dashboard session lifetime is governed by the online SSO session (`client.session.idle.timeout` / `client.session.max.lifespan`). ### Screenshots Not applicable — no UI change (auth-proxy OIDC scope only). ### Downstream repositories Walked the trigger map against the diff: the change touches only helm templates and a unittest under `packages/system/dashboard` and `packages/system/linstor-gui`. No downstream repository restates the auth-proxy OIDC scope. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(dashboard): stop requesting the offline_access scope in the dashboard and linstor-gui auth-proxies so that logout revokes the session; a copied kc-access cookie is no longer refreshable after logout. Offline sessions already issued to existing users must be revoked manually via Keycloak. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Security Improvements** * Removed requests for offline access and persistent refresh tokens from OIDC authentication. * Sessions now rely on standard scopes, helping ensure logout invalidates authentication as expected. * **Tests** * Updated authentication checks to verify the revised OIDC scope configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
705e39d3d7 |
feat(monitoring): install Grafana plugins from the in-cluster mirror
Point GF_INSTALL_PLUGINS at the grafana-dashboards static file server instead of the public Grafana catalog, so plugin installation needs no internet egress at pod startup and keeps working on air-gapped installs. Route the upstream Grafana image reference through cozy-lib.image so mirrored-registry installs keep working, and add regression tests for the image reference and the plugin install path. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
606ffd5b1b |
build(grafana-operator): mirror Grafana catalog plugins into the dashboards image
Download the pinned Grafana plugin zips (VictoriaLogs datasource + panels) from the official catalog at image build time and serve them next to the dashboards. Grafana pods can then install plugins from this in-cluster server instead of reaching the public catalog at startup, which keeps air-gapped and egress-restricted installs working. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
9c63df7545 |
build(monitoring): stop building and publishing the Grafana rebuild
Remove the custom Grafana image build now that the deployment runs on the upstream grafana/grafana image with catalog plugins. Drops the Dockerfile, the pinned image tag, the image targets in both monitoring Makefiles, and the build step from the root Makefile, so ghcr.io/cozystack/cozystack/grafana is no longer produced or published. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
44ff0fb286 |
feat(monitoring): run Grafana on the upstream image with catalog plugins
Point the Grafana deployment at the upstream grafana/grafana image and install the VictoriaLogs datasource and panel plugins at startup via GF_INSTALL_PLUGINS, pulled from the official Grafana plugin catalog (https://grafana.com/grafana/plugins) instead of being baked into a custom rebuilt image. The VictoriaLogs datasource is now published and signed in the catalog (victoriametrics-logs-datasource), so allow_loading_unsigned_plugins and the custom /var/lib/grafana-plugins path are no longer needed; plugins land in the writable default GF_PATHS_PLUGINS. Plugin versions are pinned to releases compatible with the shipped Grafana version. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Timofei Larkin
|
436743eb34 |
feat(workloadmonitor): bucket metrics endpoint override (#3443)
## What this PR does The cozystack-controller meters S3 bucket sizes for billing by querying SeaweedFS `SeaweedFS_s3_bucket_(size|physical_size)_bytes` metrics from a VictoriaMetrics vmselect it locates via the `namespace.cozystack.io/monitoring` label on the tenant namespace. That discovery hardcodes the assumption that the monitoring stack scraping SeaweedFS runs in the same cluster. Some deployments host SeaweedFS — API and metrics both — in a separate cluster, so bucket sizes must be fetched from a remote Prometheus-compatible endpoint that discovery can never produce. ### Configuration surface A single controller flag, `--seaweedfs-metrics-endpoint`, exposed as a chart value: ```yaml cozystackController: seaweedfsMetricsEndpoint: "https://vm.example.com/path/to/prometheus" ``` The value is an arbitrary base URL of a Prometheus-compatible query API — scheme, host, optional port, and an optional path prefix — and the controller appends `/api/v1/query` to it, normalizing slashes so a trailing slash or a path prefix never produces a doubled or dropped slash. Basic auth can be embedded as URL userinfo (`https://user:pass@vm.example.com/...`); net/http turns it into an `Authorization` header on every request. TLS uses the system trust roots; a custom CA or bearer-token support would be a follow-up flag if a deployment needs it. The flag is validated and normalized at startup (absolute URL, http/https, no query string or fragment), so a malformed value fails the rollout instead of failing queries at runtime. Cluster-wide scope is deliberate: where SeaweedFS and its metrics live is a property of the deployment, not of any tenant, and tenant monitoring stacks keep working and keep being discovered via the `namespace.cozystack.io/monitoring` label for everything else — the flag only redirects the bucket-size queries. Being a deployment argument, it is settable exclusively by platform administrators, so no new writable surface is exposed to tenants. Rejected alternatives: a WorkloadMonitor spec field (bucket monitors are rendered by the tenant-facing bucket chart, so the field would have to travel through tenant-writable values — an SSRF primitive — and it would be near-permanent CRD surface for a deployment-level fact) and per-namespace annotations (per-tenant granularity that nothing needs, plus a second admin-facing configuration mechanism to document and validate). ### Backwards compatibility Strictly opt-in: with the flag unset — the default — resolution walks the exact same label-discovery path and produces the same vmselect URL. No existing deployment changes behaviour. ### Billing safety: failures no longer collapse to zero This also fixes a latent billing bug in the existing path: any metrics query failure (endpoint down, non-200, malformed response) produced an empty result map, and the reconciler overwrote `Workload.status.resources` with it — silently wiping the recorded bucket sizes, which billing reads as an empty bucket. Now: - `queryAllBucketMetrics` returns errors instead of swallowing them into an empty map, so "the bucket is empty" and "the endpoint could not be queried" are distinguishable. - On failure — and per bucket, when a query succeeds but reports no series for a bucket that previously had a size — the reconciler retains the last known `s3-storage-bytes` and `s3-physical-storage-bytes` values instead of dropping them. - The failure is surfaced three ways: a `BucketMetricsUnavailable` warning event on the WorkloadMonitor, an error-level log line, and an error returned from `Reconcile`, which lands in controller-runtime's reconcile error metrics for alerting. Unit tests cover URL validation and path joining, flag-over-discovery precedence, userinfo-based basic auth, and both retention scenarios (query failure and missing series). ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff (controller Go code plus the cozystack-controller chart value and arg plumbing): none of the listed triggers match — no package added or renamed, no CRD, `ApplicationDefinition`, or platform values change, no `hack/` or layout change. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(workloadmonitor): a new --seaweedfs-metrics-endpoint controller flag (chart value cozystackController.seaweedfsMetricsEndpoint) points bucket size metering at an explicit Prometheus-compatible endpoint for deployments where SeaweedFS and its metrics live in a separate cluster; metrics query failures now retain the last known bucket sizes and surface an event and error instead of silently zeroing billing data ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an optional SeaweedFS metrics endpoint configuration. * Metrics endpoint URLs are validated and normalized before use. * Added support for endpoint overrides when discovering bucket metrics. * **Bug Fixes** * Preserved previously recorded bucket sizes when metrics are unavailable or incomplete. * Metrics failures now trigger retries and visible warning events without blocking other reconciliation. * Improved handling of invalid or unsuccessful metrics responses. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
51c34c236e |
fix(linstor-gui): stop requesting offline_access so logout can revoke sessions
Same offline-token-survives-logout issue as the dashboard: the linstor-gui auth-proxy requested offline_access, so a copied kc-access cookie stayed valid for the whole offline-session lifetime after logout. Drop offline_access from the proxy --scope and the optional client scope, and update the ingress auth unittest accordingly. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
99936a92f8 |
fix(dashboard): stop requesting offline_access so logout can revoke sessions
The dashboard auth-proxy (oauth2-proxy) requested the offline_access scope, so Keycloak issued an offline refresh token. oauth2-proxy stores that token in the kc-access cookie (no server-side session store) and silently refreshes it, and an offline token is decoupled from the SSO session: logout (end_session) does not revoke it. A copied cookie therefore stayed usable for the whole offline-session lifetime even after the user logged out. Drop offline_access from the proxy --scope so Keycloak issues an online refresh token bound to the SSO session, which logout revokes. Also remove the offline_access optional client scope for cleanliness. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Timofei Larkin
|
95e4faf68e |
feat(workloadmonitor): bucket metrics endpoint override
The controller discovers where to query SeaweedFS bucket size metrics via the namespace.cozystack.io/monitoring label, which assumes the monitoring stack that scrapes SeaweedFS runs in the same cluster. Some deployments host SeaweedFS and its metrics in a separate cluster, so add a --seaweedfs-metrics-endpoint flag to cozystack-controller (and a matching cozystackController.seaweedfsMetricsEndpoint chart value): an explicit base URL of a Prometheus-compatible query API, with scheme, host, optional port and path prefix; /api/v1/query is appended. Basic auth may be embedded as URL userinfo. The flag is validated and normalized at startup, so a malformed URL fails the rollout rather than the queries. Empty keeps label-based discovery unchanged. Bucket sizes feed billing, so metrics failures no longer collapse to "no size": previously any query failure yielded an empty result map and the reconciler overwrote Workload resources with it, silently wiping the recorded sizes. The reconciler now retains the last known s3-storage-bytes and s3-physical-storage-bytes values, emits a BucketMetricsUnavailable warning event on the WorkloadMonitor, and returns the error from Reconcile so it lands in the reconcile error metrics. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Timofei Larkin
|
2823b76802 |
feat(gateway): advertise HTTP/2 via ALPN on Gateway API listeners (#3398)
## What this PR does Browsers negotiate HTTP/2 exclusively through TLS ALPN: if the terminating listener does not advertise `h2`, every client silently falls back to HTTP/1.1, with no error surfaced anywhere. Cozystack's Gateway API path is served by Cilium's embedded Envoy, and Cilium ships with `enable-gateway-api-alpn=false` — so every Gateway listener advertised no ALPN protocols and all browser traffic through a Gateway ran over HTTP/1.1 (observable in browser dev tools as `http/1.1` on every request). The ingress-nginx path this replaces advertised `h2` out of the box, so the migration to Gateway API silently downgraded clients: head-of-line blocking and the ~6-connections-per-origin limit are back for anything served through a Gateway. This PR enables ALPN (`h2`, then `http/1.1`) on Gateway API listeners, default-on, with an opt-out at each management layer: - `packages/system/cilium/values.yaml` now defaults `gatewayAPI.enableAlpn: true`, which renders `enable-gateway-api-alpn: "true"` into the `cilium-config` ConfigMap. - The platform chart gains `gateway.http2` (default `true`) next to the existing `gateway.enabled`/`gateway.attachedNamespaces`. The `isp-full` and `isp-full-generic` bundles thread it as inline component values into the `cozystack.networking` Package CR (`components.cilium.values.cilium.gatewayAPI.enableAlpn`); inline HelmRelease values override the package `valuesFiles`, so an explicit `gateway.http2: false` actually reaches the dataplane rather than being shadowed by the new chart default. On `isp-hosted` the hosting-side CNI owns this setting and the flag is a no-op. - Tenant Kubernetes clusters (`packages/apps/kubernetes`) get the same fix: when the `gatewayAPI` addon is enabled, the tenant cilium HelmRelease now sets `enableAlpn: true`; individual clusters opt out via `addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false`. ### Why default-on rather than opt-in HTTP/2 for browsers was the platform's behavior before the Gateway API migration, so keeping ALPN off would preserve a regression, not a status quo. Defaulting on is safe because ALPN is a negotiation: clients that cannot speak HTTP/2 simply keep negotiating `http/1.1`, exactly as before. Operators who need the old wire behavior can set `gateway.http2: false` platform-wide. ### Frontend vs backend hop This change affects the client↔gateway hop only. Gateway↔backend connections stay HTTP/1.1 unless a Service explicitly opts in per GEP-1911 by declaring `appProtocol: kubernetes.io/h2c` on its port. Cilium couples that backend-protocol support to the ALPN flag (`enable-gateway-api-app-protocol` flips to true together with it), but it only acts on Services that declare a recognized `appProtocol` value — and no cozystack-shipped Service does today (only plain `http`/`https` values exist in-tree, which Cilium ignores for protocol selection). So upstreams see no change from this PR alone; h2c/gRPC backends become possible to configure, not configured. The setting is cluster-wide Cilium agent config, so it covers the root Gateway and all tenant Gateways at once — Cilium offers no per-Gateway granularity here. Flipping it re-rolls the cilium DaemonSet on upgrade (`rollOutCiliumPods: true`), the same disruption profile as any other cilium config change. ### How it was verified - New helm-unittest suites: `packages/core/platform/tests/bundles_gateway_http2_wiring_test.yaml` asserts the Package CR carries `enableAlpn` true by default and false with `gateway.http2: false` for both cilium bundles; `packages/apps/kubernetes/tests/cilium_gateway_alpn_test.yaml` asserts the tenant HelmRelease default, the addon-off case, and the `valuesOverride` opt-out. Full suites for both charts pass (93 and 184 tests). - `helm template` of `packages/system/cilium` confirms the value is consumed, not just rendered: default produces `enable-gateway-api-alpn: "true"` (and `enable-gateway-api-app-protocol: "true"`) in `cilium-config`; the threaded `false` produces `"false"` for both. ### Screenshots Not a UI change. ### Downstream repositories The website's hand-written platform values table (`content/en/docs/next/operations/configuration/platform-package.md`) needs a row for `gateway.http2` — filed as cozystack/website#625. No other downstream repository restates anything in this diff (no app `values.schema.json`, ansible-consumed platform key, or hack/ tooling changed). - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/pull/625 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(gateway): Gateway API listeners now advertise HTTP/2 via TLS ALPN (h2, then http/1.1) by default, restoring the browser HTTP/2 support lost in the migration from ingress-nginx. Opt out platform-wide with gateway.http2=false, or per tenant Kubernetes cluster via addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false. Backend (gateway-to-upstream) connections are unchanged and stay HTTP/1.1 unless a Service declares appProtocol kubernetes.io/h2c. ``` |