mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-22 12:43:32 +09:00
f83db514b0eb88cefc2923d17aad59daccb4e358
7777
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
548dfa9520 |
docs(ci): say what an empty selection actually means
Four places on this branch describe an empty selection as "classified inert", against a list of three outcomes that every path is said to land in. There is a fourth: an edit under examples/backups/<app>/ where the app has no Chainsaw suite selects nothing, decided inline where backup examples are matched rather than through the inert list. The rule is deliberate and explained where it lives; it just is not one of the three the summary enumerates. The property the two lanes actually depend on is weaker and true: an empty selection means some rule decided the change cannot affect a suite, never that nothing looked at the path. State that instead, in the header that defines the classes and in the privileged lane's comment that consumes the result. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
cc522e7def |
fix(ci): classify the last path when the list has no trailing newline
The classifier this branch introduces is only total if every line reaches the loop that applies it. POSIX read assigns the final line and then returns non-zero when the input does not end in a newline, so a plain `while read` drops it. The fall-through that escalates an unrecognised path lives inside the loop body, so the dropped line is never classified at all and the selection comes back empty. Both lanes read empty as "skip Chainsaw" and post the required status green. The shape that matters is a list whose only unclassified path is last: `docs/x.md` followed by an unrecognised path with no terminator selects nothing, where the same list with a terminator escalates to all 21 suites. The safety net this branch is built around is bypassed by exactly the input it exists to catch. No caller triggers it today, because `git diff --name-only` terminates its output and the fork lane appends a newline explicitly. That makes it caller discipline documented in one of two callers, which the next caller will not read. Guard it where the assumption lives. Both new cases are red without the guard and the twenty-five existing ones stay green, so they pin this behaviour rather than the loop's general shape. They clean up inline instead of through an EXIT trap: that trap replaces the one the bats binary installs for its own bookkeeping, and a test failing under it can print no TAP line at all, which is the opposite of what a regression pin is for. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
a1496727a1 |
docs(ci): describe the classification the fork lane now gets
The privileged resolve step explains why the TIA inputs come from the base repo, and closed with a caveat about the selector: that it escalates only for an unrecognised path inside packages/ and silently selects nothing for a path it does not recognise at all, naming hack/*.mk as the example. That was true of the selector this branch replaces. Measured against the selector as it now stands, hack/common-envs.mk escalates to the full suite, and so does the fork workflow file itself. An empty selection no longer means the path went unrecognised; it means the path was classified as unable to affect what e2e exercises. The difference is the whole point of the change, and the caveat now contradicts it in the one comment a reader consults before touching the privileged lane. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
ee4a48278f |
refactor(ci): drop the unreachable empty-selection branch in both lanes
Both lanes ran `if [ "$FULL_E2E" = "true" ] || [ -z "$SELECTED_APPS" ]` under a step gated on `steps.select.outputs.skip != 'true'`. When `select` runs, an empty result sets `skip=true` and the step never starts, so the only way to arrive with no selection is the `full-e2e` label — which sets `FULL_E2E=true`. The second test could never decide anything the first had not, while the comment above it advertised "full suite when the selection is empty" in the file whose whole point is that a green status means tests ran. Testing `FULL_E2E` alone keeps the real invariant visible. The comments now say what an empty selection means after the preceding commit: every changed path was explicitly classified inert, not that the selector had no idea. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
ae4e9e55cb |
fix(ci): classify every TIA path and escalate the unclassified
`select-e2e.sh` silently selected nothing for a changed path it did not recognise outside `packages/`. Both e2e lanes read an empty selection as "skip Chainsaw" and then post the required "E2E Tests" status green, so an unrecognised path produced a green merge gate with no suite run. The worked example is `hack/common-envs.mk`, which supplies the tag, push and output flags of every image in the tree and matched nothing: `full_suite_pattern` covered `hack/*.sh` and `hack/*.bats` but not `hack/*.mk`. `pkg/`, `tools/`, `go.mod`, `hack/lib/` and `hack/buildkitd.toml` were in the same position, as was `.github/workflows/e2e-fork.yaml` while `pull-requests.yaml` was already escalated. A blanket "empty means full suite" fallback would be the wrong fix: it escalates the inert paths the design deliberately skips. Instead every path now lands in one of three classes by an explicit rule — the full-suite list for shared build inputs and the e2e harness, an inert list for paths that cannot affect what e2e exercises, and the PackageSource graph for `packages/(apps|system|extra)/`. A path matching none of them escalates and logs why, so the next path someone adds fails safe instead of failing open. The two lists are checked in that order, so a specific escalation beats a broad inert directory: `.github/` is inert, the workflows that run the suite are not. `*.md` and `docs/`/`dashboards/` stay ahead of both, so a README under `packages/core/` does not inherit that tree's escalation. Ten tests cover both sides of the classification. Mutation-checked: reverting the fall-through fails the unclassified-path test, dropping `.github/` from the inert list fails the non-e2e-workflow test, and swapping the two list checks fails the existing pull-requests.yaml test. Note the per-entry escalations are belt-and-braces — removing `hack/*.mk` from the pattern still escalates via the fall-through, which is the point of having it. Closes #3392 Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
7dca642894 |
fix(ci): restore e2e for fork PRs and close the merge-gate bypass (#3262)
## What this PR does Fixes #3257 — fork PRs never ran e2e, and the skipped required check let them merge anyway. Fork PRs get no secrets on `pull_request`, so `make image` pushed anonymously and died with `denied`. That failed every build, collapsed the `build → finalize → e2e` chain, and left the required **E2E Tests** job *skipped* — which branch protection treats as satisfied, so an approved fork PR could merge with e2e never having run. This implements **Option A** from the issue (privileged `workflow_run` split), pushing fork e2e images to the existing CI registry (OCIR): **Build primitive (`hack/common-envs.mk`)** - New `OCI_EXPORT_DIR` mode: build each image to a per-image OCI archive instead of pushing, and force `PUSH`/`LOAD` off. The digest is captured via `--metadata-file` regardless of output type, so the refs baked into `pr.patch` match what the privileged run later pushes. Threaded through the `image-tags` macro every package already uses — no per-package Makefile change. **`pull-requests.yaml` (unprivileged, fork branch)** - `build` / `build-talos` / `finalize` export images to OCI archives and upload them as artifacts instead of pushing → the red `denied` build wall disappears and the chain no longer collapses. - The in-tree e2e job is renamed off `"E2E Tests"` and guarded to same-repo PRs. A new `e2e-report` job concludes the required `"E2E Tests"` context as an **explicit check-run** (success on a docs-only PR or when e2e passed, failure otherwise). A skipped job can no longer satisfy the required check — this also closes the same hole for same-repo PRs whose build fails. **`e2e-fork.yaml` (new, privileged `workflow_run` from the default branch)** - Resolves the fork PR, opens a pending `"E2E Tests"` check-run, and fails-closed if the fork build failed. - Pushes the pre-built image archives to OCIR **by digest** and flux-pushes the digest-pinned packages tree. The OCIR token is live only for trusted binaries (`git`/`skopeo`/`flux`/`yq`) over data — never for `make` or any fork-authored script. - Runs the e2e suite against the pushed closure with **no push token** and concludes the check-run on the PR head SHA. Fork build code never receives registry credentials; the fork's images and test scripts run only in the credential-less, ephemeral e2e job — a stronger posture than a `pull_request_target` run that would hand fork code the secrets directly. ### Validation Local: buildx OCI-archive output + digest capture, macro expansion for single- and multi-image packages, non-fork push path unchanged, `actionlint` clean. Needs live validation (why this is a **draft**): `workflow_run` artifact download across runs; that the `"E2E Tests"` commit status satisfies the required context and keeps a fork PR non-mergeable until the privileged run concludes it; `refs/pull/<N>/merge` fetch + `git apply --3way`; `skopeo --preserve-digests` end-to-end. `workflow_run` only runs from the default branch, so the privileged half is exercised by mirroring the workflow onto a personal fork's default branch and opening a throwaway fork PR into it — not by merging first. _Since review_: the fork `e2e` job ran the pre-#2826 BATS app loop that the merged Chainsaw migration deleted (already broken); it now runs the same Chainsaw + TIA path as the in-tree job, with suite selection and the `full-e2e` override sourced from `resolve`'s trusted base-repo file list / labels (not a merge-ref diff). Publish job hardened per the non-blocking follow-ups: base-tree image allowlist, `skopeo copy --all`, pinned flux CLI. The required `E2E Tests` gate is now published as a commit **status** (default token + `statuses:write`), not a COZYSTACK_CI app check-run — the app lacked `checks:write` (`checks.create` 403'd) and a default-token check-run floated under a labeler suite; a commit status has neither problem, and forks can't post one (their `pull_request` token is read-only). Two review-bot findings fixed too: fork `OCI_EXPORT_DIR` is now absolute (a relative path misplaced archives under `make -C <pkg>` and every fork build would fail at upload), and the "≥1 archive" guard no longer no-ops under `nullglob`. ### Repo-admin prerequisites One repository setting lives outside this diff: 1. **Branch protection: keep `E2E Tests` required.** The gate is now a commit **status** named `E2E Tests` (posted with the default `GITHUB_TOKEN` — see `docs/agents/e2e-testing.md` §10), not a job or an app check-run. Branch protection already requires `E2E Tests` (currently the e2e job's name, renamed here to `E2E (in-tree)`), so the required context now resolves to the status — **no GitHub App, no `checks:write` grant, no app-id pin needed**. Just don't pin the context to a specific app. 2. **Actions → "Require approval for all outside collaborators".** Each fork run must be a deliberate maintainer click before it executes, since the fork's unprivileged run produces the artifacts the privileged `e2e-fork.yaml` consumes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff (7 files: two CI workflows, `docs/agents/e2e-testing.md`, `hack/common-envs.mk` + its bats suite, and two package `Makefile`s). The one trigger worth naming is "change developer tooling (`cozyvalues-gen`, `cozypkg`, the package `Makefile`s) → `development.md`" on the website. It does not fire here: `OCI_EXPORT_DIR` is a CI-only build mode that defaults off, and the local developer workflow is unchanged — `make image` still builds and pushes exactly as before, which the new default-path case in `hack/common-envs_test.bats` now pins. No chart values, no API surface, no app list, no platform variant, no Talos version bump and no release asset rename are touched. - [x] No downstream repository is affected by this change ### Release note ```release-note fix(ci): fork pull requests now run e2e via a privileged workflow, and a fork PR can no longer merge on a skipped (never-run) required E2E Tests check ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a dedicated fork E2E workflow that runs after the base PR and reports results via the required **E2E Tests** commit status. - Fork PRs now export build images as OCI archive artifacts for later publishing. - Added a same-repo E2E reporting job and renamed the in-tree E2E flow to **E2E (in-tree)**. - **Bug Fixes** - Added tighter, fail-closed gating to ensure merge blocking when **E2E Tests** isn’t correctly finalized. - **Documentation** - Documented fork vs same-repo E2E split, OCI export behavior, and required fork security settings. - **Tests** - Added Bats coverage validating `OCI_EXPORT_DIR` and preventing unintended registry pushes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
bc6d5e67b7 |
fix(e2e): keep failure snapshots complete and gate tenant patch on aggregated-API auth (#3444)
## What this PR does Two unrelated defects made the sandbox install job's failure handling unreliable. This fixes both. The first hit every red run. When a test fails, the diagnostic snapshot runs `crust-gather collect`, and its host-log leg spawns a privileged debug pod per node. The sandbox runs baseline PodSecurity, which rejects each of those pods with a 403, so crust-gather keeps retrying until it burns its whole collection budget and exits 1. The snapshot verdict then printed `FAILED` on every failed run even though the namespaced object state was already captured, so the line that says whether the uploaded snapshot is trustworthy was always wrong. The host-log leg collects nothing usable here anyway, so this passes `--disable-additional-logs` at all three collect sites: the host snapshot from the BATS runner, the host snapshot from the Chainsaw error handler, and the nested tenant snapshot. Object collection now finishes and the verdict again separates a real collection failure from a complete snapshot. The second is a race in the install test. The "Configure Tenant" step issues its first `kubectl patch` of the root tenant as soon as an earlier test sees the aggregated APIService go `Available`. That condition only tracks endpoint availability, not the serving pod's auth state. A freshly rolled cozystack-api pod can start answering before it has loaded the requestheader client-CA, and until it does it drops the front-proxy identity and answers as `system:anonymous`, so the patch is rejected with a 403 on the test's very first command. This adds a short authenticated GET against the same resource before the patch, so the step proceeds only once the aggregated API both serves and authenticates the request. If the aggregation layer is actually wedged past two minutes the gate fails loud instead of hiding a real regression. Both were traced from a recent sandbox failure (run 30004030331, during PR #3441's CI). Out of scope: the broader convergence flakiness of the install step is not touched here. That covers the long reconcile windows for the storage and platform releases and the app-level races in the Chainsaw suite. The durable product-side fix for the second defect, where cozystack-api should not report ready before the requestheader client-CA is loaded, is a separate follow-up. This partially addresses #3368, its crust-gather diagnostics item, and does not close it. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI changes. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> The diff touches only the e2e test harness (`hack/cozytest.sh`, `hack/e2e-chainsaw/.chainsaw.yaml`, `hack/e2e-chainsaw/_lib/run-kubernetes.sh`, `hack/e2e-install-cozystack.bats`). The trigger map's `hack/` couplings are `hack/e2e-prepare-cluster.bats` (node prerequisites for ansible-cozystack and talm) and `hack/package.mk`, `hack/common-envs.mk`, `hack/update-crd.sh` plus the package layout (ccp, external-apps-example, cozyhr). None of those are moved, renamed, or edited here, and nothing else in the map matches. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(e2e): keep failure snapshots complete when PodSecurity blocks crust-gather's host-log leg, and wait for the aggregated API to authenticate before patching the root tenant so the install step no longer flakes on a system:anonymous 403 ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved failure diagnostics during automated Kubernetes and tenant tests by avoiding unsupported privileged log collection steps. - Ensured failure snapshots complete more reliably instead of timing out or returning incomplete results. - Added a readiness check before configuring the root tenant, preventing intermittent authorization failures during installation tests. - **Tests** - Increased reliability of installation and end-to-end test workflows under baseline security policies. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
85fd592cc7 |
fix(cozy-lib): make the cozyconfig helpers survive uninjected values
`_cluster` and `_namespace` reach a chart through valuesFrom on the
cozystack-values Secret, so they are absent whenever a chart is rendered
on its own: `helm lint`, a plain `helm template`, and any helm-unittest
suite that does not set them. Twelve of the helpers in _cozyconfig.tpl
read straight through the missing map and abort the render rather than
returning their documented empty default.
Two spellings, one cause. `index .Values._cluster "root-host"` raises
`index of untyped nil`, and `.Values._namespace.host` raises
`nil pointer evaluating interface {}.host`. In both cases the trailing
`| default ""` never runs: it applies to the result, and there is no
result, only an error. That default covered a key missing from a map
that exists, never a missing map.
Guard the map instead, which is what `cozy-lib.images-registry` in this
same file already did:
(index (.Values._cluster | default dict) "root-host") | default ""
`index` on an empty map returns nil harmlessly, so the trailing default
applies as it always read. With the map present, `| default dict`
returns it untouched.
This changes no rendered output. Every one of the 34 charts that symlink
cozy-lib was rendered before and after, both with the injected values
populated and with them set to empty maps, and every result is
byte-identical. Eight charts differ between any two runs because they
generate passwords; rendering the unchanged tree twice reproduces
exactly those eight, which is how they are excluded.
What this does not do is fix a chart that fails today. The charts that
cannot render standalone read the injected values directly in their own
templates rather than through these helpers, so they are unaffected by a
change to the library and are being addressed where they live.
Twelve of the thirteen helpers here have no callers anywhere in
packages/ at present; only `cozy-lib.images-registry` is reached, from
`cozy-lib.image`, and it was already guarded. The guard is still worth
having on the rest: they are the library's public surface, they are what
a chart author copies when reaching for cluster configuration, and the
broken spelling propagates by being read. `cozy-lib.image` is called
from four templates, so these helpers do get adopted once they exist.
Add a suite in cozy-lib-tests, which is where library behaviour is
pinned because a library chart renders nothing of its own. One case
covers the uninjected state, which needs no mocking since neither key is
in values.yaml, and one covers the injected state so the guard cannot
quietly change what a populated map returns. The injected case includes
`scheduling` and `branding` specifically: those two are the helpers
whose body was restructured rather than wrapped, so they are the ones
whose present-map behaviour most needs pinning. Against the unguarded
helpers the first case fails with `index of untyped nil`.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
|
8d5ab4cfee |
fix(hack): arm cozytest cluster captures for e2e suites only
The runner's on-failure EXIT trap gates each cluster capture on a binary being installed rather than on there being a cluster to read. A unit runner has kubectl and no cluster, so a red unit test walked into those legs on its way out. What that cost depends on how kubectl fails. Against an endpoint that refuses immediately it is a second; against one that hangs, measured with a stub that never returns, it is ~28s for the previous-logs leg, which self-bounds its pod list and so never reaches the 300s ceiling above it, plus the full 600s for the data-plane leg, whose first pod list carries no bound of its own. Ten minutes of dead waiting for a failure that should cost seconds. Where crust-gather is installed too the bill stops being empty and starts being wrong: that leg needs no reachable cluster to be pointless, only an ambient KUBECONFIG, and it will spend up to 360s snapshotting whatever cluster the current context names. Gate the three legs on the e2e- prefix instead, read from the suite's own name or from the directory holding it. At the top level that prefix is what the project already sorts on: the Makefile builds BATS_UNIT_FILES as $(filter-out hack/e2e-%.bats,$(wildcard hack/*.bats)), so a top-level e2e-* suite is exactly the one `make unit-tests` refuses to run and packages/core/testing/Makefile runs against a live cluster. That wildcard is not recursive, and hack/e2e-apps/ holds live-cluster suites whose own filenames carry no prefix, so the directory is checked too: matching the basename alone would take the captures away from suites that need them, which is the opposite of this change. The e2e path is otherwise unchanged, every leg, bound and message as it was. Not a reachability probe, though that would also tell a cluster from a bare binary. A failing e2e run is frequently one whose apiserver is degraded, and a probe answering "no" there would skip the snapshot in precisely the case the snapshot exists for. The test drives the runner as a subprocess over generated fixtures that differ only in where they sit, with stub kubectl and crust-gather on PATH. Stubbing both matters: the runner calls crust-gather directly rather than through a script that needs kubectl output first, so a kubectl-only stub would leave that leg reaching for the real binary against the ambient KUBECONFIG. Each leg is pinned by the artefact it leaves in the report tree, not only by the one leg that echoes, so no single leg can be ungated without a test going red; each was checked by ungating it alone. All three directions are asserted, and the two armed cases are the positive control for the disarmed one, whose assertion is negative and would otherwise pass just as well if the harness ran nothing; that case also pins that the fixture did fail. Two shell traps are handled rather than left to be rediscovered. The awk transform rewrites any line beginning `@test "` and any line that is exactly `}`, and knows nothing of heredocs, so a fixture written as a heredoc has its own `@test` line harvested into the outer suite as a phantom test that fails first and ends the run before the real tests; the fixtures are built with printf instead. And `set -e` ignores a command whose status is inverted with `!`, so the negative assertion is spelled as an `if`. Both were checked by building the broken variant, not reasoned about. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
3af1ad381b |
fix(ci): count only the pull requests at the head, not near it
The guard requiring exactly one open pull request was written for PRs that share a head commit, but the endpoint it reads is broader than that. Per the API reference it lists pull requests ASSOCIATED with a commit, which for a commit outside the default branch means any open PR whose branch contains it. Stacking one branch on another and opening both therefore counts as two, and the lower one fails closed on a state its author cannot clear by pushing, because pushing does not remove the PR above it. That is the same unfixable red this workflow refuses to inflict elsewhere, and it costs a contributor the one thing fail-closed is supposed to protect: a way forward. Filter the associations by head SHA as well as state. Two pull requests genuinely pointing at the same commit still fail closed, which is the case the guard exists for and the case where a verdict computed for one PR would otherwise satisfy the other. Checked against all four shapes: stacked branches now resolve, a real collision still refuses, a lone PR resolves, and an association with no open PR at the head still refuses. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
806b66f25c |
docs(ci): stop describing a one-workflow E2E CI
Two sentences predate the second workflow and were left describing the world before it. The scope line at the top is the pointer AGENTS.md routes agents through before they touch the E2E CI, and it named only the in-tree workflow. A reader sent there to change the fork lane would not learn that the section holding its invariants exists, which is the one thing that line has to do. It now names both workflows and says which pull requests each serves. The TIA escalation list said edits to "the E2E workflows" escalate to the full suite. That was true while there was one. The selector's list names `pull-requests.yaml` and not `e2e-fork.yaml`, so a change touching only the fork workflow selects nothing: the Chainsaw step is skipped, while the platform install and the OpenAPI tests still run. Say that plainly, and say why it is tolerable rather than pretending it is designed. A `workflow_run` always executes the default-branch copy of the file, so running the suite against a change to it would not exercise that change anyway. It is the same shape as the fail-open the section already cites. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
9d1d6808db |
test(seaweedfs,api): bound both Roles against a respelled rule
Each suite forbade a rule by naming it in full, and helm-unittest compares every field it is handed by value equality, so one respelling walks past. For seaweedfs that is not theoretical. Ungating the cnpg block and adding a verb to the rule (get,list,delete rather than get,delete) is the same grant and more, and the guard stayed green while a Client instance held delete on another app's Postgres cluster. Bound the Role by shape: a Client renders exactly two rules, both in the core group, and the cnpg grant is gated between them, so ungating it adds a third whatever its spelling. The rules themselves stay unpinned because the secret list is long and changes on its own schedule. For cozystack-api, add a count bound beside the existing spelling assert rather than in place of it. The two catch different things and neither covers the other: templates/rbac.yaml carries no template directives, so the list is pinned at exactly sixteen rules, while a grant swapped in for an existing one leaves the count where it was and only the named-rule assert sees it. The count is pinned from both sides — a dropped grant is the failure this suite is about, since the apiserver then returns forbidden and List silently drops the dropdown. Measured on all three: append, delete, and swap. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
a3c1408d08 |
test(harbor): select hook and release documents by kind, not by index
Both suites addressed their subject by document index. An index is a position, so a template that gains a document ahead of the one under test moves every later assertion onto a different object, and what happens then depends on the assertion: measured by inserting one document at the head of each template, the cleanup suite reddens outright, while two of the four jobservice tests keep passing because a `notExists` against an object that never had the path is satisfied for the wrong reason. Indices are also easy to get wrong by hand. helm-unittest numbers documents in template source order, while `helm template` prints them sorted by kind, so the rendered output is not the order being counted. Every document these suites assert on is unique by kind within its template, so each test now names its subject. A selector that matches nothing fails, including on the `notExists` tests, so the anchor holds in the direction that was silent before. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
eac7e1ea15 |
test(tenant): anchor the upgrade.force guard to its HelmRelease
Each test asserted only `notExists: spec.upgrade.force`, which passes on a template that renders no document at all. Renaming the value that gates one of these releases makes the template render nothing and leaves its test green: measured on etcd.yaml, where the guard stayed green while a sibling suite's `equal` was the only thing that noticed the release had vanished. Pin each HelmRelease name alongside the absence so the assertion has a document to be about. Cover computeplane.yaml. Seven templates in this chart emit a HelmRelease and the list held six; the module arrived after the suite was written. The invariant holds there today, so this closes an unguarded template rather than a live regression — which is the same gap that produced this suite, whose first pass missed gateway.yaml. Correct two claims in the header. The template restriction was said to be there because namespace.yaml's `lookup` errors under helm-unittest; it does not, the lookup returns empty offline and the template renders, which is what its own header says and what namespace_ancestor_labels_test.yaml relies on. The restriction stays, for the reason that actually applies: it keeps an unrelated template breaking out of this suite's result. helm-controller also does not deprecate upgrade.force. The CRD vendored at internal/fluxinstall/manifests/fluxcd.yaml, which is what the pinned Flux 2.8.x ships, describes it as a replacement strategy and carries no deprecation marker; `recreate` is the neighbouring field that does. Later upstream documentation adds that the field is ignored under server-side apply, and that sentence is not in the pinned CRD, so the comment says which is which. The reason to keep the flag off is client-side apply, where it replaces resources instead of patching them. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
6d403c3015 |
fix(tenant): retry failed etcd and seaweedfs installs in place
The tenant chart renders both releases with Flux's default install strategy, and install remediation is an uninstall in every case Flux offers. Neither release disables hooks on it, so a remediated install runs each chart's post-delete cleanup Job, and those Jobs delete data on purpose. The etcd one removes the data-etcd-<ordinal> PVCs by name, because the etcd-operator does not reclaim them when the EtcdCluster goes. The seaweedfs one removes the volume PVCs by label and explicitly reclaims the keep-protected CNPG Cluster holding the filer metadata, precisely because nothing else removes it. Both are correct for a real uninstall. Remediation is what calls them, on a release that is only starting slowly. So an install that misses its budget does not merely restart. It destroys the datastore the tenant control planes keep their state in, or the object store's data together with the metadata database that indexes it, and the next attempt begins from nothing. RetryOnFailure keeps the manifests applied and retries the failed release, so no uninstall happens and neither cleanup Job runs. Both actions carry the strategy because the controller performs that retry as an upgrade, so on install alone the retry's own failure falls through to upgrade remediation. The cost is that a real upgrade failure retries rather than rolls back, which gives up no stable safety net: with retries: -1 and the default strategy, upgrade remediation already rolled back and retried without end. The remediation blocks stay because an absent release consults install and an out-of-sync one consults upgrade, both by retry count rather than by strategy. The other side of the trade is that the uninstall was also the only automatic clean slate. On a tenant holding no data yet, an install that wedges on a bad resource state now retries against that state until an operator removes the release. For a release that owns data this is the right exchange, because lost data does not come back and a wedged install does. This covers two of the tenant's data-owning releases. The chart's monitoring release destroys its storage on a remediated install as well and is outside this change, as is the rest of the catalog, whose nested releases want the question asked one at a time. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
cc05c92e47 |
test(ci): require the discarded-label guard per job, not just agreement
The check collected every comparison against the label name and required them to agree on one string. Agreement is not presence: deleting `e2e-report`'s guard outright leaves the other two agreeing with each other, so the assertion stayed green while a discarded label event began publishing again and clobbering the verdict of the run that had actually tested the SHA. That clobber is the defect two earlier commits were written to stop. Require the guard inside each of the two jobs that must carry it, and keep the agreement check for the rename it does catch. Deleting either copy now fails. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
e2e591ac6a |
fix(ci): claim the fork status before the lookups that can throw
`resolve` posted its `pending` last, after four paginated API calls. Any of them returning 5xx fails the job, `should_run` is never set, `report` skips on it, and nothing writes the context at all. A head SHA that was already green then stays green with no suite behind it, for good rather than for the length of a build, and the privileged run's own red is invisible on the pull request because a `workflow_run` check suite hangs off the default branch. That is the exact state the opener exists to prevent, and the same-repo lane already treats this failure mode twice, with `continue-on-error` on its opener and a try/catch around its terminal post. Post the pending as soon as the run is known to own the SHA, which is immediately after the two silent returns. Three of the four lookups now happen with the context already claimed, so a failure in any of them leaves `pending` standing, and pending blocks. The jobs lookup keeps its place ahead of the opener because the discarded-label decision reads it, and posting before that decision is exactly what lets a run stamp a status it does not own. A failure there still leaves the SHA untouched; that half is a deliberate trade between two ways of being wrong, and it is now written down next to the code rather than implied by the ordering. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
bd3f3b93d0 |
fix(tenant): render the chart without the platform's injected values
`helm lint packages/apps/tenant` fails, and two of its three errors are
the chart's own.
`templates/keycloakgroups.yaml` reads `index .Values._cluster
"oidc-enabled"`. `_cluster` is injected by the platform, so it is absent
whenever the chart is rendered on its own, and `index` on a nil map is a
hard error: `index of untyped nil`. The chart cannot be rendered or
linted outside a cluster at all.
The idiom the neighbouring charts use for the same lookup does not
rescue this on its own, which is worth stating because it looks like it
does. `(index .Values._cluster "x") | default "y"` still errors: `index`
evaluates first and fails on the nil map, so `default` never sees a
result to substitute; it only ever covered a key missing from a map that
exists.
The spelling alone does not decide whether a chart breaks, though.
`packages/system/cozystack-basics` uses it and lints clean, because line
1 of its values.yaml is `_cluster: {}`, which makes the map non-nil
before `index` runs. So a chart is broken only when it neither declares
the empty map nor guards at the use site. Guarding at the use site is
preferred here because it makes the template correct whatever
values.yaml contains, which is what lets the chart render standalone;
and `_cluster` is platform-internal, while this chart's values.yaml,
schema and README are generated together from
api/apps/v1alpha1/tenant/types.go as user-facing output.
The guard has to sit on the map rather than on the value it returns,
hence `index (.Values._cluster | default dict) "oidc-enabled"`.
That is behaviour-preserving rather than merely quieter. With `_cluster`
present, `| default dict` returns it untouched and the lookup is what it
was. With it absent the lookup is nil, the `eq` below is false, and the
block stays unrendered, which is what an unconfigured tenant should
produce. Rendered both ways before and after, with
`--api-versions v1.edp.epam.com/v1` so the capability-gated block
actually executes, the output is identical byte for byte: four
KeycloakRealmGroups when `oidc-enabled` is `"true"`, none when it is
`"false"`. Rendering with no values at all goes from a hard error to a
clean render with no groups.
`Chart.yaml` also does not declare `cozy-lib`, which is present as a
symlink under `charts/`, so lint reports missing chart metadata. Declare
it the way `packages/apps/qdrant` does, that being the one chart in the
tree that declares it and does not carry this error. Rendering is
unchanged by the declaration, checked the same way.
`tests/keycloakgroups_oidc_test.yaml` pins all three branches of the
gate: four realm groups when `oidc-enabled` is `"true"`, none when it is
`"false"`, and none when the platform injected no `_cluster` at all.
That third case is the one the guard exists for, and it needs no mock,
because `_cluster` is absent from values.yaml and so "not injected" is
simply the chart's own default state. Against the unguarded template the
suite fails with `index of untyped nil`; the chart's other seven suites
pass identically either way, which is what left the change uncovered.
The third error, `invalid icon URL '/logos/tenant.svg'`, is deliberately
left. Lint wants a URL with a scheme, and all 31 charts in the tree that
carry an icon use a relative path, none an absolute one. Changing this
one would make it the only chart with a different convention, and
choosing the URL a relative path should become is a decision about where
those assets are served from, not about lint.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
|
cc8b01d5cd |
docs(ci): say when each lane's status opener actually lands
The reference section claimed both lanes open the required context when their run starts, and enumerated the triggers it protects against: a re-run, a reopen, a `full-e2e` label. Every one of those starts a Pull Request run, and on a fork that run cannot open the status at all. The opener is guarded same-repo only, because a fork's `pull_request` token is read-only, and the fork's opener lives in the privileged workflow, which starts only once the unprivileged build has finished. So on the lane the sentence was most likely to be read about, an already-green head SHA keeps reading green for the whole build. The gap is structural. A read-only token cannot post a status at any point in the run, and that same read-only token is what makes publishing the gate with the default token safe in the first place. It is bounded by the build rather than open-ended. What was wrong was only the claim that it does not exist. Say so in all three places that carried the overstatement: the section itself, the opener's own comment, and the contract test, which pins that each lane has an opener and cannot express when it lands. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
a27c341843 |
fix(tenant): retry failed monitoring and gateway installs in place
Both releases carry Flux's default install strategy, and install remediation is an uninstall in every case Flux offers. Neither disables hooks on it, so a remediated install runs the app chart's post-delete cleanup Job against a release whose only fault was missing a readiness budget. The monitoring Job deletes PVCs labelled apps.cozystack.io/application.name=<release>-system. That label is not incidental: the metrics and logs claim templates stamp it precisely so the operator copies it onto the storage PVCs, and the cleanup Job selects on it to stay release-scoped. A slow first install therefore deletes the tenant's metrics and logs, and nothing reconstructs them. The gateway Job deletes the cozystack-acme-account Secret. That key regenerates on the next install, so this one costs no data. It costs a re-registration against the ACME provider, and repeated cycles run into its rate limits, which can hold certificate issuance for the tenant down for hours. RetryOnFailure keeps the manifests applied and retries the failed release, so no uninstall happens and neither cleanup Job runs. Both actions carry the strategy because the controller performs that retry as an upgrade, so on install alone the retry's own failure falls through to upgrade remediation. Two things are traded away. A real upgrade failure now retries rather than rolls back, which gives up no stable safety net, since with retries: -1 and the default strategy upgrade remediation already rolled back and retried without end. And the uninstall was the only automatic clean slate, so an install wedged on a bad resource state retries against that state until an operator removes the release. The remediation blocks stay because an absent release consults install and an out-of-sync one consults upgrade, both by retry count rather than by strategy. The chart's etcd and seaweedfs releases lose storage the same way and are outside this change. Beyond those four, computeplane, ingress and info remain on the default strategy, but their charts ship no destructive post-delete hook, so remediation costs them a wasteful teardown rather than data. That is a different question from this one. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c45179a9d0 |
docs(ci): correct the reason the build matrix keeps the rename-detected list
The comment justified the split by saying a path that no longer exists has nothing to build. That is only half the picture: on a rename between packages the source package is still there and still needs rebuilding, and rename detection drops it from the list the matrix reads, so the matrix under-builds. The behaviour predates this change and is out of scope for a docs-only decision, but the comment presented it as deliberate and correct rather than as inherited and left alone. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
9c68b1f83e |
fix(ci): stop a cancelled fork run from publishing a verdict
The invariant that a cancelled run publishes nothing was written down in three places and held in two of them. The fork lane's terminal reporter ran under `always()`, which includes cancellation. When a second privileged run supersedes the first, the dying one still posted `failure` on a head SHA it no longer owned, for a suite that never finished. It self-corrected once the superseding run reported, but that can be two hours later, and until then a legitimate pull request reads red. The same-repo lane was moved off `always()` for exactly this reason. Move this one too: a failed publish or e2e still reports, since neither is a cancellation, and a cancelled run now leaves the `pending` it opened standing until its successor concludes. The same-repo guard's rationale cited a scenario that no longer exists. It described a label event cancelling the run that opened the pull request, which the concurrency split made impossible for every label the workflow discards. The guard is still needed, for a push, for a `full-e2e` label superseding the head run, and for a manual cancel, so the comment now names those instead of an extinct one. The reference section listed the pieces this invariant rests on and omitted the reporters, which is the half a reader is most likely to touch, and counted them, which was already wrong and would go wrong again on the next addition. It now enumerates without counting. The contract pins both reporters, because both lanes have had this defect at different times in the same shape. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
8a21085324 |
fix(ci): decide docs-only from both sides of a rename
The two lanes answered "is this docs-only" from different sources and could disagree. `plan` reads `git diff --name-only`, which applies rename detection and reports a rename as its new path alone, so moving a code file into docs/ read as docs-only. The fork lane reads the PR file list, where a rename carries `previous_filename`, and reads the same change as code. Disagreement is not a near-miss here. The same-repo lane builds nothing and uploads no patch artifact; the fork lane concludes e2e is required and then fails closed because that artifact is missing, reporting that the fork build did not complete. It completed — it correctly built nothing. There is no push the contributor can make to clear it and no label that overrides it, so a legitimate contribution is permanently red. Point the docs-only decision at the rename-blind list, which the job already computes for the downstream-trigger-map flag and whose comment names this exact trap. Both lanes now agree, and the same-repo side stops shipping a removed code path untested — until now such a change skipped every build and reported e2e as not required. The build matrix keeps reading the rename-detected list: it picks packages to rebuild, and a path that no longer exists has nothing to build. The contract file gains the agreement as a pin, and grows a header to match what it now covers. The fork half is pinned on the docs-only expression rather than on `previous_filename`, which also appears in the file list a few lines down and would have kept the assertion satisfied with the check deleted. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
8e097488c2 |
docs(ci): name every writer of the "E2E Tests" status, and why it opens
The section that tells the next editor to keep exactly one run per head SHA responsible for the status listed only two of the three jobs that write it, and described the same-repo lane as posting once at the end. Both stopped being true when the same-repo lane gained an opener. The section also spent a paragraph on the concurrency invariants without saying why an opener has to exist at all, which is the part a reader would otherwise remove as redundant. Say that each lane opens with `pending` and concludes, that a status never expires so the opener is what stops a re-run inheriting the previous run's green, and that `plan`'s opener is `continue-on-error` because every build and test job hangs off that job. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
d3515509c8 |
fix(ci): let a status-API blip skip the opener instead of failing the run
The step that claims the required context is the first in `plan`, and `checks`, `build`, `build-talos`, `finalize` and `e2e` all hang off that job. A transient failure from the status API would therefore discard a full build and suite for a call that is an optimisation, not a gate. Mark it `continue-on-error`. Losing the opener degrades exactly to the behaviour that preceded it — the head SHA keeps whatever status it had until `e2e-report` concludes — while a run that dies before reporting leaves the merge blocked either way. The terminal post keeps its own error handling, because that one failing silently is what would let a run finish with no verdict at all. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
7fc568db4f |
test(ci): pin each status publisher by its call, not by a shared string
The case asserting that both lanes open the required status before they conclude it counted occurrences of the context string per file and required two. That measured nothing on the fork lane: the two occurrences there are the shared `setStatus` helper and the terminal report, while the opener calls the helper and contributes no occurrence of its own. Deleting the opener outright left all six tests green, under a comment promising that dropping either half would be caught — and the window it guards is the wider of the two, since a privileged run holds the head SHA for up to three hours. Scope every assertion to the job that owns it and pin the call rather than a string that happens to appear twice: `plan` opens with `pending` under a same-repo guard, `e2e-report` concludes, `resolve` opens through the helper, `report` concludes. Removing any one of the four now fails this test. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
ee844c98a5 |
fix(ci): open the "E2E Tests" status when the same-repo run starts
Moving the gate from a job to a commit status lost a property the job had for free. A commit status is latest-wins per (sha, context) and never expires, so a head SHA that already went green keeps reading green from the moment a new run starts until `e2e-report` posts at the end. Re-running all jobs, reopening, or adding full-e2e therefore leaves the pull request mergeable for the whole of a suite that may be about to fail. While the gate was a job, GitHub created a fresh non-success check run each time and the window did not exist. Claim the context at the start of `plan`, the way the fork lane already claims it in `resolve`, and let `e2e-report` conclude it as before. The call is same-repo only: a fork's `pull_request` token is read-only, so posting from there would 403 and fail the job on every fork PR — and that read-only token is what makes the whole commit-status design safe. Fork PRs get their opening status from the privileged workflow instead. The contract test grows a case pinning that each lane writes the context from two points, one of them pending. Both assertions on the same-repo opener are scoped to the `plan` job: the bare fork guard occurs on several steps in other jobs, so a whole-file "at least one" passed with the opener's guard deleted, pinning unrelated guards and reporting that as coverage. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
8068619732 |
test(ci): state the scope note's reasoning as transitive, not direct
The header claims the third silent path is unreachable because every job either carries the discarded-label guard or needs `plan`'s outputs. The `e2e` job does neither literally: it gates on the results of `finalize`, `build-talos` and `resolve_assets`, which are the jobs that read those outputs. The conclusion survives — a skipped `plan` still skips it, one hop further along — but transitivity is the part the claim rests on, and a reader checking the stated reason against `e2e` finds it false and has no way to tell whether the conclusion failed with it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
3722b97fb5 |
fix(ci): terminate the fork TIA file list so its last path is read
The privileged lane builds the changed-file list in JavaScript and hands it to `hack/select-e2e.sh` through base64. `Array.join` puts a separator between entries and none at the end, so the decoded file had no trailing newline — and the selector reads it with `while IFS= read -r file`, which POSIX defines as returning non-zero at EOF without a newline. The loop body never runs for that last path, so it is silently dropped. For a single-file pull request, the modal fork contribution, that is the whole list: the selection comes back empty, the Chainsaw step is skipped on it, the job concludes success and the required status is published green with no suite executed. With more files the last one is dropped instead, which under-selects — and it can also suppress an escalation, since a path the selector does not recognise inside packages/ is exactly what would have widened the run to the full suite. Nothing in the log gave it away: the step prints the list with `sed`, which renders an unterminated last line the same as a terminated one, so the log showed a path the selector had never seen. Terminate the list where it is built. The in-tree lane is unaffected and always was — it feeds `git diff --name-only`, whose output ends with a newline — which is why this is the first caller to meet the behaviour. The read loop itself still drops an unterminated last line for any future caller; fixing that belongs with the selector and its own test surface, not here. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c20d47a245 |
feat(kubernetes): let a node group log the guest serial console
A worker VM that stalls during boot before Talos apid answers is invisible to every diagnostic the platform has: no Node registers, no certificate request arrives, and talosctl needs the apid that never came up. The guest serial console does show that window, but KubeVirt attaches its guest-console-log container only when the VMI asks for it, and the platform turns console logging off cluster-wide. Add an opt-in per-node-group logSerialConsole that sets the KubeVirt field, which takes precedence over the cluster-wide setting. The key renders nothing unless true -- including when set to false -- because the KubevirtMachineTemplate is content-hash named, so a byte emitted for a group that did not ask for it renames the template and rolls every live worker VM. The description carries the two costs the field name does not show. Changing the value rolls the group it is set on, so the stuck VM whose console was wanted is replaced along with the rest and the setting buys visibility into the next boot rather than the current one. And the cluster-wide default it overrides exists because that same container has been seen holding virt-launcher in PodInitializing, which would take a group down rather than explain it. Worker VMs are emitted by two charts. Both carry the field, and the golden-parity check gains a case that keeps their output identical. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
f4dc5d4d3c |
fix(harbor): retry the nested system install instead of uninstalling it
The harbor app chart renders a nested HelmRelease that owns the whole workload: the harbor Deployments, the inline RedisFailover and its volume. It carried Flux's default install strategy, and install remediation is an uninstall in every case Flux offers, so a first install that went over budget was torn down between attempts and the next one re-provisioned the redis volume and re-ran the master election from zero. That is reachable on a loaded runner. harbor-core blocks on redis while redis waits for its volume and its master election, and the nginx Deployment reaches the default progressDeadlineSeconds of 600s, which Flux reads as a stalled resource and fails the install early. The e2e wait budget for the release is the same 600s, so one teardown cycle consumes all of it and the second attempt cannot finish however fast it converges. RetryOnFailure keeps the manifests applied and retries the failed release, so the convergence already under way continues instead of restarting. Both actions carry the strategy because the controller performs the retry of a failed install as an upgrade, so on install alone the retry's own failure falls through to upgrade remediation. The cost is that a real upgrade failure retries rather than rolls back, which gives up no stable safety net: with retries: -1 and the default strategy, upgrade remediation already rolled back and retried without end. The remediation blocks stay because an absent release consults install and an out-of-sync one consults upgrade, both by retry count rather than by strategy. The change lands on the nested release rather than on the parent: the parent is built from the Application by the apiserver, which already stamps this strategy on both actions, while the nested release is the one that owns the workload and the volume a remediation deletes. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
7696d2b98c |
test(ci): record which silent path the concurrency pin does not cover
The contract asserts that the conclusions routed into a per-run group match the ones `resolve` posts nothing for, which reads as covering every case where a run stays silent. It does not. `resolve` also returns silently when the triggering run succeeded but its `Plan build` job skipped, and such a run sits in the shared group where it can still cancel the live publisher. That path is unreachable today, because every job in pull-requests.yaml either carries the discarded-label guard or needs `plan`'s outputs, so a skipped `plan` yields a skipped run and the conclusion check catches it first. Adding a job that runs independently of `plan` would reopen it, and the extraction here matches on the conclusion constant and would not notice. Say so in the header rather than leaving the next reader to infer coverage the pin does not provide. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
fd06fb32e0 |
docs(ci): say what fork TIA hardening buys, and what it does not
Four comments and a documentation bullet claimed that routing the changed-file list through the base-repo API stops a fork from making test selection pick nothing. It does not. The `e2e` job runs from the fork's own tree, so `hack/select-e2e.sh` is the fork's script: a PR that edits it to print nothing gets `skip=true`, the suite step never runs, the job succeeds and the required status goes green — the exact outcome the paragraph presented as prevented. The same holds for the suites under `hack/e2e-chainsaw/`, the install bats and the testing package's recipes, all of which the fork authors and this job executes. Hardening the inputs is still worth doing, since a merge-ref diff would be a second bypass on top of the one that exists, so the code does not change. What changes is the claim: the inputs are trusted, the decision is not, and the fork lane is a correctness gate against an honest contributor's regression rather than a boundary against a hostile fork. State plainly what does contain a hostile fork — the maintainer approval required before any fork run executes, the absence of any credential in that job, and a diff editing the harness being visible in review. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
6309eff784 |
fix(ci): paginate the fork PR association lookup
The check that the head SHA carries exactly one open pull request is a check on list completeness, and it was reading a single unpaginated page. GitHub returns 30 associations by default, so the one shape the check could not see was the shape it exists to catch: a second open PR sitting past the page boundary. Route the call through `github.paginate` like the three other lookups in the same script, and read the result as the array paginate returns rather than as a response envelope. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
6cfeaddf78 |
test(hack): clean up the Talos diagnostics tests without an EXIT trap
Eight of the ten tests in this file removed their scratch directory from a `trap ... EXIT`. Such a trap replaces the one the bats binary installs for its own bookkeeping, so a test that fails prints no TAP line at all: not `not ok`, nothing. The run ends with a warning that it executed nine of ten tests, and anyone reading the tail of the output, or grepping it for `not ok`, sees a suite with no failures in it. Break one assertion in this file and the difference is the whole point. Before: `1..10` followed by `# bats warning: Executed 9 instead of expected 10 tests`, with no line naming the test that failed. After: `not ok 2 reader Certificate is short-lived and never embeds credential data`. Remove the directory at the end of each body instead. Both runners set -e, so on failure the removal is unreachable and the directory survives for inspection, which is what a failed test wants anyway. The one test that runs the orchestrator from inside its scratch directory returns to where it started first -- not because rm would refuse, since BSD, busybox and GNU rm all remove a directory from inside it and return 0, but because the runner's own bookkeeping still runs in that subshell after the body returns, and would be doing so on an unlinked cwd. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
575b41e1cd |
test(hack): let each bats file declare the EXIT traps it still holds
The ban on EXIT-trap cleanup in hack/*.bats was enforced by two lists inside hack/cozyreport.bats: one naming the files known to be clean, one freezing an exact trap count for each file that was not. Both had to be edited from whatever change moved them, and that is where the guard kept failing. A change adding a trap to its own file had to edit a string in a suite it otherwise never touches, so two changes sharing no line still invalidated each other: each stayed green against its own base, git merged both cleanly, and the guard went red only once the second landed. A file arriving with traps hit the same wall from the other side -- the inventory did notice it, since the string it compared was built by scanning the directory, but absorbing it meant an edit in a file nothing in the author's diff pointed at. Replace both lists with a declaration each file makes about itself: one "# EXIT-TRAP DEBT: N" comment, exact rather than a ceiling. A file carrying none must install no EXIT trap, which covers the converted files, the files that never had one, and every file added later. Growing or shedding a trap now fails in the file the change already edits, so two changes that disagree about a count collide textually instead of silently. The declaration is read from the leading comment block, not from anywhere in the file. A .bats file is shell that writes shell, so the same line turns up inside a heredoc, a fixture writer or an expected-output string, where it is data belonging to one test. Reading it there as a statement about the whole file would let an unrelated fixture excuse a real trap, and would do it silently, since nothing in that test's own diff looks like a declaration. A comment block is the region with no interior: stopping instead at the first @test would still read a line out of a helper's heredoc. Not every counted handler is debt. A trap inside an explicit subshell does not replace the bats binary's own, so a test failing inside `( ... )` still prints its `not ok`. hack/e2e-test-openapi.bats kills a backgrounded kubectl proxy that way, and moving the kill to the end of the body would leak a process holding a fixed port. Its declaration records that rather than scheduling a conversion, and because the ratchet is exact, removing the trap fails too -- the count protects the construct. What the count cannot do is tell the two apart: substituting a test-level trap for the subshell one keeps the total at 1 and stays green, which the header states rather than leaves to be found. Counting bounds the keyword and the signal the same way, at any character that cannot be part of an identifier, and matches the signal in either case. Whitespace on the right missed `trap ... EXIT; cd "$tmp"`; whitespace on the left missed `tmp=$(mktemp -d);trap ... EXIT` and `(trap ... EXIT; true)`; upper case missed `trap ... exit`, which bash and dash both install. All are real handlers that scored zero, and the left boundary matters most, since the inventory being replaced had none and caught the semicolon form. A bare word boundary is not enough either way: `bootstrap ` ends in `trap `, and it must keep scoring nothing. A quoted signal counts for the same reason as the rest. Two handlers sharing one line are reported rather than counted, since the count is a count of lines and the second would otherwise arrive without moving the total; splitting the line properly needs a shell parser, a semicolon inside a handler's own action not being a separator, and guessing wrong undercounts -- the one direction a ratchet cannot afford. The scan recurses, so hack/e2e-apps/*.bats is covered rather than sitting one directory below the guard that claims the tree. It reads .bats and nothing else, so a handler arriving through a sourced .sh stays outside it -- hack/e2e-chainsaw/_lib/run-kubernetes.sh installs two, each benign for its own reason rather than by design: one sits in a function declared with `(` and so runs in a subshell, the other in a brace function no @test calls. That boundary is stated in the header rather than papered over. The guard moves to hack/bats-no-exit-trap.bats: its subject is every unit suite under hack/, not the report collector it grew up in. Its fixture helpers assemble the trap keyword and the signal from separate arguments, because the guard scans its own source and a fixture written as a literal would be counted as a real trap in it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
999aee5638 |
test(ci): count occurrences in the gate contract, not matching lines
Six assertions ran `grep -c` against a variable holding a single line, so `-eq 1` only asked whether the pattern appeared at all. A key naming the same field twice — two run-id interpolations, two branch components — satisfied the pin exactly as the correct key does. Demonstrated on the run-id assertion: duplicating the interpolation leaves `grep -c` reporting 1 and passing, while counting occurrences reports 2 and fails. Use `grep -o … | wc -l` for the single-line pins. The two whole-file `cancel-in-progress` counts stay on `grep -c`, where counting matching lines is the intended question. This is the same correction made once already in the sibling contract suite; the form did not survive being copied. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
9022d352c9 |
fix(ci): derive the fork image allowlist before loading the push token
Two comments described the privileged push job as running nothing but fixed binaries over data, never `make`. The allowlist derivation added later runs `make -n` once per image-building package — 32 on the current tree — and it sat after the OCIR login, so the credential was live for all of them. The property still held in substance, since those Makefiles come from the base tree rather than the fork, but it was a narrower property than the one written down, and the written one is what a future reader would try to preserve. Order the derivation ahead of the login. It needs nothing the login provides, and above it the make expansion cannot see the push token at all, which makes the stronger claim true rather than merely arguable. Restate both comments to say what now holds: no fork-authored script in the job at all, one base-tree `make -n` before the credential loads, fixed binaries only once it has. Drop `yq` from the list of binaries the push job runs — yq is used in the e2e job, not this one. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
260ce359ac |
fix(ci): resolve the fork PR only when the head SHA has exactly one
A branch may have several open pull requests at once, against different base branches, and they all sit on the same head commit. `resolve` took the first open association and derived everything from it — the merge ref it checks out, the docs-only verdict, the file list TIA is scoped from, the full-e2e label — while writing the `E2E Tests` status to the head SHA every one of those PRs shares. A verdict computed for one of them therefore satisfies the others, and a PR whose own base sees code can be greened by a sibling whose diff is entirely under docs/, with no suite run. The `HEAD^2 == head_sha` assert further down cannot notice: it holds for every PR on that commit by construction. Require exactly one open association and fail closed otherwise, naming the count in the status description so the state is diagnosable without opening the run. The zero case, previously reported as "no open PR", folds into the same check. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
e2643ef4b9 |
docs(ci): explain the fork image allowlist and fix a test comment
Two statements that were not true of the code next to them. The fork `publish` job accepts only archive names it can derive from the base parent, so a fork cannot name an artifact into an arbitrary registry repository. The consequence is that a fork PR adding a new image fails on `image '<name>' is not in the base-tree allowlist` with no way to pass, and nothing said so — including the E2E section that AGENTS.md points a reader at when a fork run fails. Record the constraint, the exact log line, and the two ways around it, so the failure reads as the guard working rather than as a broken PR. The concurrency contract's comment on `code_lines` claimed a real grep error propagates out of the helper. It does under the bats binary, and not under `hack/cozytest.sh`, whose translator appends `return 0` to every line that is exactly `}` — file-level helpers included — so a helper's last command never decides its status there. Both runners still fail closed, because each test asserts its extraction is non-empty before comparing, so a swallowed error surfaces as empty input rather than as a pass. Say that instead. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
7c5e96947e |
fix(ci): stop two runs from publishing "E2E Tests" on one SHA
Giving every `labeled` event its own concurrency group stopped a discarded label from cancelling a live run, but it also stopped a `full-e2e` label from doing so — and that label is not discarded. `plan` proceeds for it and `e2e-report` publishes with it, so the labelled run and the head run it was meant to widen both post the `E2E Tests` context, on the same head SHA, with no ordering between them. That was a deliberate trade at the time: commit statuses are latest-wins per context, so the reasoning went, the full-suite verdict lands last whenever its suite finishes last, and the label is an explicit opt-in to a longer run. This commit reverses that decision, because the premise only holds while the full run succeeds. The full suite is the superset, so a full-e2e run that fails early — a build flake, a suite failing at minute twenty — finishes before the narrower head run it was meant to widen, and that run's later green erases the failure. The required status then reads success for a SHA whose suite did not pass. One publisher per head SHA is worth more than the minutes of runner time a cancelled narrow run costs. Move aside only the label events `plan` discards, by making the concurrency condition the exact complement of `plan`'s guard. A `full-e2e` run stays in the main group and supersedes the head run the way a push does, so exactly one publisher of the context is alive per head SHA. Opting into the full suite now costs one suite instead of two, and a second label landing afterwards can no longer cancel the full-e2e run, because the two are no longer grouped together. The contract test gains the complement — the group key must exclude the publishing label rather than every label — and a check that the label name is one and the same string in all three guards that read it, since renaming it in one place splits the decision without touching a line that reads load-bearing. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
3c50a09457 |
fix(ci): keep a discarded run out of the fork e2e gate's group
The privileged fork e2e run groups by fork repo + branch with cancel-in-progress, so every completed Pull Request run for that branch lands in the same group -- including a run that built nothing. Labelling an open fork PR produces exactly that: pull-requests.yaml triggers on `labeled` and `plan` discards every label other than full-e2e, so all jobs skip. The privileged run for that event cancels the one that is mid-suite on the same head SHA, then `resolve` recognises the discarded trigger and posts no status at all. The required "E2E Tests" context is a commit status, so the head SHA is left at "Expected" with nothing left to flip it -- the run that replaced the owner is the very one that decided to stay silent -- and the PR stays unmergeable until the contributor pushes again. Key the group by the triggering run's id when that run concluded `cancelled` or `skipped`, so a run that will post nothing can cancel only itself. Real runs keep superseding each other on fork repo + branch. The contract test ties the conclusions in the key to the ones `resolve` stays silent for. Those two lists sit in different languages in different halves of the file, and a third conclusion added to the guard alone would restore the wedge without touching a line a reviewer reads as load-bearing. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
353b8a1a6d |
ci(backport): skip a target whose backport already merged
A second label event on a PR whose first backport already merged (for example backport-previous added shortly after backport merged) re-enters the backport job for the target that already has the change. The action's branch name for a backport is deterministic, so that leg's cherry-pick is empty rather than conflicting, and git still exits non-zero for an empty pick. Under this repository's draft_commit_conflicts setting the action reads that exit code as a conflict and tries to commit an empty working tree, which also fails and surfaces as a false backport-failed comment on the original PR for a branch the change has been on for a while. Add a guard step that looks up whether a merged pull request already exists on this target's deterministic backport branch, and skip the checkout and the backport action for that leg when one does. The guard changes nothing about the empty-cherry-pick behavior itself, which lives in the action and needs a fix there; it only keeps a known-stale re-run from reaching the action and misreporting. Extend the existing backport.yaml contract tests with the new guard's invariants: the merged (not merely closed) check on the deterministic branch name, both downstream steps gated on the guard's output, and step ordering. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
605030b5a7 |
test(backupstrategy-controller): anchor the metrics-absence guard to its container (#3605)
## What this PR does The `metrics.enabled=false` case in `deployment_test.yaml` guards against the `metrics.enable` typo regression by asserting that the metrics arg and port are absent. Both assertions address the controller by index, and `notContains` passes whenever its path resolves to something that lacks the forbidden content. A different container at that index qualifies, so a sidecar placed ahead of the controller leaves the guard green while the flag it exists to forbid comes back on the real container. This adds an `equal` on `containers[0].name` ahead of the two negatives. It fails when the index stops pointing at the controller, which is the case the negatives structurally cannot see. Verified rather than argued. Rendering a sidecar ahead of the controller against the current tests leaves the case green and reddens only the two neighbouring tests, which use positive `contains` and defend themselves; the guard that exists for this scenario is not among them. With the anchor, the case fails on the anchor. This is not a new convention. `packages/system/opensearch-operator/tests/leader_election_test.yaml` and `packages/system/dashboard/tests/gatekeeper_test.yaml` already anchor index-addressed negatives the same way, the second with the reasoning spelled out in a comment. This brings one more package in line with what the tree already does. ## What this PR does not do It does not switch the assertions to a name filter. helm-unittest resolves `containers[?(@.name=="...")]`, but `notContains` against a filter that matches nothing passes just as quietly, so that trade swaps "wrong container" for "no container" inside the same family of silent green. It does not touch the other two cases in the file. Both are positive `contains`, both went red under the same injection, so they anchor themselves. ## Tests The change is itself a test fix. Package suite is green: 11 tests, 3 suites. ### Screenshots None. This changes a test file. ### Downstream repositories Walked the trigger map against the diff. One helm-unittest file, nine added lines, no chart output changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved deployment validation by confirming the correct container is targeted when metrics are disabled. * Added checks that metrics-related arguments and ports are absent in this configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
0ba567dd73 |
fix(tenant): bound the pre-delete cleanup waits and the hook Job (#3598)
## What this PR does The tenant pre-delete hook deletes the tenant's HelmReleases in two waves with `kubectl delete --wait=true` and no `--timeout`. kubectl reads `--timeout=0` as "wait forever" and substitutes a week, so a wave hangs for as long as any child HelmRelease cannot finalize. helm then returns from the uninstall without deleting anything: the release stays in storage, flux keeps the finalizer, and the same teardown gets retried. The delete policy carries no `hook-failed`, so the waiting Job survives between attempts too. Each wave now gets 90s. Neither is guarded, on purpose. Nothing stands behind these deletes, so a wave that did not finish ends the run instead of reporting a cleanup that never happened. That also keeps the order the two waves exist for, since the tenant modules are what the applications run on. Whatever got deleted stays deleted, so the next attempt resumes from there. The Job gets `activeDeadlineSeconds: 270`. It bounds what the waits do not, since the DELETE calls carry kubectl's default `--request-timeout` of 0, and it sits under the 300s uninstall budget flux applies when the HelmRelease has no timeout of its own. That is the budget for tenants cozystack-api generates; `tenant-root` is the exception, a static release in `cozystack-basics` carrying `timeout: 15m0s`, and the ordering holds there too since 280 is under 900. The 30s gap is not cosmetic. The deadline runs from the Job's `.status.startTime` while helm's budget runs from before the Job is created, so at equal values helm gives up first, on a Job that is still running. That is the state this change is meant to end, and it would look fixed, because a number would be there. `terminationGracePeriodSeconds: 10` is part of the same arithmetic. Since 1.31 the Job controller holds back the terminal condition until every pod has finished terminating, so what helm waits for is the deadline plus the termination. At the 30s default that lands on 270+30, level with the budget, which is the gap the deadline was set to buy. That period gets spent in full, every time. The SIGTERM goes to PID 1, which is the shell: the script runs several commands, so the shell never execs into kubectl and stays PID 1 itself. A signal carrying its default disposition is discarded by the kernel when the target is PID 1, and a non-interactive shell installs no SIGTERM handler, so nothing acts on it. kubectl is a child and was not signalled at all. The pod ends on SIGKILL. Nothing in that script needs to finish gracefully, the DELETE calls are already sent and the process is only watching. The pod also gets resource requests, matching the sibling hook in `packages/apps/kubernetes`. Without them it is BestEffort and first to go under node pressure, which mattered less when a retry inside the Job absorbed an eviction. With a single attempt it does not. `backoffLimit: 0` and `restartPolicy: Never` go together, and neither implies the other. At the default limit of 6 the Job does not reach `Failed` inside the budget, so helm falls back on its own timeout with the same nameless "timed out waiting for the condition" as before. Under `OnFailure` the pod survives its own failure and the kubelet re-runs the container in place, and the Job controller finds out afterwards by summing `restartCount`. The script starts over, re-enters the applications wave that just expired, and gets cut down mid-run when the controller catches up. `Never` puts a failed pod in front of the controller directly. ## What this PR does not do No remedy behind the bounded waits. The `packages/apps/kubernetes` hook force-clears child finalizers when its own wait expires. That needs `patch` on HelmReleases in the Role, and it is only safe there because the whole tenant control plane is going away in the same run. Here a wave that expires ends the attempt. `packages/apps/kubernetes/templates/delete.yaml` is untouched. It does not make a doomed teardown converge. A tenant whose children cannot finalize still gets its uninstall retried. What changes is that each attempt ends in bounded time, instead of eating the whole uninstall budget and leaving a pod behind. 90s is a guess, and it is tight. The first wave deletes every application HelmRelease in the namespace, including kinds that are slow to tear down: a nested Tenant runs this same hook one level down before its release finalizes, and an app of kind Kubernetes tears a control plane down. Either one can outrun 90s. The bound is what is left once the Job deadline sits under helm's budget instead of level with it, split between the two waves. The 90s the deadline keeps back for scheduling and the image pull is not calibrated either, and I would rather say so than imply it was. `docker.io/clastix/kubectl:v1.32` comes from Docker Hub with no mirror in front of it, and this repo has measured Docker Hub pulls of a smaller image at about 1m42s in CI. So that remainder is roughly the observed worst case rather than comfortably above it. A cold pull that slow lands on the `activeDeadlineSeconds` path, which is the one that deletes the pod and leaves nothing to read. There is a band that pays for this, and I would rather name it than round it off. Neither shape above fits inside 300s today, so for them the bound costs nothing that works. But every generated child HelmRelease carries a wait strategy, so a wave inherits its slowest child, and a wave that settles in something like 95s does complete inside the budget today. After this change it expires instead, and that tenant trades one clean attempt for an extra flux retry. In wall clock that retry is about five minutes, since `helmrelease-interval` defaults to `5m`, so a tenant in that band takes noticeably longer to delete than it does now. I think that is the right side of the trade: an unbounded wave does not fail, it spends the whole budget and leaves helm reporting a timeout that names nothing. If the band turns out to be populated, the waits and the deadline move together, and the bats suite pins their ordering rather than their values. One caveat on the diagnostics, since the comments in the chart used to promise more than they deliver. The failing wave leaves its output in the failed pod, which `ttlSecondsAfterFinished: 300` then reaps while flux's uninstall backoff is still climbing. And on the `activeDeadlineSeconds` path there is no log at all, because the Job controller deletes the active pods when the deadline expires. That path is the backstop for a kubectl that never returns; the expected path is the wave's own `--timeout`, where the script exits on its own and the pod survives to be read. ## Tests `hack/tenant-pre-delete-hook.bats` renders the chart, pulls the hook script out of the Job and runs it against a stub `kubectl` on `PATH`. It covers a clean run reporting success, every delete reaching kubectl with a non-zero bound, the ordering of the four numbers, and a wave that gave up ending the run without going on to the next one. `--timeout` is implemented inside kubectl, so no stub can make that deadline actually elapse. The tests read the arguments that reached kubectl and inject the non-zero exit an expired delete produces. No EXIT traps. The ordering test reads the numbers out of the rendered Job instead of pinning them, so moving a value keeps it honest while a value that breaks the ordering still fails. An absent deadline or termination period is rejected explicitly, otherwise `null` satisfies the numeric comparison and a Job with no bound at all passes. The Job's own shape is pinned in the chart's helm-unittest suite, since the script cannot observe that about itself. Both suites are green locally, and I mutated each pinned value to confirm they fail for the right reason: dropping either `--timeout`, dropping `activeDeadlineSeconds`, dropping `terminationGracePeriodSeconds`, dropping the requests, `backoffLimit` back to 6, `restartPolicy` back to `OnFailure`, and a `|| echo` appended to the applications wave. Each one goes red in the test that claims that ground. Worth flagging so the check marks are not read as more than they are: `make bats-unit-tests` is red at the merge base, unrelated to this branch, and the loop it runs exits on the first failing file. `hack/cozyreport.bats` sorts ahead of this suite, so on CI the new bats file is not reached at all right now. It is not failing, it is not running. The helm-unittest cases are unaffected and do execute. ### Screenshots None. This changes a Job manifest and its tests, with no user-visible surface. ### Downstream repositories Walked the trigger map against the diff. The diff is one Job manifest inside the tenant chart, its helm-unittest suite, and a new `hack/*.bats` file. No package is added, renamed or removed; no `values.schema.json`, `values.yaml` default or version enum changes; no `ApplicationDefinition` semantics, namespace, variant or release asset changes; nothing under `hack/` is moved or renamed and no make target changes behaviour, since `bats-unit-tests` already globs `hack/*.bats`. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(tenant): bound the pre-delete cleanup hook's HelmRelease deletions and give its Job a wall-clock deadline, so a tenant whose children cannot finalize no longer leaves the hook waiting indefinitely and blocking the uninstall ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Reliability Improvements** * Tenant cleanup jobs now enforce execution deadlines and terminate gracefully. * Cleanup operations have bounded deletion timeouts and no automatic retries. * Failed cleanup steps now stop subsequent operations and correctly report failure. * Cleanup jobs now specify CPU and memory resource requirements. * **Tests** * Added comprehensive coverage for cleanup behavior, timeouts, failure handling, and job lifecycle settings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
954be15341 |
fix(e2e): bound every read in the report collector and the step that runs it (#3596)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does A single hung read in `hack/cozyreport.sh` did not truncate `cozyreport.tgz`, it lost the artifact outright. The collector writes its tarball on its last line, the `Collect report` step carried no `timeout-minutes` of its own, and `Upload cozyreport.tgz` is the step directly after it — so a read that never returned spent whatever was left of the job budget, the job was cancelled, and the upload never ran. The runs where the report is the only evidence left are the runs that produced none. `|| true` guards against a non-zero exit, not against a read that does not return. Two halves, independent of each other. **The step gets a ceiling of its own**, 30 minutes, in all three workflows that run it. The number is measured against the step rather than divided out of the job: pulling the `Collect report` timings out of recent e2e runs gives 3 to 42 seconds, on failed jobs as much as on green ones, and 30 minutes leaves five sixths of the job budget for the steps that upload what this one produced. It carries `continue-on-error: true` beside it, because the step's `|| true` is its standing statement that collecting diagnostics is never fatal and a step killed at its ceiling never reaches that `|| true` — without the tolerance, a passing run whose collection ran long would be published as failed, and every later `failure()` in the job would start meaning "the tests failed or the report was slow". That tolerance also renders the overrun green, so a following step turns the collect step's `outcome` into a `::warning::` annotation: a bound that fires is recorded, and the outer bound is not exempt from the rule the inner ones follow. **Every read in the collector is bounded**, against four sections before, and gates and host commands are reads too. An `if kubectl get deploy …` that decides whether a module runs costs more unbounded than anything it guards, because the module below it never starts; `df -h` blocks indefinitely on a stale mount and takes the tarball with it by exactly the mechanism above. A gate cut off short is recorded, since skipping the module leaves the same empty tree as a cluster that never ran the component, while a gate answering "not installed" is recorded nowhere — that is the cluster answering correctly, and a marker firing on every install without KubeVirt is one a reader learns to skip. The six LINSTOR reads run a command inside a pod and go through a reader of their own. They are the only reads whose duration a workload sets rather than the apiserver, and `kubectl exec` returns the remote command's exit status — a 124 there is either the collector's deadline or a command that itself exited 124, so the existing classifier would have named this script's own timeout as the cause of something that may have happened inside a pod. The new note states what the status leaves open, and sits beside the output rather than inside it, since two of the six stream a gzipped tarball. A cut-off bundle is kept rather than dropped, and the note says so: the emptiness check at those call sites asks whether the archive lists a first member, which a truncated gzip does. What this does not do is bound the report. The pod, VM, VMI and services walks carry a count cap and a time budget; every other per-object walk carries neither, so their cost is the cluster's to set, and a failed install with a hundred not-Ready HelmReleases spends four hundred bounded reads in one of them. A per-read ceiling removes the read that never returns, not the thousand reads that each return slowly. The file header says so where it previously implied the opposite, and the step ceiling is what covers the rest. Two frozen lists in the tests are gone, replaced by one invariant derived from the code: no read below the library guard runs without a ceiling. Both lists were already wrong. The header named four bounded sections while the file had ten, and the per-section counts of unbounded reads only ever matched a redirect into `$VAR/`, which no gate, selector or command substitution has — so they described the scan rather than the file. The guard that held the old exec assumption was satisfied by leaving every exec unbounded, which is the state the file was in. It now also requires each exec to be bounded, and requires that reader not to borrow the classification written for reads whose 124 is unambiguous. Matching on the verb rather than on the string `kubectl exec` is part of it: half of these calls spell it `kubectl -n cozy-linstor exec`, which that string never saw. Two properties worth knowing before this lands. `kubectl get events -A --sort-by=.lastTimestamp` sorts client-side, so kubectl buffers the whole event list and writes nothing until it finishes — under a flat per-read ceiling a cut-off there leaves a file holding only the marker, not a short tail. That read had no ceiling at all before, where the same slowness could take the whole tarball, so this is a narrower failure rather than a new one. And `kubectl logs -n cozy-kamaji deployment/kamaji` gains `--tail=2000`, matching every other controller log in the file: it was the only unbounded-in-size log, which under a wall-clock ceiling is the one most likely to be cut at its *end*, where a control-plane bootstrap failure sits. Longer kamaji history than 2000 lines is no longer collected. Two accepted imperfections in how the change is split across commits, named rather than left to be found. `docs/agents/e2e-testing.md` keeps this material in a single shared paragraph that all three commits touch, so in the middle commit one sentence about the module gates lands a commit ahead of the gates themselves. And two corrections to the exec reader — it now keeps the remote command's whole message in the listing rather than quoting the line `kubectl exec` always appends last, and its cut-off note names both of the two things that can happen to a partial bundle instead of asserting one — sit in the last commit rather than in the one that introduced that reader. Both are true of the branch tip, where every statement is correct and appears once; the cost of moving them is another rebase across the one shared paragraph, which is not worth what it buys. Refs #3568. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(e2e): every read in the diagnostic report collector now carries a wall-clock ceiling — module gates, the reads that run a command inside a LINSTOR pod, and the host commands included — and a ceiling that fires is recorded in the artifact rather than only in the job log. The workflow step that runs the collector carries a ceiling of its own, so a read that does not return costs one file and a warning annotation instead of consuming the job budget and losing cozyreport.tgz entirely. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added bounded diagnostic collection across Kubernetes, Talos, LINSTOR, host, controller, and command outputs. - Reports preserve partial results and identify timeouts, failures, missing output, and truncated artifacts. - Added warnings when report collection fails or exceeds its time limit. - **Bug Fixes** - Corrected resource targets, selectors, deployment extraction, and command matching. - Improved handling for empty selections, unavailable resources, and missing timeout tools. - **Documentation** - Expanded guidance for diagnostic markers, collection limits, timeouts, and incomplete output. - **Tests** - Added coverage for bounded reads, failure reporting, markers, selectors, logs, and subprocess mocking. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
0b7a8f67d1 |
fix(apps/kubernetes): report the pre-delete cleanup outcome honestly (#3592)
## What this PR does The pre-delete cleanup Job for the `kubernetes` app ended with an unconditional `echo "Cleanup completed successfully"`. Several of its steps swallow a failure behind `|| echo "WARNING: ..."` so that `set -e` cannot end the run, which means that line was printed in exactly the situation those warnings exist to report: child HelmReleases still holding Flux finalizers, or a Kamaji datastore Secret still pinning its namespace. Someone reading the Job log to find out why a namespace will not delete was told the cleanup had worked. The Job now records what a step left behind and closes with either the success line or the list of steps that left something. Only residue gets recorded. A bounded delete that times out has a remedy right behind it (Step 2 falls through to force-clearing finalizers, Step 4 falls through to Step 4b), and whether anything survived is decided by that remedy, which reports its own outcome. A tenant with no working nodes takes that route as a matter of course, which is the reason the wait is bounded at all, so recording the timeout itself would report a finished teardown as incomplete on the commonest path there is and send an operator hunting for objects that are not there. That is the same defect as the one being fixed, pointing the other way. A read that fails looks exactly like one that found nothing. An empty list and a failed list both give a loop nothing to iterate; an absent object and an unreachable apiserver both make a probe non-zero. So the whole script was walked for that shape instead of patching the instances as they were reported. Four reads reach the closing line. Step 2's force-clear re-list now keeps its status and reports when it cannot list what it was about to clear. Step 4b's two probes became three-way, present, absent or unknown, so a Secret whose existence could not be established is no longer announced as absent. Step 1's list is the one left swallowing its status, and that is deliberate: Step 2 deletes by label rather than from that list, so a suspend that never happened leaves nothing of its own behind, and reporting it would name residue that does not exist. Making the TenantControlPlane probe three-way also repairs the gate above it, and that is intended. The gate is there so the datastore Secret's finalizer is not stripped while Kamaji may still own it, and a two-way probe defeated it in the one direction that does damage: any error read as "confirmed absent" and the strip went ahead. Its likeliest cause, a throttled or briefly unreachable apiserver, is also likeliest exactly when a broken tenant is being torn down. Step 4b declining to strip a finalizer while the TenantControlPlane is still alive is the one residue that no failing command marks: every call in that branch succeeds, and the namespace stays blocked regardless. It now reports, and names the TenantControlPlane as the thing holding it up. What this PR does not do: the hook still exits zero, even with every backstop engaged. That is the difference between a main path and a backstop. `packages/apps/tenant/templates/cleanup-job.yaml` is the same kind of hook and handles it the opposite way, correctly: deleting the HelmReleases is that hook's whole purpose, so a failure there means the cleanup did not happen, and `set -e` ending the run before the success line is honest. The swallowed sites here are backstops. They run only after the normal deletion has already timed out, and they exist to push teardown further along. Failing there aborts the Helm uninstall before the release is removed, so the release stays in storage and the same doomed teardown is retried forever, at precisely the moment the backstop was written for. The steps that are not backstopped still end the run under `set -e` and never reach the closing message at all. This makes the message truthful. It does not make it reachable. The Job carries `helm.sh/hook-delete-policy: hook-succeeded`, and the hook exits zero by design on every backstopped run, so Helm reaps the Job, and its pod log, right after the hook completes. Whether the closing line survives long enough to be read is a question about the delivery channel, not about its wording, and changing that channel (an Event on the CR, or keeping the Job when something was left behind) has a different blast radius. Left for a separate change. The step comment and the runtime warning both cited the tracking issue for the Kamaji datastore-secret leak as though it were live. It is closed, and closed by this step rather than by a fix in Kamaji, so following the pointer showed a resolved report and no reason for the code to exist. The comment now says the ordering it covers is still reachable, and the warning names the finalizer and the object to clear by hand instead of an issue number. The workaround itself is untouched: nothing indicates the upstream cause has gone away. `hack/kubernetes-pre-delete-hook.bats` renders the Job, extracts the script and runs it against a stub `kubectl`, covering a clean run, a fallback route that succeeded, a TenantControlPlane outliving Step 4, each probe that could not answer, every remedy failing at once, an unbackstopped step ending the run, and the zero-exit invariant. The helm-unittest cases match the script's source text and cannot tell whether the closing message is reachable. Every case was checked by mutation, including reinstating the unconditional success line and reinstating the over-reporting the fallback case guards against. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note fix(kubernetes): the pre-delete cleanup hook no longer reports success when a cleanup step left something behind, and no longer reports a completed teardown as incomplete when it merely fell back to force-clearing finalizers. The Job log now names the steps that left residue. The hook still exits zero so the Helm uninstall can remove the release. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved Kubernetes cleanup handling during application deletion. * Cleanup now distinguishes unavailable resource checks from confirmed absence, avoiding unsafe finalizer removal. * Reports leftover resources when recoverable cleanup steps fail while still completing the deletion process. * Improved handling of child releases, control-plane resources, and datastore secrets. * **Tests** * Added comprehensive coverage for successful cleanup, partial failures, resource residue, probe errors, and unrecoverable steps. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
f4cde3054a |
docs(ci): correct the job-level permissions comment (#3591)
## What this PR does The comment above the top-level `permissions:` block said jobs request the minimum extra scopes on top of it. That's backwards. A job-level block replaces the workflow-level one, and [every scope it does not name is set to `none`](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#permissions). Write `packages: write` alone in a job and it loses `contents: read` and can't check the repo out. Same wording in every file that carried the old line. It matches what #3574 uses for the workflow it adds. `backport.yaml` and `pull-requests.yaml` also appear in open #3569. Its changed lines there are the `cancel-in-progress` expressions two lines below the comment, and a three-way merge is clean in either order. Comment-only: every changed line starts with `#`, the files still parse, and actionlint reports the same 45 pre-existing findings before and after. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff. The only row that names a file here is the ccp one on `.github/workflows/tags.yaml`, and it triggers on release-prep behaviour, which this doesn't touch. ### Release note ```release-note docs(ci): the comment above the top-level `permissions` block in the workflow files now says a job-level block replaces it instead of adding to it ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified workflow permission guidance across automation processes. * Documented that the default token permissions are read-only. * Clarified that job-level permissions replace top-level defaults and must explicitly declare all required scopes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
cab44d56ac |
test(tests): pin the keep annotation on cert-manager and etcd-operator CRDs (#3590)
## What this PR does
`cert-manager-crds` and `etcd-operator-crds` are the two packages this
repository points at as the in-tree precedent for protecting CRDs with
`helm.sh/resource-policy: keep`. Both carry the annotation, neither had
a test, so it could be lost with nothing reporting the loss. This adds a
suite to each, plus the `test:` target that `hack/helm-unit-tests.sh`
needs before it will run one. No annotation is added and no rendered
output changes: the point is to close the way of losing what is already
there.
The two are exposed differently, so the suites say so instead of sharing
one explanation. In `etcd-operator-crds` the annotation is ours. `make
update` stamps it with an awk step anchored on the
`controller-gen.kubebuilder.io/version:` line that the upstream CRDs
carry. If a future controller-gen stops emitting that line, the step
matches nothing, writes the CRDs out unstamped and exits zero.
In `cert-manager-crds` the annotation comes from the upstream template,
emitted under `{{- if .Values.crds.keep }}`, and the value that switches
it on is ours. That is two independent ways to lose it. The templates
are also not maintained in that package:
`packages/system/cert-manager/Makefile` deletes and refills its
`templates/` directory during its own `make update`, so a cert-manager
bump rewrites a neighbouring package that whoever runs the bump has no
reason to open.
Each suite is scoped with the wildcard `templates/*.yaml` and asserts
across every document rendered under it, rather than listing CRD names
or pinning a document count, so a CRD that upstream starts shipping is
covered without editing the test. Each also asserts the kind, because
the annotation assertion on its own would accept a non-CRD document that
happened to carry it. Both charts render only CRDs today, checked with
`helm template`.
The `test:` target is load-bearing rather than boilerplate.
`hack/helm-unit-tests.sh` runs a package's suite only when its Makefile
defines that target. With the target present, breaking a chart makes the
whole run exit 1 and name the directory. With the fixtures present but
the target removed, the same broken chart gives exit 0 and "All Helm
unit tests passed", having executed the suite zero times.
### Screenshots
Not a UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
I walked the trigger map in `docs/agents/contributing.md` against the
file list of this diff. It adds two test files and two `test:` targets
under `packages/system/`, and touches nothing else: no
`hack/package.mk`, no app `values.schema.json`, no platform values, no
`ApplicationDefinition`, no release asset names. The website's
"developer tooling (the package Makefiles)" trigger is the one worth
checking, so I read `content/en/docs/next/development.md` there: it
documents `make test` in general terms and does not go stale from a
package gaining that target.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added automated validation for cert-manager and etcd operator CRD
charts.
* Confirmed rendered resources are correctly identified as
CustomResourceDefinitions.
* Verified CRDs retain the required `keep` resource policy during Helm
operations.
* **Chores**
* Added convenient test commands for running Helm chart validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
177da7843d |
fix(kubernetes): keep tenant addon CRDs when a release is uninstalled (#3586)
## What this PR does Four tenant addon HelmReleases render their CRDs from a chart's `templates/` directory and remediate a failed install by uninstalling the release. An uninstall deletes what the chart rendered, and deleting a CRD makes the API server garbage-collect every custom resource of that kind, so a failed reinstall takes every Gateway, VerticalPodAutoscaler, ServiceMonitor or VolumeSnapshot in the tenant cluster with it. This annotates the CRDs of the four packages behind them, `gateway-api-crds`, `vertical-pod-autoscaler-crds`, `prometheus-operator-crds` and `vsnap-crd` (that last one is declared by `helmreleases/volumesnapshot-crd.yaml`), with `helm.sh/resource-policy: keep`, which is what `cert-manager-crds` and `etcd-operator-crds` already carry in this tree. Three of the four regenerate `templates/` from upstream on `make update`: two delete the directory outright and the third overwrites its file. So the annotation is written into the vendored files and reapplied by a step in each `update:` target. Without that second half the next regeneration drops it silently, with nothing reporting the loss. I ran `make update` for real to check the pair holds: `gateway-api-crds` and `vsnap-crd` come back byte-identical to what is committed here, and `vertical-pod-autoscaler-crds` reproduces the annotation on both of its CRDs. The rest of the VPA diff is upstream drift, because that package's `update:` tracks a branch rather than a pinned ref; the bump itself is not in this PR. The step is idempotent, so running it twice does not stack a second annotation into the same mapping. `prometheus-operator-crds` ships its CRDs in a vendored subchart that `make update` re-pulls, so nothing under `charts/` is edited. The upstream template already renders `.Values.annotations` onto every CRD it emits, and the annotation is set through the package's `values.yaml`. The Gateway API kustomization also emits a `ValidatingAdmissionPolicy` and its binding. Those hold no user data, so the stamping step selects on `kind: CustomResourceDefinition` and leaves them alone. Each package gains a helm-unittest suite and the `test:` target that `hack/helm-unit-tests.sh` requires before it will run a suite at all. Every suite asserts the annotation across all CRDs its chart renders, not only the ones listed by name, so a CRD arriving in a later upstream bump is covered without editing the test. The annotation has a cost worth stating plainly: uninstalling one of these releases on purpose now leaves the CRDs behind, to be removed by hand. That is the same trade-off `cert-manager-crds` already carries. This covers the first of the two directions in #3555. The second one, setting `install.strategy.name: RetryOnFailure` on these releases, is a separate decision with a different blast radius and is not part of this PR. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: I walked the trigger map in `docs/agents/contributing.md` against the file list of this diff. It touches four packages under `packages/system/` and nothing else: no `hack/package.mk`, no app `values.schema.json`, no platform values, no `ApplicationDefinition`, no release asset names. The one line that looked close is the website's "developer tooling (the package Makefiles)" trigger, so I read `content/en/docs/next/development.md` in the website repo: it documents `make update` and `make test` in general terms, and neither goes stale from this change. ### Release note ```release-note fix(kubernetes): tenant addon CRDs for Gateway API, VerticalPodAutoscaler, Prometheus Operator and VolumeSnapshot now carry `helm.sh/resource-policy: keep`. A failed install that remediates by uninstalling the release no longer deletes those CRDs and, with them, every custom resource of those kinds in the tenant cluster. In exchange, uninstalling one of these releases on purpose now leaves the CRDs in the cluster to be removed by hand. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Custom Resource Definitions for Gateway API, Prometheus Operator, Vertical Pod Autoscaler, and VolumeSnapshot are now retained when their Helm charts are uninstalled. * Prevents loss of custom resources and related configuration during chart lifecycle operations. * **Tests** * Added automated validation confirming all included CRDs receive the retention policy. * Added chart test commands to simplify verification of CRD updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |