mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 14:02:56 +09:00
c1da673b085a845a014db5b4c2f0e30bb205bd8a
60
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Andrei Kvapil
|
3dc0f56477 |
feat(kube-ovn): build the chart and image in-tree, retiring the wrapper repo (#3460)
## What this PR does Brings the `kubeovn-chart` wrapper into the tree, so `cozystack/kubeovn-chart` can be archived. Until now `packages/system/kubeovn` pulled a chart tarball from the latest `cozystack/kubeovn-chart` tag at `make update` time, and that repository also built and pushed the `kubeovn` image. The wrapper existed only to cache the upstream chart; with the current caching approach that reason is gone, and the split cost a second repository, a second release step, and a package whose provenance a reader cannot follow without leaving the tree. The package is now shaped like `packages/system/cilium`: - `update` pulls the upstream `kubeovn/kube-ovn` chart directly and applies `patches/mtu.diff`, then syncs the version into the Dockerfile. The MTU patch already carried monorepo-relative paths, so it applies unchanged. - `image` builds `images/kubeovn` (carrying the existing `fix-keepvmip-portgroup` patch) and pins the digest into `values.yaml`, using the shared `image-tags` / `cache-args` helpers. It also carries `cozystack/kubeovn-chart#5`, which never got a tag: `OVN_IPSEC_KEY_DIR` is pinned under `/var/lib`. The kube-ovn default sits in `/etc/origin`, which a Talos node cannot create, so enabling IPsec left the node without ESP security associations. ### The nightly host rewrite had to be fixed first Moving the image build here activates a gap recorded in `docs/agents/image-refs.md`. The nightly host rewrite is a literal `<src-registry>/` substring replace, and kubeovn keeps its host in `global.registry.address` with the bare repository in a sibling key — no trailing slash, no match. While the image came from a separate repository the baked reference carried the public host and the ownership filter skipped it; once CI builds the image here it stamps the build-registry host into that key, and the result is an image mirrored to the public registry while the published tree keeps pointing at the private build registry. The second commit adds an end-of-line-anchored expression that fires only when the source registry is the complete scalar value, so a contiguous reference — which continues past the host — is still handled by the first expression and the two can never both fire on one reference. `keycloak-operator` splits at a different boundary and stays unfixed; that is documented, and dormant while no CI path rebuilds it. Verified: `hack/nightly-mirror_test.bats` passes with the new case and the case fails without the fix; `promote-retag` and `promote-rewrite-tags` suites unaffected; `helm template` and `helm unittest` on the package pass. The pre-existing `helm lint` complaint about a missing `kube-ovn` dependency reproduces on `main` and is left alone. ### Follow-ups this PR does not do - `cozystack/kubeovn-chart` should be archived once this merges, with a pointer here. Its open PR #5 is carried by this change and needs no separate landing. - `cozystack/website` links the Kube-OVN license at `cozystack/kubeovn-chart` in `operations/configuration/licenses.md` and `guides/platform-stack/_index.md` (all versions, including `next`). Archived repositories stay readable so nothing breaks, but the link should move upstream or in-tree. **No follow-up has been opened yet** — flagging rather than ticking the box. ### Screenshots N/A — build and packaging change, no user-facing or UI impact. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff. The website row matches — not through the app-list or platform-component triggers, but because two of its pages link the repository being archived. Left unticked because no follow-up exists there yet; a human should decide whether the license link moves to `kubeovn/kube-ovn` or to this package. No other row matches: the change touches no `packages/apps/` or `packages/extra/` package, no `values.schema.json`, no `ApplicationDefinition`, no platform variant, and no node prerequisite. ### Release note ```release-note feat(kube-ovn): the Kube-OVN chart and image are now built in this repository instead of cozystack/kubeovn-chart. OVN_IPSEC_KEY_DIR now defaults to /var/lib/ovs_ipsec_keys, so enabling IPsec on Talos nodes no longer leaves them without ESP security associations. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Updated Kube-OVN image build to use upstream chart metadata and keep registry/version alignment during `update`. - Added conditional MTU support for the OVN CNI DS via chart values. - Introduced `OVN_IPSEC_KEY_DIR` to standardize the OVN IPsec keys directory. - **Bug Fixes** - Preserved VM networking port-group assignments during live migration when an alive virt-launcher sibling pod exists. - Improved nightly mirror host rewriting for split and standalone registry host formats. - **Tests** - Added coverage for the end-of-line “whole scalar value” host rewrite case. - **Documentation** - Refreshed “Known gaps” guidance for nightly mirror host rewriting behavior and limitations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
cc44f6c981 |
ci(release): move release validation to rc time — e2e lane, changelog button, promote gate, docs at promote (#3449)
## What this PR does Moves release validation from promote time to rc time, making promotion itself a pure retag operation. This is the follow-up to the v1.6.0 retrospective: the promote flow's checks ran on the promote PR (coupled to promotion, skippable by never promoting), the changelog and website docs were produced only after the release went public, and the finalize tag push silently failed to trigger `tags.yaml` at all (persisted-credential bug, root-caused below). The release lifecycle after this PR: 1. `cut-prerelease.yaml` cuts an rc; `tags.yaml` builds, drafts, pushes the staging branch — and now runs a **mandatory full e2e** for every `-rc.N` tag via the new reusable `e2e-tag.yaml` lane. Alpha/beta (and any published tag) get the same validation as a **manual button** (`workflow_dispatch`). 2. Optionally, a maintainer presses the **`changelog-rc.yaml` button**: the release changelog is generated from the rc tag (reusable `changelog-generate.yaml`) and committed to the rc staging branch, ready for review long before promotion. 3. `promote-rc.yaml` dispatch is **gated on green rc e2e** (evidence: the rc's `tags.yaml` run containing a successful `rc-e2e` job, or a successful manual `e2e-tag` dispatch for the tag; `skip_e2e_gate` input for emergencies, always surfaced in the PR body as ✅/⚠️). Promote stages the release, picks up the rc-time changelog (or generates one), and **opens the website docs PR** on cozystack/website from the staging branch — parked with an explicit DO-NOT-MERGE-until-finalize contract. 4. The promote PR **no longer gets `full-e2e` automatically** — its tree is the rc's already-validated digests with only tag strings rewritten; a maintainer can still add the label manually to force the suite. 5. Finalize performs the irreversible side effects (tag, publish, retag, chart) — and with `persist-credentials: false` its tag push now actually authenticates as the CI app, so the `tags.yaml` backstops (`generate-changelog`, `update-website-docs`) fire for promoted tags. This is the one-line fix for the v1.6.0 incident where the tag triggered nothing: the checkout's persisted `GITHUB_TOKEN` `http.extraheader` silently defeats the app token injected via `git remote set-url`, and `GITHUB_TOKEN` pushes create no workflow runs. Everything is pinned by executable contract tests (`hack/promote-gate-contract.bats`, extended `hack/release-changelog-contract.bats` / `release-changelog-behaviour.bats` — 48 tests total, including behaviour tests that execute the new `hack/parse-rc-tag.sh` and `hack/select-changelog-source.sh` for real), and `docs/release.md` is rewritten to describe the new lifecycle end-to-end. ### Commit map (review commit-by-commit) - `75b5192f1`…`dd308644c` + `6fc970851` — the pre-existing changelog-before-publication rework (changelog generated at promote, committed to the staging branch before the PR opens, finalize publishes it as the release body; tags.yaml job becomes a verbatim-port backstop), rebased onto main. - `5aec70b3a` — `e2e-tag.yaml`: reusable published-tag e2e lane (prerelease → staging branch, stable → tag; draft releases fail fast; dispatch-only SSH `debug` breakpoint; tag-suffixed artifacts). - `025564d96` — `tags.yaml::rc-e2e`: mandatory full e2e for freshly cut rc tags (also on `release_exists` re-runs); fixes the `cut-prerelease.yaml` comment that falsely claimed e2e already existed. - `2dfbeea77` — `changelog-generate.yaml` (reusable core) + `changelog-rc.yaml` (rc-time button committing to the staging branch); promote-rc gains a top-level `parse` job with a strict no-whitespace tag policy (`hack/parse-rc-tag.sh`); source selection is validate-then-fallthrough (`hack/select-changelog-source.sh`) so an invalid higher-priority changelog can never silently win. - `ff7ddd3c9` — the promote e2e gate + `skip_e2e_gate` override + removal of the automatic `full-e2e` label (pull-requests.yaml release-path e2e now requires an explicit label; a post-open label now actually starts the run). - `936190d4b` — `website-docs` job at promote time behind a FETCH_REF-support guard (see dependency below), with the DO-NOT-MERGE-until-finalize contract and a pin-freshness pass before `release-next`; `tags.yaml::update-website-docs` stays as the backstop. - `01e1e7188` — the `persist-credentials: false` finalize fix (v1.6.0 root cause). - `717c8860f` — `docs/release.md` coherence pass over the whole new lifecycle. ### Dependency The promote-time website docs job needs cozystack/website#631 (`FETCH_REF` + fail-loud fetch). Until that merges, the job's guard detects the missing support and degrades to a ⚠️ line in the promote PR body — promotion is unaffected and the tag-time backstop backfills, so this PR is safe to merge first. ### Merge instructions Please **merge-commit, do not squash**: the 13 commits are deliberately atomic (the credential root-cause fix is separable from the feature work) and squashing would collapse ~3k lines into one commit, hurting bisectability. ### First live run The e2e lane is verified by lint, contract tests and dry-traces, but has not yet executed against a real tag; the intended first live run is a manual `e2e-tag.yaml` dispatch against an existing published rc tag before the gate is relied on for a real promotion. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added reusable changelog generation with validation, “present” signaling, and safe artifact upload behavior (including RC-time support). * Added an E2E release-tag workflow with tag parsing, deterministic sandboxing, and comprehensive report collection. * Improved RC promotion with a stricter parse front-door, verified E2E green gate (optional bypass), and earlier parallel changelog/asset staging. * **Documentation** * Refreshed release-process docs to match the updated RC-to-stable, changelog, and website-docs timing rules. * **Bug Fixes** * Improved PR finalization to source release bodies from verified changelogs and warn/fallback on missing or empty content. * Tightened label gating so E2E runs only under the intended conditions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
6cb579338b |
fix(nightly): rewrite an image host that is the whole scalar value
The nightly host rewrite is a literal "<src-registry>/" substring replace, so it only reaches a host sitting contiguously in front of the repository. kubeovn keeps its host in global.registry.address with the bare repository in a sibling key: no trailing slash, no match. The image was mirrored to the public registry and the tree kept pointing at the private build registry. The gap was dormant while kubeovn's image came from a separate repository, because the baked ref carried the public host and the ownership filter skipped it. Building the image in-tree stamps the build-registry host into that key and turns it live, so fix it here. Add an end-of-line-anchored expression that fires only when the source registry is the complete value; a contiguous ref continues past the host, so the two expressions cannot both fire on one reference. keycloak-operator splits at a different boundary and stays unfixed -- neither of its keys holds the source registry as a whole. Documented, and dormant while no CI path rebuilds it. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
a7ce4371c2 |
docs(agents): correct the chainsaw podLogs capability note
The e2e conventions, the previous-logs capture script, and the global chainsaw catch all claimed the v0.2.15 podLogs collector exposes only container/tail/selector. It also accepts name, namespace, timeout and the cluster fields; only the previous option is genuinely missing. The wrong list steers catch authors toward imperative scripts for cross-namespace log capture that podLogs handles declaratively. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
1eded27900 |
test(release): scope the trim pin to the parse step, fix two stale references
Three review nits, all verified against the tree, plus a tighter scope than the first attempt at them. The behaviour suite asserted "promote-rc does not trim the rc tag" against the whole file. promote-rc.yaml has no .trim() today, so it passed, but any unrelated .trim() added anywhere in the workflow would have failed a test about tag policy. Scoping it to the parse JOB was not enough either: that job is ~180 lines, most of them the e2e gate's github-script, whose string handling has nothing to do with tag policy. Both greps are now scoped to the parse STEP, and run through a code_lines filter copied from the contract suite so a YAML comment mentioning .trim() cannot fail them. Mutation-checked in both directions: a .trim() injected into the parse step fails the test, one injected into the e2e-gate step of the same job does not. - The Step 2 checklist entry still said "must be main" while Step 2 itself now lists three valid starting points (main, detached at the release tag, detached at the rc tag being promoted). The checklist is what an agent ticks off, so it was the half that mattered; it now points at Step 2 rather than re-encoding the count, so a fourth configuration cannot desync it. The heading said "Checking current branch", which two of the three states do not have. - hack/testdata/needs-semantics.yaml named release-changelog-contract.bats as its consumer; the only consumer is release-changelog-behaviour.bats. 50/50 tests green across the three suites (11 + 24 + 15). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
717c8860f6 |
docs(release): make the release playbook coherent with the promote flow
Final coherence pass over docs/release.md (and the releasing.md pointer) after several incremental patches to the release-cycle rework left the whole-document narrative uneven. - Rewrite the regular and patch release sequences so a maintainer reading top-to-bottom performs the actual flow: cut rc -> watch mandatory rc E2E -> [optional Generate RC Changelog button] -> promote dispatch (E2E gate; changelog generated or picked up; website docs PR opened and parked) -> review the promote PR (no E2E by default; E2E/changelog/website-docs lines) -> merge -> finalize -> merge the parked website PR, then announce. - Purge the last old-model sentence: "vX.Y.0 is tagged from the last RC or a later commit in main" -> promoted from the rc by digest, tag cut write-once at the promote PR merge commit. - State the alpha/beta contract where prereleases are discussed: built and published, button-only E2E (no mandatory rc-e2e), and no promotion path (promote-rc accepts vX.Y.Z-rc.N alone). - Fix the Phase 4 heading (was "chore(release): cut vX.Y.Z"; the PR is "chore(release): promote ... -> ...") and add the missing website-docs line to the reviewer checklist; add a keep-parked-until-publish item to the pre-release checklist. - Add cut-prerelease.yaml, changelog-rc.yaml, and changelog-generate.yaml to "See also" so the workflow inventory is complete and consistent with the "seven workflows + reusable changelog-generate" count. - releasing.md: replace the stale "typically involves ... creating git tags / building artifacts" list (invalidated by the promote-not-rebuild model) with the real high-level sequence. Every workflow/job/step name cited was verified against the YAML on this branch. hack/release-changelog-contract.bats stays green (24/24); no test pins docs/release.md strings, so no test change was needed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
34e45295fe |
ci(release): fix second-round review findings
Two more review passes (Claude + Codex) on the previous commit. Codex found the one that would have broken the very next promotion. The agent reads its own instructions from the working tree, which is the rc tag. Every rc cut before this change carries the old §2 of docs/agents/changelog.md — two valid configurations, "stop" on anything else — and a detached rc-tag checkout is exactly that "anything else". So promoting v1.6.0-rc.4 would have presented a compliant agent with instructions telling it to refuse, silently, producing no changelog on every promotion of an existing rc. Verified against the tag. The same applies to hack/validate-changelog.sh, absent from those trees entirely, which would have failed the Verify step with "command not found" and, in open-pr, taken the promote PR down with it. Both are now overlaid from the dispatch ref, best-effort, in both jobs. The structural validation added last round rejected every PATCH changelog. It asserted a `# Cozystack vX.Y.Z` header and a 20-line floor; patch releases use `# vX.Y.Z (date)` and v1.5.1 is a complete 19-line release. Good AI output would have been discarded and the promote PR mislabelled "generation failed". Measured across the tree: the header form holds for 23 of 115 changelogs, so it was never an invariant. The predicates now live in hack/validate-changelog.sh, tested directly against real changelogs rather than through YAML greps — the reason the defect shipped is that the suite pinned finalize's guard but nothing on the promote-side validator, and stayed green when the validator was mutated away. It accepts both header conventions, has no line floor, and checks the version in the header, the release-link comment and the compare link, so a changelog generated for the wrong version is caught. All 16 current-era changelogs pass; whitespace, empty, truncated and wrong-version inputs are rejected. Also fixed: - A fatal artifact download in open-pr would abort the job before the PR step — the same wedge as a skipped job, one step lower. Download is now best-effort and the commit and PR body key off what is on disk, not an upstream job output that says nothing about whether the transfer worked. - A pre-existing changelog (hand-written, or preserved across the force-push) reached the PR without ever being validated, was advertised with ✅ and published verbatim. It now goes through the same validator, as does the tags.yaml backstop. - `gh pr view <branch>` matches closed PRs, so re-dispatching after closing an abandoned promote PR edited the dead PR and opened nothing. Branch on state explicitly. - `gh pr edit --body` unconditionally overwrote the PR body, destroying the upstream breaking-change notes docs/release.md instructs reviewers to paste there. Refresh only while the body is still the generated one. - open-pr consumed raw dispatch input while its siblings used the validated parse; the changelog job's bash parse did not trim, so an input with a stray space passed promote and failed there. Both fixed. - The prompt and §2 said "trailing release-link comment"; it is the first thing in every changelog. Tests grow to 16, each mutation-checked: dropping the overlay, making the download fatal, reverting the existing-changelog check to `-s`, and removing the backstop validator each fail the suite. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
f49d54a689 |
ci(release): fix review findings; keep website docs at tag time
Dual-model review (Claude + Codex) of the previous two commits found seven substantive defects, two of them destructive. Both models independently reached three of them; the split is noted per item. Descope the website half (Claude, verified by reproduction). The docs job was moved to promote time, but the website Makefile pins the fetch ref with `override BRANCH := $(RELEASE_TAG)`, which defeats the command-line BRANCH assignment. `make -p` resolves BRANCH to v1.6.0 — a tag that cannot exist at promote time, since promote's own validate step hard-fails if it does. update_apps.sh logs the failed fetch and exits 0 AFTER truncating each destination file, so every app doc would become a stub, `changed=true` would open the PR, and the new auto-merge would land it unreviewed. Not fixable from this repo. update-website-docs returns to tags.yaml where RELEASE_TAG resolves to a real tag, and the finalize auto-merge is dropped. Opening it earlier needs a cozystack/website change first and is left as a follow-up. This also removes the unordered-sibling race Codex found, where the release PR could merge before the docs job opened its PR, leaving finalize nothing to merge and no retry. Fix the non-blocking guarantee (both models). Only the AI step carried continue-on-error; the changelog JOB did not, so a flaky `npm i -g @github/copilot` failed it and `needs` skipped open-pr entirely — staging branch pushed, release drafted, no promote PR. Gate open-pr on promote alone. Generate from a checkout the contract permits (both models). The job checked out the rc staging branch, which docs/agents/changelog.md §2 rejects — a compliant agent stops and writes nothing, so the normal path produced no changelog at all. Check out the rc tag detached and document that configuration in §2. The checkout ref now comes from the validated parse output rather than raw dispatch input. Port, don't regenerate, in the backstop (both models). For a patch release promoted onto release-X.Y the changelog never reaches main, so the backstop fired and ran a second AI pass; once its PR merged, update-releasenotes.yaml overwrote the published body with unreviewed text. Detect the file at the tag and port it verbatim; the AI runs only when it is absent from both main and the tag. Survive re-dispatch (Codex). promote force-pushes the staging branch, erasing a hand-written changelog — the documented recovery when generation fails. Preserve an existing one across the rebuild, and stop open-pr overwriting it with generated output. Validate structurally, not by size (Codex). A truncated or whitespace-only AI run passed `[ -s ]`, was committed, advertised as valid in the PR body, and published verbatim as release notes. Assert the header, the compare link and a plausible line count; finalize additionally refuses whitespace-only content rather than blanking the notes. Add hack/release-changelog-contract.bats — nine tests pinning each link in the chain, mutation-checked: removing open-pr's if:, breaking finalize's body read, and restoring paths-ignore each fail the suite. Update docs/release.md (~10 sites) and docs/agents/changelog.md §2, which described the old flow throughout. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
55195ded0c |
test(e2e): capture previous container logs and stop killing the snapshot (#3346)
## What this PR does Root-causing a recent E2E failure was nearly blocked because the diagnostics captured on failure had collected almost nothing. Two independent mechanisms were responsible, and both had been silently in place for a long time. **The failing container's previous logs were never captured.** The pod had crash-looped, so the decisive evidence lived in the first container instance, reachable only through `kubectl logs --previous`. Chainsaw's `podLogs` action has no `previous` field, so this could not be expressed per-suite; the conclusion had to be reconstructed from indirect evidence instead. A new capture runs from the global catch and from the BATS failure hook, selecting containers at runtime by `restartCount > 0` across both init and regular containers, so a healthy run stays quiet and future suites are covered without per-suite wiring. **The catch was structurally guaranteed to be killed.** A Chainsaw `script` op is bounded by `timeouts.exec`, which is 5m, while the catch's own inner budgets summed to roughly 960s. It reported `SCRIPT | ERROR | signal: killed` and produced nothing — and its second leg, the data-plane capture, had therefore never run at all. The kill signature was itself misleading: `timeout` exits 124 and cannot produce that message, which comes from Go's `exec.Cmd` killing the process group. Each catch op now carries an explicit timeout above the budgets it contains, so the inner one fires and leaves a partial, usable artifact instead of losing everything. **The snapshot was truncated at 60 seconds.** `crust-gather --duration` defaults to 60s and was never passed, so every snapshot ever taken was cut short, with nothing in the artifact saying so. The API discovery it performs first is not covered by that budget at all, which is where the outer wall-clock was going on a cluster with a degraded aggregated APIService — precisely the state a failed run is in. Both budgets are now set explicitly, and the collector's output goes to a log inside the snapshot with a completeness verdict, so "is this snapshot complete?" is answerable from the artifact. Failure paths are reported rather than swallowed: an unreachable API says so instead of implying nothing had restarted, a truncated capture is kept and marked, and every dropped item is logged, since silent truncation reads as full coverage. ### What is not verified here The mechanism claims are read from the pinned versions of Chainsaw and crust-gather. Everything cluster-facing is not verified by this PR: the go-template's output shape against a live API server, `--previous` returning the first instance for a real crash-looper, and whether the new collection budgets suffice on a degraded cluster. The bats suite covers the selection helpers only, and those cases are mutation-tested. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Diagnostics** * Improved end-to-end failure evidence by capturing “previous instance” logs for crash-looping pods (including restarted init containers) and prioritizing relevant namespaces. * Added time-bounded diagnostics collection with explicit outer timeouts; collector output is saved to per-snapshot log files. * Classifies collector outcomes as complete, timeout-truncated, or failed, while preserving the original test failure signal. * **Documentation** * Expanded E2E failure-diagnostics guidance with new timeout and evidence-capture conventions. * **Tests** * Added unit-style tests for previous-log selection, prioritization, row limiting, and deterministic log artifact naming. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
1f302d0d78 |
docs(agents): correct issue ref, mirror claim and example in image-refs
The build-list violations bullet pointed at #3143, which tracks the adjacent gap (packages in the list going stale between releases) and never covers packages absent from the list; they are now tracked in the newly filed #3416. The nightly split-host paragraph claimed both affected images are mirrored while the published tree points at the build registry — today neither ref carries the source-registry host (keycloak-operator is never rebuilt by CI, kubeovn is built externally by cozystack/kubeovn-chart), so the ownership filter skips them and the gap is latent, not live; reworded to say exactly that. The opening example paired a v1.6.0 tag with the real v1.5.0 grafana digest — a ref that will never exist — and grafana is slated to stop being built; replaced with the literal committed cozystack-api ref. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
76f7267ff2 |
test(e2e): describe --previous as the preceding instance, not the first
kubectl logs --previous returns the immediately preceding terminated container instance, not the original first one — after multiple restarts the first instance is already gone. Reword the helper, the BATS and Chainsaw failure handlers, and the diagnostics doc to match the actual --previous semantics. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
9bd8c17971 |
fix(ci): shape-filter the promote postcondition and close test gaps
Review of the previous commit found the postcondition itself could abort
a release. It grepped the bare "X.Y.Z-rc.N" substring across the tree, and
three live files carry such strings without being image refs:
kubevirt-csi-driver's go.sum pins github.com/golang/protobuf v1.4.0-rc.2
-- a version cozystack actually cut, so promoting v1.4.0 would have
failed on a Go checksum line -- metallb_test.yaml names v1.5.0-rc.2 in a
comment, and the console pnpm-lock.yaml carries rolldown@1.0.0-rc.15. The
remediation the error message gave would have made things worse: declaring
go.sum as a ref source would sed a checksum file.
Match the version only in image-reference position now: after an
image/repository/tag key with no intervening '#', or immediately before an
@sha256 digest. Narrowing it to "${RC}@sha256:" alone would have been
wrong -- cilium keeps `tag: v1.5.0` and `digest:` under separate keys, so
the version is not always adjacent to the digest. Verified against four
real rc versions that previously aborted.
The suite also only tested the fail-closed direction, which is why that
defect survived, and the collector had no completeness guard at all: a
first-party ref added at an unenumerated path on a `latest@` tag was
mirrored by nothing and passed all eighteen tests. Since the mirror's
failure mode is the worst of the three -- the published nightly points at
the private build registry -- that gap mattered most where coverage was
thinnest.
Added: a fail-open test using the three real rc versions above; a
collector completeness diff that discovers refs by content and fails on
any the enumeration misses; coverage for the OCI-artifact shape, whose
deletion previously left every suite green; and an unreadable-file test,
since a read error was being treated as "no match" and silently skipped.
The component-versioned test was vacuous -- its rc version shared no
substring with the kamaji tag it claimed to protect, so it passed by
construction. It now uses an rc version sharing kamaji's 0.19.0 prefix,
which a loosely-anchored implementation would corrupt.
Round-trip fixture discovery matched a combined 'ghcr.io/cozystack/
cozystack/' prefix, which skips keycloak-operator (host in a sibling
registry: key) and kubeovn (host in global.registry.address). It now
matches 'cozystack/cozystack' and stamps split `tag:` keys too.
Also documented two pre-existing gaps rather than half-fixing them: the
nightly host rewrite is a contiguous-substring replace and cannot reach a
split host, and a ref inside a gzip is invisible to every tool including
the postcondition, which skips binaries. Both are host-dimension defects;
the version dimension this change owns is unaffected, because the version
always lives in a tag key regardless of where the host sits.
Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Aleksei Sviridkin
|
07c67a8913 |
test(e2e): capture previous container logs and stop killing the snapshot
Root-causing a crash-loop failure was blocked twice over by the diagnostics collected on failure. `kubectl logs` shows only the current container instance, so for a crash-looping pod the suites' `podLogs` collector captured the Nth restart -- a fresh replay of the same startup path -- while the evidence that explained the failure sat in the first, interrupted instance. Chainsaw's podLogs collector has no `previous` option, so add e2e-capture-previous-logs.sh and drive it from the global catch and from the BATS failure hook. It discovers containers with restartCount > 0 at runtime (init containers included, where an interrupted bootstrap lives), orders the app suites' namespace first, and dumps each to the job log and the snapshot tree. Containers that never restarted are skipped, so a healthy run emits one line rather than a page of "previous terminated container not found". Every way that capture can come up short is named rather than inferred, because kubectl exits 1 for a garbage-collected container, an RBAC denial and an unreachable node alike: the container cap, a pod list that never returned (which is not the same as nothing having restarted), a read that timed out versus one kubectl refused and the reason it gave, and a previous instance that produced no output. A partial capture is kept and marked truncated rather than deleted -- it is the only copy of a log already gone from the cluster. The crust-gather snapshot was killed by the Chainsaw op containing it. A script op is bounded by timeouts.exec (5m), and the catch's own wall-clock wrappers summed to 960s, so the script was SIGKILLed partway through -- taking out the process group mid-write and skipping the data-plane capture entirely. Give each catch op an explicit timeout above the budgets it contains, so an inner timeout fires instead and leaves a partial, still-servable snapshot behind. The data-plane leg drops from 600s to 300s at that call site so both legs fit one envelope; the BATS hook keeps the wider budget, having no containing op. Set crust-gather --duration explicitly as well. It is the collection phase's own budget, it defaults to 60s, and on elapse it abandons the collection and skips its finish step, leaving a truncated tree with no indication anything is missing. The outer wall-clock has to exceed it because the API discovery that runs first is not covered by that budget at all -- with an unhealthy aggregated APIService, the state a failed run is usually in, discovery alone can consume the whole timeout before a single object is written. Send each collector's output to a log beside its snapshot instead of /dev/null, with a complete/truncated/failed verdict, so that question is answerable from the artifact. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
09323fff13 |
fix(ci): cover every image-ref storage shape in promote, retag and mirror
Promotion rewrote the rc version substring across the depth-2 package values.yaml plus packages/apps/kubernetes/images/*.tag alone, on the premise that the kubernetes app was the only one whose .tag files carry the cozystack version. Nine other .tag files do, and system/multus seds its ref straight into a vendored upstream manifest. v1.6.0 was therefore staged with 33 of 55 first-party refs still reading v1.6.0-rc.4. promote-retag.sh and nightly-mirror.sh had the same blind spot from the other direction: both scanned only the depth-2 values.yaml, so those images were never retagged to the stable version (30 refs selected before, 42 after) and never mirrored to the public registry for a nightly. A retag miss merely leaves an image short of a tag, since the digest still resolves inside one registry; a mirror miss is worse, because the host rewrite walks the same file list and a ref that is neither mirrored nor rewritten leaves the published tree pointing at the private build registry. The three call sites each carried their own idea of where a ref can live, which is how they drifted apart. Move that knowledge into hack/lib/image-refs.sh as the single enumeration -- image_ref_files() for the files, collect_image_refs() for the refs inside them -- and have all three source it, so a new storage location is declared once and reaches every consumer. The rewrite itself moves out of promote-rc.yaml into hack/promote-rewrite-tags.sh. Being workflow-inline is why this was only observable by cutting a release: there was nothing to unit test. It now also asserts its own postcondition, scanning wider than it rewrites and failing the promotion when any ref survives in a location the enumeration does not know about, rather than shipping it quietly. Component-versioned images (kamaji's v0.19.0-cozystack.N) and third-party pass-through images do not ride the cozystack version line and are deliberately left alone; a test pins that so a future fix here cannot over-reach. capi-providers-cpprovider is a documented known gap: its ref is also embedded in a gzipped copy that the chart actually ships, and rewriting only the readable copy would diverge the two. docs/agents/image-refs.md records the contract -- three tag classes, three storage shapes, the invariants, and which consumer does what. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey KolkovandClaude Fable 5
|
f1627354b4 |
docs(contributing): point the ansible migration note at loadBalancerClass
The downstream-coordination checklist still directed contributors at an
"externalIPs to exposureClass" migration; exposureClass was removed in
|
||
|
Aleksei Sviridkin
|
44507d2cd0 |
docs(agents): add downstream repository checklist to the PR template
Cozystack is upstream for repositories that nothing keeps in sync with it, and no CI job compares the two sides, so a change here breaks them in silence. Only two narrow paths are automated, and both fire on a release rather than on a PR. It has already happened. The website docs generator works from an app list hardcoded in its own Makefile, and packages/apps/opensearch was never added to it, so an application we ship has no reference page at all. The Terraform provider still offers a Kubernetes version this repo removed. The plugin repo still sends operators to hack/ scripts that were deleted, one of them rewritten as a Go command. Add a Downstream Repositories trigger map to the contributing guide, naming per repository which change here forces a change over there and which file to touch, and a checklist in the PR template that links to it. The template addresses AI agents directly: they author a large share of PRs and will otherwise tick a box without opening the follow-up it claims. The coupling is wider than the obvious ones, and mostly invisible. talm restates the node contract, overlapping ours but already diverged, so it must be diffed rather than overwritten. cozyhr reads an annotation the operator writes and silently merges the wrong values if it is renamed. cozy-proxy is vendored in from its own repo, yet consumes labels produced here and quietly stops handling external VM services if they move. The telemetry server queries metric names emitted here, and a rename just empties its dashboards. None of these fail loudly. Guard the map with a unit test, because a map about silent drift must not drift silently itself. It pins every in-repo path the map cites to a file that exists, pins the map and the checklist to the same repository list, and pins the workflow carve-out below to the map's location, to the job that reads the flag, and to the output that carries it: cutting any one of those three leaves the other two looking wired up. The header states plainly what the suite cannot check, so a green tick is not mistaken for a correct map: whether a coupling is real, and whether it still holds over there, stays a human's job. Editing the map alone is a docs-only PR, which skips the unit tests and would exempt the map from its own guard. Rather than widen the docs-only rule, which also gates the Talos image build, the plan step exports a separate flag that only the unit-test job reads: a docs PR now runs the tests and still builds nothing. It detects the map by its old path too, since a rename is reported as the new path alone. Document that gh pr create --body and --body-file replace the body wholesale, which would otherwise drop the checklist from every CLI-authored PR. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
18715f8c63 |
docs(agents): document Chainsaw v0.2.15 gotchas in the e2e-testing guide
The v0.2.15 syntax/semantics traps were tribal knowledge scattered across PR review notes (the number-literal caveat lived only in #2826's body, which evaporates on merge). Consolidate them into a durable subsection so the next author finds them before hitting each one fresh: filter-as-list conditions, no number-literals/backticks in JMESPath, error: fails hard on an unknown GVK, no conditional Test-level skip, step-level-only finally, bounded/blocking cleanup surfacing latent teardown bugs, projection-style failure diffs, and the catch-script cwd. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
9297e51170 |
Merge origin/main into feat/e2e-chainsaw-example
Reconcile the Chainsaw e2e migration with ~500 commits of main drift: - keep main's run-kubernetes.sh helpers (kubectl_wait_retry, tenant drain, LINSTOR pool-free wait, SC-fallback probe) under the branch's _lib/ rename with the Chainsaw finally/snapshot deltas re-applied - resolve modify/delete on 10 drifted hack/e2e-apps/*.bats as deleted; their post-branch assertion drift is re-ported into the Chainsaw suites in follow-up commits - drop the 4 new main-side bats (kubernetes-oidc-*, securitygroup, serviceexposure) from _lib/ relocation; ported as Chainsaw suites in follow-up commits - docs/agents/e2e-testing.md: merge main's plan-job TIA wording with the Chainsaw suite phrasing Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
902cbe2d74 |
ci: address CodeRabbit review on build-matrix workflow
- scope create-github-app-token to the current repo with read-only contents permission (was org-wide, inheriting the app's blanket installation permissions); both token steps only read releases/assets - fail-fast the Talos image upload when the disk is missing (if-no-files-found: error) so drift fails in build-talos, not later as a confusing e2e download error - download the flux install script to a tempfile before executing it, so a failed curl is not masked by the piped `sudo bash` exiting 0 - fix the regular-PR artifact-source comment (Talos disk comes from build-talos, pr.patch from finalize) and the TIA scope wording in docs/agents/e2e-testing.md Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
2940093102 |
test(e2e): capture per-failed-test crust-gather snapshots in Chainsaw harness
The Chainsaw app suite runs via `chainsaw test` directly, bypassing hack/cozytest.sh — whose EXIT trap captures a crust-gather host snapshot per failed BATS test. As a result a failing Chainsaw run uploaded a cozyreport.tgz with zero per-test cluster snapshots, the richest diagnostic for failures like the gateway/etcd/bucket/openbao ones on this branch. Add the Chainsaw analog: a global spec.error.catch in .chainsaw.yaml that collects a bounded host snapshot into _out/cozyreport/snapshots/<test>/host on any failure (same --exclude-kind Secret flags and timeout -k 30 300 as cozytest.sh), keyed by the $test.metadata.name binding with a mkdir-guard so repeat catch invocations don't re-collect. Wire COZY_SNAPSHOT_NAME into the kubernetes-latest/previous suites so the existing tenant snapshot from run-kubernetes.sh co-locates under the same snapshots/<test>/ dir. cozyreport.sh already folds snapshots/ into the uploaded artifact. The e2e-testing guide and the sandbox Dockerfile comment are updated to record the convention. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
c82b55c621 |
docs(agents): align E2E guide and references with the Chainsaw migration
Rewrite docs/agents/e2e-testing.md for the Chainsaw suite: declarative
asserts replace the until-get/wait backstops, Chainsaw cleanup replaces
test-level EXIT traps, catch blocks replace the `|| { ...; false; }` idiom,
and the TIA/diagnostics sections describe suites + `make test-chainsaw
CHAINSAW_SUITES`. Move the BATS->Chainsaw bullet out of 'in-flight' since it
is now the standard.
Fix references the migration left stale: hack/e2e-apps/ -> hack/e2e-chainsaw/
in the gemini styleguide and the nats/etcd backup READMEs. The
backup-postgres.bats / backup-foundationdb.bats files cited by the postgres
examples, the postgres README, and the foundationdb controller comment never
existed -- drop the false 'CI e2e drives this' claims and describe those
backup flows as the manual reference flows they are.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov DaniilandClaude Opus 4.8
|
e22d84f2c8 |
test(e2e): migrate app suite from BATS to Chainsaw and wire into CI
Replaces the per-app BATS E2E suite (hack/e2e-apps/*.bats) with declarative Kyverno Chainsaw tests under hack/e2e-chainsaw/, extending this branch's pilot (postgres + bucket) to all 18 app suites. - Declarative ports assert on status/conditions and concrete fields; the repeated "timeout ... until kubectl get" + "kubectl wait" boilerplate collapses into a single polling assert with structured failure diffs and automatic events/describe/podLogs capture on failure. - Inherently imperative suites keep their logic in script steps: openbao init/unseal, kuberture split-horizon probes, vminstance, the gateway admission/impersonation cases, and kubernetes-latest/previous, which wrap the relocated hack/e2e-chainsaw/_lib/run-kubernetes.sh verbatim. - Gateway tests derive the tenant apex from the namespace host label at runtime, so they are host-independent. CI: the e2e job now runs "chainsaw test hack/e2e-chainsaw/" via a new test-chainsaw target and uploads the JUnit chainsaw-report.xml; the chainsaw binary is added to the e2e-sandbox image. install-cozystack and test-openapi stay BATS. The per-app 3-retry loop is dropped, since assertion polling replaces the fixed-timeout flakiness it papered over. Verified against a dev cluster (DB/app, storage and VM suites pass). Service port assertions use the ports[*].port projection form because Chainsaw v0.2.15 mis-evaluates JMESPath number-literal port filters. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
fa49dbaa33 |
Merge origin/main into refactor/build-matrix-2937
Reconcile the parallel-matrix PR build (#2983) with main, which since this branch diverged landed a competing single-VM build restructure (#2939) and test-impact-analysis E2E (#2559). Per maintainer decision, #2983's matrix build supersedes #2939: - .github/workflows/pull-requests.yaml: keep #2983's plan/checks/build(matrix)/build-talos/finalize jobs; keep main's newer resolve_assets/e2e (carrying #2559 test-impact E2E). The build->e2e artifact interface (pr-patch + talos-image) is identical on both sides, so e2e depends on finalize/build-talos with no input rewiring. main's top-level permissions and SHA-pinned action refs applied across all jobs. - All other files merged cleanly (only pull-requests.yaml conflicted). Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
a036cfd1e9 |
ci(tests): remove redundant Release E2E workflow (#2978)
## What this PR does Removes the `Release E2E` workflow (`.github/workflows/release-e2e.yaml`), which ran the full bats suite on every release tag. **Why it's redundant.** `release-e2e.yaml` was a non-blocking alarm — its own header states *"a failure here is an alarm, not a release blocker."* The coverage it provided is already delivered by the release-cut PR: the "Prepare release" commit bakes image digests into `packages/core/` (installer, platform, testing) and other packages, and `select-e2e.sh`'s full-suite pattern escalates any `packages/core/` change to the **full** suite. So the release PR already runs the whole suite against the commit the tag will point at. **Why now.** The workflow has never been green since it was added (2026-05-02) — every tag run (`v1.5.0-rc.2`, `-rc.3`, `v1.5.0`) failed at the build step. Even after #2973 fixed the registry login earlier today, the next run still failed at *Build Talos image*. A perpetually-red, non-gating duplicate is pure noise. **Changes** - Delete `.github/workflows/release-e2e.yaml`. - Drop `release-e2e` from the full-suite pattern in `hack/select-e2e.sh`. - Repoint the workflow-trigger unit test in `hack/select-e2e_test.bats` to `pull-requests.yaml` (the remaining E2E workflow), preserving "workflow edit → full suite" coverage. All 11 `select-e2e_test.bats` cases pass. - Update `docs/agents/e2e-testing.md` to describe the release-cut PR as where the full suite runs. Refs #2397. ### Release note ```release-note ci(tests): remove the redundant Release E2E workflow; the full E2E suite already runs on the release-cut PR ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Removed automated release end-to-end testing workflow * Updated E2E test selection logic to reflect testing infrastructure changes * **Documentation** * Updated E2E testing procedures documentation to reflect current testing approach <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a1b88cfa6e |
docs(tests): drop the merged k8s-version pin from the in-flight e2e list
The "Pin the e2e management cluster Kubernetes version" item is no longer in-flight — it is merged and live in hack/e2e-prepare-cluster.bats (--kubernetes-version v1.33.12). Remove it from the "not yet the merged standard" section; its rationale lives in the bats comment. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
35d8024b8e |
docs(agents): update e2e-testing for the matrix build pipeline
The "What TIA does and does not do" section described the old single Build job and the `detect-changes` paths-filter. Update it for the new `plan` job (docs gate + build matrix), note that image builds are now diff-scoped by hack/build-matrix.sh (distinct from TIA's app-loop trim), and point the "widen the skip" advice at the plan docs-gate. Refs #2937 Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
b81e92ac9c |
ci(tests): remove redundant Release E2E workflow
The Release E2E workflow (`release-e2e.yaml`) ran the full bats suite on every release tag as a non-blocking alarm — its own header notes "a failure here is an alarm, not a release blocker." It duplicates coverage the release-cut PR already provides: the "Prepare release" commit bakes image digests into `packages/core/` (and other packages), which TIA escalates to the full suite, so the release PR exercises the whole suite against the commit the tag will point at. It has also never been green since it was added (2026-05-02): every tag run failed at the build step. Even after #2973 fixed the registry login today, the next run still failed (Build Talos image). Removing a perpetually-red, non-gating duplicate removes noise without losing coverage. Removes the workflow, drops `release-e2e` from `select-e2e.sh`'s full-suite pattern, repoints its unit test to `pull-requests.yaml`, and updates the e2e-testing doc. Refs #2397 Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
a8c3a57fa2 |
docs(agents): consolidate build, test, and verification guidance (#2933)
## What this PR does Promotes universal project guidance that previously lived only in a local (gitignored) `CLAUDE.md` into the committed agent docs, so every contributor and every tool (Codex/Cursor/etc., not just Claude) gets it from a fresh clone. - `overview.md`: Build & Development Commands (root + per-package make targets, build env vars), Values schema generation, a Testing section, Domain references - `contributing.md`: a "Verify Before Considering Done" matrix + Review Expectations - Dedup: `overview.md`'s Git Commits block now points to `contributing.md` > Note: `overview.md` links to `docs/agents/e2e-testing.md` added in #2932 — merge > that one first or together. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Expanded contribution guidance with a “Verify Before Considering Done” pre-submission checklist and clearer expectations for including behavioral regression tests in bug-fix PRs. * Broadened build/development documentation with a “Build and Development Commands” guide, including required environment variables, values-schema generation, and test entry points. * Updated contribution workflow notes by consolidating commit-convention guidance and adding pointers to task-specific design references. * Added a Claude entry-point document that directs to the main agents guide and explains local override placement. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
be716a7e0e |
docs(agents): drop broken dashboard doc reference
Address review feedback from coderabbitai on docs/agents/overview.md:145: internal/controller/dashboard/README.md does not exist (no such directory), and there is no dashboard reference doc to point to. Remove the dead bullet; the backups DESIGN.md reference is valid and kept. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
680e3cc47b |
docs(agents): clarify PR description template wording
Address review feedback from gemini-code-assist on docs/agents/contributing.md:145: reword the awkward "on the template" phrasing to "aligned with the template". Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
71d76ec3d3 |
docs(agents): distinguish make helm-unit-tests from full suite
Address review feedback from gemini-code-assist on docs/agents/overview.md:103: the Helm-only target is make helm-unit-tests; make unit-tests runs the full unit suite. Point the bullet at the precise target and note the difference. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
aaa2688b68 |
docs(agents): correct make unit-tests description in overview
Address review feedback from gemini-code-assist on docs/agents/overview.md:72: make unit-tests runs the full suite (Helm, BATS, Go, presets, readiness), not only Helm unit tests. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
943d5932c0 |
docs(agents): consolidate build, test, and verification guidance
Promote the universal project guidance that previously lived only in a local CLAUDE.md into the committed agent docs, so every contributor and every tool (Codex/Cursor/etc., not just Claude) gets it from a fresh clone. overview.md gains Build and Development Commands (root + per-package make targets, build env vars), Values schema generation, a Testing section, and Domain references. contributing.md gains a "Verify Before Considering Done" matrix and Review Expectations. Dedup: overview's Git Commits block now points to contributing.md instead of restating it. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
d522c5b5c5 |
docs(agents): add E2E and CI testing conventions
Capture the e2e stabilization conventions established over the last few months as a routed agent doc: no retries on deterministic CI steps, event-driven backstops over fixed sleeps, no EXIT/RETURN traps, fail-fast HelmRelease-readiness gating, the status.history remediation guard, first-class failure diagnostics, and a reviewer checklist. Also documents Test-Impact Analysis (TIA) and its real scope — it trims only the per-app loop, runs after install, and most PRs escalate to the full suite. Wire it into AGENTS.md so every agent reads it before e2e/CI work. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
39605f213f |
docs(agents,ci): finish "tag-not-HEAD" sweep and fix changelog-exists gate
- tags.yaml: the check_changelog step now looks at origin/main rather than
the working tree. After Round 2 we check out the release tag, which never
has the changelog file (the PR opened by this job merges to main after
the tag is cut), so the previous `[ -f ]` test always returned false and
the gate was effectively dead on reruns.
- changelog.md: Step 6's `RELEASE_END=$(git log -1 --format=%ai HEAD)`
(lines 179, 222) was missed in the previous "never HEAD" sweep — now
uses v<new_version>, matching Step 5.
- changelog.md: Step 2 ("on main branch") was incompatible with the CI flow
which now runs detached at the release tag. Relaxed it to list both
configurations (interactive on main; CI detached at v<version>), while
the "Scope and boundaries" section continues to forbid the agent from
switching branches.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
ce63bfcdb0 |
docs(agents): fix new-contributor detection and tags.yaml/doc drift
- tags.yaml: drop "(or HEAD)" from the changelog prompt; the doc says "never HEAD" and the workflow now matches. - docs/agents/changelog.md: exclude the current release file from the "previous contributors" set used for first-time-contributor detection. The previous globbing on `docs/changelogs/v*.md` swept up the file being generated, so `comm -23` would never report any newcomers. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
0f9198d510 |
docs(agents): address changelog rule review feedback
- Drop the "orphan backport, drop entry" rule: a backport whose original is on main but hasn't shipped in a stable tag is the first delivery and must stay in the changelog. - Add `v` prefix to `<previous_version>`/`<new_version>` placeholders in all four `git log` invocations for consistency with the rest of the doc. - Apply the bot/CI account filter to the new-contributors detection (and the standalone previous-contributors extraction) so bot accounts can no longer leak into the human Contributors list. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
2420d8949b |
fix(ci): generate changelog from the release tag, not from main
The generate-changelog job in .github/workflows/tags.yaml previously checked out main and ran the AI agent there. The agent followed docs/agents/changelog.md, which instructed it to compute the release range with `git log <previous_version>..HEAD`. For minor releases (vX.Y.0) cut from main this happens to give the right answer; for patch releases (vX.Y.Z, Z>0) cut from a release-X.Y branch, HEAD-on- main is a strict superset of the release tag and contains commits that were merged to main both before and after the tag. The v1.3.1 changelog generated by this workflow (#2480) demonstrated the failure mode: it included 8 PRs merged to main but never shipped in v1.3.1, 6 backport PRs that landed on release-1.3 *after* v1.3.1 was tagged, both originals and their backports as separate entries, a hallucinated 2024 PR, and the cozystack-ci bot in the contributors list. The corrected v1.3.1 changelog contains the single fix that v1.3.1 actually ships. This PR fixes the root cause and tightens the agent guardrails: * tags.yaml: check out the release tag commit (`ref: ${{ steps.tag. outputs.tag }}`) instead of `main`, so HEAD == the release commit and `git log v<prev>..HEAD` corresponds to the actual release. The Create-changelog-branch step still creates the PR branch from origin/main, so PRs continue to merge cleanly into main. * tags.yaml: update the AI prompt to state explicitly that HEAD is the release commit and that the upper bound for the release range is the new tag, not main. * docs/agents/changelog.md: replace `..HEAD` with `..v<new_version>` in every example so the instruction is unambiguous regardless of which branch is checked out. * docs/agents/changelog.md: add a hard rule that backport PRs MUST be merged into the original entry (single line, two PR numbers), never listed as a second entry. Document the edge case where the original PR is outside the range — drop the entry. * docs/agents/changelog.md: forbid using the brief description verbatim as the detailed description, and forbid inventing entries for PRs that are not in the release range. * docs/agents/changelog.md: filter bot/CI accounts (`app/*`, `*[bot]`, cozystack-ci, github-actions, dependabot, renovate) out of the human Contributors list. Bot attribution on individual entries is still valid. The corresponding correction for the v1.3.1 changelog itself is in PR #2480 on the changelog-v1.3.1 branch. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
bc1eca10cb |
chore(ci): adopt CNCF/k8s label conventions (#2495)
## What this PR does Adopt CNCF/Kubernetes label conventions for issues and PRs and add automated labeling. **Canonical label file**: `.github/labels.yml`. Synced into the repository by `.github/workflows/labels.yaml` (EndBug/label-sync@v2) on push to `main`, weekly cron, and manual dispatch. UI-only label edits are overwritten — propose changes via PR to this file. ### Label namespaces Following the [Kubernetes label scheme](https://github.com/kubernetes/test-infra/blob/master/label_sync/labels.md): - `kind/*` — issue or PR type (bug, feature, documentation, support, cleanup, regression, flake, failing-test, api-change, breaking-change) - `priority/*` — urgency (critical-urgent, important-soon, important-longterm, backlog) - `triage/*` — review state (needs-triage, accepted, needs-information, not-reproducible, duplicate, unresolved) - `lifecycle/*` — issue/PR lifecycle (active, frozen, stale, rotten) - `area/*` — subsystem; 15 seeded plus `area/uncategorized` fallback. Extensible — propose a new area when no existing one fits. - `do-not-merge/*` — PR merge blockers (work-in-progress, hold) Cozystack-specific labels preserved: `epic`, `community`, `help wanted`, `good first issue`, `quality-of-life`, `upstream-issue`, `backport`, `backport-previous`, `release`, `automated`, `debug`, `sponsored`, `lgtm`, `ok-to-test`, `security/*`, `size:*`. ### `area/*` set 15 areas seeded by activity in open issues and PRs: `area/ai`, `area/api`, `area/build`, `area/ci`, `area/dashboard`, `area/database`, `area/extra`, `area/kubernetes`, `area/monitoring`, `area/networking`, `area/platform`, `area/release`, `area/storage`, `area/testing`, `area/virtualization`. Plus `area/uncategorized` as the auto-labeler fallback. ### Migration safety Existing labels are renamed via `aliases:` in `labels.yml`. GitHub preserves the label ID, so all currently tagged issues and PRs keep their tags under the new name without losing references: | Old | New | |---|---| | `bug` | `kind/bug` | | `enhancement` | `kind/feature` | | `documentation` | `kind/documentation` | | `question` | `kind/support` | | `frozen` | `lifecycle/frozen` | | `stale` | `lifecycle/stale` | | `duplicate` | `triage/duplicate` | | `do-not-merge` | `do-not-merge/work-in-progress` | | `do not merge` | `do-not-merge/work-in-progress` | `delete-other-labels: false` on the initial rollout. Generic GitHub-default labels (`wontfix`, `invalid`) are preserved untouched and will be removed in a follow-up cleanup PR. EndBug processes aliases sequentially, so the second of the two `do-not-merge*` aliases hits a name collision and logs a warning — the legacy label survives that one sync and is cleaned up in the same follow-up. ### PR auto-labeling `.github/workflows/pr-labeler.yaml` parses each PR title on `opened`, `edited`, `reopened`, and `synchronize` and applies labels additively (never removes): - **type → `kind/*`**: feat, fix, docs, chore, refactor (others get no kind) - **scope → `area/*`**: scope mapping covers all current cozystack components (full table in `docs/agents/contributing.md`) - **`!` after type or `BREAKING CHANGE:` footer**: applies `kind/breaking-change` - **`[Backport release-1.x]` prefix**: stripped before parsing; `area/release` and `backport` labels added - **Composite scope** (`feat(platform, system, apps): …`): each part mapped independently - **Bracket fallback** (`[scope] description`): maps `area/*` but cannot infer `kind/*` - **Unmapped scope or non-conventional title**: applies `area/uncategorized` for human review ### Schema validation `.github/workflows/labels.yaml` runs a `validate` job on every PR touching `labels.yml` or its workflow. Asserts: - description ≤ 100 chars (GitHub REST API limit) - color is 6-char hex without leading `#` - unique top-level names - aliases do not collide with top-level names Sync runs only on push to main, weekly cron, and manual dispatch; PR runs validate-only. ### Hardcoded label/title references updated - `.github/ISSUE_TEMPLATE/bug_report.md`: `labels: 'bug'` → `labels: 'kind/bug'` - `.github/workflows/tags.yaml`: - changelog PR labels `['documentation', 'automated']` → `['kind/documentation', 'automated']` - release PR title `Release v${version}` → `chore(release): cut v${version}` (so the auto-labeler applies `kind/cleanup` + `area/release`) - changelog PR title `docs: add changelog for v${version}` → `docs(release): add changelog for v${version}` (so the auto-labeler applies `kind/documentation` + `area/release`) ### Documentation - `AGENTS.md`: Activation entry pointing agents to `labels.yml` and the PR title auto-labeling rules. States explicitly that `area/*` accuracy outweighs reuse — propose a new area when none fits, do not shoehorn into a wrong one. - `docs/agents/contributing.md`: PR Title Auto-Labeling section with type→kind and scope→area tables. ### Out of scope (follow-up PRs) - Removal of redundant labels (`wontfix`, `invalid`, plus the surviving legacy `do not merge` if alias-rename collision keeps it) - Org-wide sync from `cozystack/.github/labels.yml` - Dosu bot configuration update for `lifecycle/stale` (requires dashboard access) ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Automated PR labeling from Conventional Commits (type → kind/*, scope → area/*), plus a comprehensive namespaced label taxonomy. * **Chores** * Workflows to validate, sync, and auto-apply labels (including scheduled/manual runs and validation checks). * Added repository-wide label configuration and normalized bug label metadata to namespaced form. * Updated release PR titling/labeling conventions. * **Documentation** * Contributor and agent guidance on PR title conventions, label mappings, and triage procedures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
f527ce683b |
docs(agents): clarify that Scopes list is illustrative
The Scopes section was read as an exhaustive enumeration, which led to review feedback flagging any scope outside the list as invalid. The intent has always been that contributors pick the most specific scope for the change and extend the list when a genuinely new area appears. Reword the section accordingly and add operator as an example scope. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d86bc7760a |
docs(agents): document PackageSource upgradeCRDs field
Describe when to set upgradeCRDs: CreateReplace (operators that evolve their CRD set additively between versions) and the data-loss risk of enabling it on operators that drop fields. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
738762994e |
docs(agents): fix markdown table cell spacing in contributing.md
Address review feedback from gemini-code-assist on docs/agents/contributing.md:73: add space before closing pipe in the type to kind mapping table for consistency with other rows. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
31f4435eb0 |
docs(agents): replace Unicode ellipsis with ASCII in contributing.md
Address review feedback from gemini-code-assist on docs/agents/contributing.md:88: … replaced with ... for compatibility across editors and tools. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
c8ed1c652c |
chore(ci): adopt CNCF/k8s label conventions
Add .github/labels.yml as the canonical label set, synced into the repository by .github/workflows/labels.yaml using EndBug/label-sync. Conventions follow the Kubernetes scheme: https://github.com/kubernetes/test-infra/blob/master/label_sync/labels.md Six namespaced groups: kind/, priority/, triage/, lifecycle/, area/, do-not-merge/. Cozystack-specific labels preserved (epic, community, security/*, size:*). Migration via aliases keeps references on existing issues and PRs: - bug -> kind/bug - enhancement -> kind/feature - documentation -> kind/documentation - question -> kind/support - frozen -> lifecycle/frozen - stale -> lifecycle/stale - do-not-merge -> do-not-merge/work-in-progress delete-other-labels is false on the initial rollout; redundant labels ("do not merge", duplicate, invalid, wontfix) stay until a follow-up PR removes them after stabilisation. The labels workflow has a validate job (python3 schema check) that runs on PR. Sync runs only on push to main, weekly cron, and manual dispatch. Schema invariants: - description <= 100 chars (GitHub REST API limit) - color is 6-char hex without leading # - unique top-level names - aliases do not collide with top-level names PR auto-labeling (.github/workflows/pr-labeler.yaml): - Parses PR title as Conventional Commits header (type, scope, !). - type -> kind/* (feat -> kind/feature, fix -> kind/bug, docs -> kind/documentation, chore/refactor -> kind/cleanup; style, perf, test, build, ci, revert -> no kind label). - scope -> area/* via embedded mapping; composite scopes split on comma. Bracket-style fallback ([scope] description) maps area/* but cannot infer kind/*. - '[Backport release-1.x]' prefix is stripped; area/release and backport labels are added. - '!' after type or 'BREAKING CHANGE:' footer in body adds kind/breaking-change. - Unmapped scope or non-conventional title adds area/uncategorized to flag for human review. - Additive only — never removes existing labels. Hardcoded label references updated: - .github/ISSUE_TEMPLATE/bug_report.md (bug -> kind/bug) - .github/workflows/tags.yaml (documentation -> kind/documentation) AGENTS.md gains an Activation entry pointing agents to labels.yml as the source of truth and to contributing.md for the title auto-labeling table. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4813566a30 |
docs(agents): mark scopes list as illustrative examples
Address review feedback from gemini-code-assist on docs/agents/contributing.md:11: Scope linters kept flagging valid scopes like 'agents' as unknown because the list read as exhaustive. Annotate it as examples (not exhaustive) and add 'agents' to the Other group so both humans and review bots stop tripping on scopes that are already in regular use across the repo history. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
41fd80711b |
docs(agents): fix grammar in regen discovery hint
Address review feedback from gemini-code-assist on docs/agents/contributing.md:30: Reword "likely to need regenerated" (regional construction) to "likely needs to be regenerated" for standard technical prose. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
c34a9db6bd |
docs(agents): broaden make generate example to apps packages
Address review feedback from coderabbitai on docs/agents/contributing.md:26: Replace the hard-coded packages/extra/<name> path in the example with packages/<apps-or-extra>/<name> so the example matches the preceding text that describes both apps and extra packages. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ecd2ead5de |
docs(agents): document make generate requirement before committing
Pre-commit CI runs make generate in every package and fails with exit 123 on any uncommitted generator output. Add explicit guidance so agents stage regenerated README.md, values.schema.json and packages/system/<name>-rd artifacts alongside the hand edits. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
3c95f30521 |
docs(agents): scope git-write ban to cozystack working tree
Review feedback on PR #2460: the previous "do not write to refs, HEAD, or remotes" wording contradicted the explicit allowance of `git fetch` (which updates remote-tracking refs) and the mandatory cross-repo checks in Step 6, which `cd` into `_repos/<repo>` and run `git checkout`, `git pull`, etc. Sharpen the scope paragraph: - The git-write ban is now explicitly scoped to the cozystack working tree — it bans writing to local branches, tags, or HEAD in that repo, not "refs/HEAD/remotes" globally. - `git fetch` is called out as expected. - Local git operations inside disposable `_repos/` clones (`git checkout`, `git pull`, etc.) are explicitly allowed, with the remaining rules (no push, no PR creation, no API writes) applying to any repository. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
e1c6f9c029 |
docs(agents): scope changelog.md to a file-only deliverable
The v1.3.0 release pipeline broke because Copilot, invoked by .github/workflows/tags.yaml with --allow-all-tools, committed the generated changelog onto HEAD of main on its own. The workflow's next step — `git checkout -b ... origin/main` — then wiped the file, and `git add` failed with a pathspec error. The root cause is in this document. The checklist ends with "Save the changelog", which an agent with broad tool access can reasonably interpret as "also commit it, push it, and open a PR". There was no explicit boundary. Add a "Scope and boundaries" section at the top and an explicit "then exit" at the end of Step 9: - The single deliverable is docs/changelogs/v<version>.md. - Forbidden by default: git commit / push / checkout (to switch branches) / branch / tag / reset / merge / rebase; PR creation; GitHub API writes (POST/PATCH/DELETE); modifying any file other than the changelog. - Read-only analysis (git log/show/fetch/diff, gh pr view, gh api GET) remains expected. - Auxiliary repo clones under _repos/ remain allowed for cross-repo analysis per Step 6. - Scoped "unless the caller explicitly instructs otherwise" so interactive use with an IDE remains flexible. With the rules in the doc, CI and interactive callers share the same boundary; the workflow can invoke the doc with a one-line prompt instead of re-stating the constraints every time. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |