mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-03 10:37:21 +09:00
ci/shellcheck-gate
353
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
c5f352829c |
ci(tests): gate the shell scripts this repository owns on shellcheck
shellcheck ran nowhere: not from the Makefile, not from a workflow, not from pre-commit. Seven files under hack/ nevertheless carried `# shellcheck disable=` directives, twelve of the twenty-eight with a rationale beside them, so the tree was written as though the check existed. hack/shellcheck.sh lints every shell script this repository owns and compares the result with hack/shellcheck-baseline.txt. Enumeration is a union of shebang and `.sh` suffix, and neither half is redundant. The suffix alone misses fifty-seven files with no extension, fifty-three of them the platform migrations run-migrations.sh runs on a cluster upgrade and which no linter had seen before this; the shebang alone misses thirteen held by the suffix, the sourced libraries that open with a `# shellcheck shell=` directive and four with no marker at all. Narrowing to either one loses files, and what the run then reports never names the enumeration as the cause. Measured, and it differs by half: a script that leaves scope carrying baseline entries goes STALE, one carrying none -- sixty-six of the hundred and ninety -- leaves in silence, and dropping the suffix branch also removes the sourced libraries from the command line, so their sourcers gain SC1091 as NEW. The reader is shown a fixed finding to regenerate away, or a new finding to go fix, and in neither case a narrower gate. The baseline records the findings that predate the gate as `path check count` triples. Among the scripts in scope no file is skipped and nothing is excluded on the command line, though in-file `# shellcheck disable=` directives are still honoured, so a finding suppressed that way is absent from the list rather than recorded in it; two classes sit outside the scan and say so in the baseline header, the vendored chart scripts that make update regenerates and the .bats files, whose bats shebang the enumeration does not match. shellcheck parses those perfectly well, with a dialect and checks of its own for them; they are not baselined yet, and the .bats arm of the filter is a guard for one that acquires a shell shebang rather than an exclusion that does anything today. Dropping line and column numbers keeps an unrelated edit from reddening the ratchet, while the count still catches a new finding in a script already listed. An entry whose findings are gone fails the run too, so fixing a script forces the list to shrink rather than leaving a standing licence to reintroduce what was removed. The regenerate advice is withheld whenever the same run also reports a NEW finding. Printed together, the last line read wins, and regenerating a mixed run records the new findings rather than reporting them -- the cure offered for one half launders the other. Three ways the gate could have passed without checking anything are closed. A missing binary fails rather than skips. The version is pinned on both sides and the script refuses to run when they disagree, because findings differ between shellcheck releases and an unpinned binary reports against a baseline not written for it. And shellcheck's own exit status is kept instead of being piped through xargs, which collapses it: a status above 1 means an input was never analysed, and that now fails rather than reading as a clean file. A path holding a space or a tab is refused instead of linted. A baseline entry is three space-separated fields, so such a path cannot be recorded: the summary splits it and the entry names a file that does not exist. Refusing is also what licenses the word splitting the enumeration relies on, which until now was a comment asserting the tree had no such name rather than a check that it does not. hack/shellcheck.bats drives the gate through a mutation that introduces a violation and its removal, both directions of the ratchet, each of the four refusals above, and the enumeration each half of the union catches, the shrink half in the case where a count drops without reaching zero as well as the one where an entry vanishes, and the whitespace refusal on a tab as well as on a space. Every red path asserts the exit status before it asserts the message: `! cmd` alone proves nothing, because `set -e` never fires on an inverted command, and a gate that printed each ERROR line while exiting 0 satisfied all thirteen output greps when that was measured. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrei Kvapil
|
df157da618 |
ci(workflows): open the E2E breakpoint for maintainer-authored PRs (#3158)
## What this PR does Decouples opening the E2E SSH breakpoint from the `debug` label. Today, the only way to open a breakpoint on a failed feature PR is to add the `debug` label. That same label also flips every job's `runs-on` onto self-hosted runners, so you cannot get a breakpoint on the normal ephemeral runner — debugging always drags the whole pipeline onto self-hosted infrastructure. This adds a maintainer-authorship path to the breakpoint's `if` condition: a failed PR whose author is an `OWNER`, `MEMBER`, or `COLLABORATOR` now opens the breakpoint automatically, with no label required and without moving the jobs onto self-hosted runners. The breakpoint simply holds the default `oracle-vm` ephemeral runner the job already runs on. The existing `debug` and `release` label paths are unchanged, and the `authorized-users` list still controls who may actually SSH into an open breakpoint (second defense layer). Workflow triggers are untouched — this is purely an `if`-condition change. ### Release note ```release-note ci(workflows): open the E2E SSH breakpoint automatically on failed maintainer-authored PRs, without requiring the `debug` label or moving the jobs onto self-hosted runners ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Updated E2E failure debugging so breakpoints can now open for pull requests authored by repository maintainers, not just when specific labels are applied. * Clarified the conditions and runner behavior for breakpoint access in the workflow documentation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
dfb1a41f28 |
ci(release): refuse a build-release dispatch from a non-line ref
The workflow_dispatch trigger added for the freeze takes any ref, and the "Run workflow" ref selector preselects the default branch — which is exactly where this workflow's own recovery advice, and cut-prerelease's warning, send an operator. IMAGE_TAG is github.ref_name, so a dispatch from main spends a two-hour `make build` republishing cozystack-packages:main and every main image tag while build-main.yaml may be writing the same ones: the 409 tag-collision class #2711 fixed. Nothing is ever mislabeled, since the content really is that ref, so the cost is a wasted runner rather than a wrong artifact. Guard it anyway: it is five lines, and the mis-click is easy. Anchored at both ends so the per-release staging branches (release-1.6.1, release-1.6.0-rc.4) do not qualify — those are built by tags.yaml — and gated on ref_type so a tag named release-1.6 cannot pass the pattern on its own. Runs before the checkout and the OCIR login, so a refused dispatch never touches the credentials. Fails rather than skips: a skipped job reads in the run list like one that produced the artifact. The push trigger was already filtered to release-X.Y; this closes the dispatch path to the same set. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
e8675dd321 |
docs(release): reword the changelog-backstop comments for the freeze model
Three comments still described the changelog reaching main by default, with a release-X.Y-targeted promote PR as a patch-release corner case. After the freeze that is inverted: promote-rc.yaml bases the promote PR on release-X.Y whenever the branch exists, and cutting vX.Y.0-rc.1 creates it, so every release merges its changelog onto the maintenance line and none reach main on the way. The tags.yaml backstop is now the normal route, and its self-skip on `exists` is the exception. These comments are the first thing anyone debugging a missing changelog reads, so pointing at the wrong model of which path is normal costs real time. Reworded in place: the generate-changelog job header, the three-paths comment above check_changelog, its inline at_tag note, finalize's explanation of why update-releasenotes.yaml is not trusted with the release body, and the contract test that pins the porting behaviour. Prose only, no behaviour change. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
5ed6b2204a |
ci(release): build the packages artifact when a freeze creates a line
build-release.yaml publishes `cozystack-packages:<line>` on push to a release line, and pull-requests.yaml's overlay reads it so a release-line PR tests its own line's binaries instead of main's (#3471, #3437). It cannot fire for the push that creates the line. The freeze points release-X.Y at a commit that is already on main, so the push carries no new commits, and GitHub does not run a workflow whose paths/paths-ignore filter finds no changed files ("If there are no files changed, the workflow will not run"). The line would therefore have no artifact until its first cherry-pick merged, and in that window the overlay finds nothing to pull and leaves every package on its committed ref — at freeze time the previous release's, which is exactly the cross-generation mix #3437 fixed. The old flow had no such window: it branched at the promote merge commit, which carried real commits and its own release's refs. Add workflow_dispatch to build-release.yaml and have the freeze step dispatch it for the branch it just created, so the artifact exists from the moment the line does. The dispatch is non-fatal: without it the overlay no-ops and early cherry-pick PRs test their committed refs, which is where they were before #3471. The tag is pushed and the line is frozen by that point, so failing there would misreport both. The trigger also gives a line build a re-run button, which previously needed an empty commit pushed to the line. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
66ed32c6f9 |
ci(release): print the recovery command if the freeze push fails
Address review feedback from lexfrei on .github/workflows/cut-prerelease.yaml:293: the tag push and this branch push are independent, so a transient failure here leaves the line tagged but not frozen, and the write-once tag guard refuses a same-tag re-dispatch — recovery is manual. Fail with the exact push command rather than leaving the operator to reconstruct it. Deliberately not an atomic two-ref push: the tag push relies on being anchored to the branch tip for GitHub to populate the push event's base_ref, which tags.yaml requires, and whether a multi-ref atomic push preserves that is only observable on a live release cut. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
a59ad88eb5 |
ci(release): distinguish absent from broken in the freeze branch probe
Address review feedback from lexfrei on .github/workflows/cut-prerelease.yaml:286: the release-X.Y existence probe piped ls-remote straight into cut, so a transport or auth failure produced empty output and read as "branch not found", steering into the create path. The write-once tag guard, the stale-tip guard, and the refuse step above all make this distinction; match them here. Behaviour is unchanged on the happy path — pipefail already failed the step on a real error, and the non-forced push below fails closed on a conflict. This makes the intent explicit rather than incidental. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
6ff5b98d56 |
ci(backport): target the newest existing release line
Determine the backport targets by enumerating real release-X.Y branches instead of deriving them from getLatestRelease. getLatestRelease returns the newest published stable. Now that cutting an rc creates release-X.Y before vX.Y.0 exists, that name lagged by a full line for the whole freeze window: a `backport` on a fix for the release being stabilised would have landed on the previous line and missed the release it was written for. The freeze window is when backports matter most, since the frozen branch is the only way in. Also drops the contiguous-minor assumption — `previous` was computed as min-1, naming a branch that does not exist whenever a minor is skipped. Sorting is numeric, so release-1.10 correctly outranks release-1.9. Outside a freeze window the result is unchanged: with release-1.0..1.5 present it still resolves to release-1.5 / release-1.4. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
53a7fc4dcd |
feat(release): cutting an rc freezes the line into release-X.Y
Cutting the first vX.Y.0-rc.N now creates release-X.Y at the tagged commit, and every later cut for that line must be dispatched from the branch — a dispatch from main is refused. Before this, only the stable promote-PR merge created release-X.Y, so rc.2 and rc.3 were cut from main's tip and silently picked up every feature merged since rc.1. The release therefore shipped content no rc had ever validated. Freezing at the rc closes that: main stays open for the next minor, and fixes reach the release by cherry-pick onto the branch. promote-rc.yaml needs no change — it already prefers release-X.Y as the promote PR base when that branch exists, so the stable release is now cut from the frozen tree. Gated to -rc (alpha/beta are pre-freeze builds off an open main) and to patch 0 (a patch-line rc is cut from a release-X.Y that already exists). The branch is create-only and never force-moved: moving it would re-open the freeze and drag in whatever the new tip holds. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
f1f383626e |
fix(ci): overlay images from the PR base branch, and publish per-line artifacts (#3471)
## What this PR does
The PR finalize job overlays image refs for every package a PR did not
rebuild, so e2e and the installer do not test last-release images for
everything outside the PR's build matrix. It could only ever read
`cozystack-packages:main`, which meant a **release-line PR was handed
main's binaries to run against its own line's charts**.
cozystack/cozystack#3437 is the demonstration: a one-line change on
`release-1.6` deactivating an app failed install deterministically,
twice, with
```
helmrelease/backupstrategy-controller: Helm install failed …
error validating data: SchemaError(github.com/cozystack/cozystack/pkg/apis/core/v1alpha1.Option.spec):
unknown model in reference: "github.com~1cozystack~1cozystack~1pkg~1apis~1core~1v1alpha1.OptionSpec"
```
main's `cozystack-controller` served an aggregated OpenAPI that
`release-1.6`'s charts could not validate against. Nothing in that PR
was broken; the lane was. Both branches carry `option_types.go` and key
`OptionSpec` identically in the committed generated OpenAPI, so this is
a generation mismatch at runtime, not a codegen drift.
This PR fixes it by giving every base branch its own artifact to overlay
from, rather than by turning the overlay off.
### The change
1. **`pull-requests.yaml`** reads `cozystack-packages:${BASE_REF}`
(`github.base_ref`) instead of a hardcoded `:main`.
2. **`build-release.yaml`** (new) publishes that artifact for maintained
`release-<major>.<minor>` branches exactly as `build-main.yaml` does for
main: every image tagged with the branch, and the whole packages tree
pushed with each reference digest-pinned to what the run just built.
An earlier revision of this branch simply skipped the overlay for
non-main bases. That fixed the wrong-images problem but left
release-line PRs testing their line's *last release*: for any package
the PR did not rebuild, the committed ref is the released digest, so a
component changed by an earlier backport was still exercised as its
pre-backport binary until the next rc. Per-line artifacts remove that
gap too, which is why the guard was replaced rather than kept.
### Three deliberate choices
**The trigger matches line branches only** (`release-[0-9]+.[0-9]+`).
The per-release and rc staging branches `promote-rc.yaml` and
`tags.yaml` create — `release-1.6.1`, `release-1.6.0-rc.4` — must not
trigger a full rebuild; their images come from the tag build and
rebuilding them is waste.
**`WRITE_CACHE` stays `0`.** `CACHE_REGISTRY/<img>:buildcache` is a
single ref per image and `build-main.yaml` is deliberately its only,
serialized writer so concurrent builds cannot race on the cache manifest
— the 409 class #2711 fixed for image tags. A line build can overlap a
main build, so writing here would reintroduce that race. Line builds
read the cache.
**A missing artifact still degrades to committed refs, but says so.** On
a release line it emits a `::warning::` naming the branch. Silent
degradation is indistinguishable from a working overlay, which is how a
mis-specified branch filter could hide for a whole release cycle.
### Cost
One `make build` per push to a maintained line — in practice per merged
backport. That is the price of release-line PRs testing their line's tip
instead of its last release.
### Verification
`hack/overlay-main-images_test.bats` pins the artifact tag to the base
branch, rejects a hardcoded `:main` in either overlay step, and pins
`build-release.yaml`'s branch filter, image tag and `WRITE_CACHE: '0'`.
Mutation-checked: restoring `:main`, setting `WRITE_CACHE: '1'`, and
broadening the filter to `release-*` each fail a test. 13/13 green;
`actionlint` and `zizmor` clean.
Worth an explicit ack in review: the branch-filter pattern is the one
thing no local test can prove, since only GitHub evaluates it. If it
does not match, `build-release` never runs and the new `::warning::` is
what surfaces it on the next release-line PR.
### Backport
`release-1.6` needs this too — for `pull_request` events GitHub builds
the workflow from the merge ref, so a release-line PR only stops
receiving main's images once the change is on its base branch.
cozystack/cozystack#3472 carried the interim guard and is closed in
favour of backporting this instead.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a “Build release line” workflow that builds and publishes images
and the packages artifact for maintained `release-<major>.<minor>`
branches.
* **Bug Fixes**
* Updated PR workflow finalization to use base-branch–specific package
overlays, avoiding incorrect main-branch image references when targeting
release branches.
* Improved fallback behavior when the base-branch packages artifact is
unavailable.
* **Tests**
* Added workflow wiring tests to verify base-branch artifact usage and
that each maintained release line publishes its own packages artifact.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Timofei Larkin
|
e5d4940338 |
chore(codeowners): don't gate per-app generated files on architect review
api/apps/v1alpha1/*/types.go and packages/system/*/cozyrds/*.yaml are entirely produced by `make generate` from each app's own values.yaml, but the /api/ codeowner rule pulled every app's regenerated types.go into an architect-only review, regardless of which area actually changed. Correctness there is already enforced by the pre-commit make-generate CI hooks, not by human review, so list no owner for these generated paths and let that check carry the weight instead. Also fixes a stale api/.gitattributes pattern that never matched any real file (zz_generated_deepcopy.go vs the actual zz_generated.deepcopy.go), and adds a matching linguist-generated attribute for the cozyrds ApplicationDefinition manifests, so both collapse in the PR diff view like the Go files already do. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Andrei Kvapil
|
e16df56019 |
chore(codeowners): scope CODEOWNERS to per-area reviewers (#3228)
## What this PR does Replaces the single catch-all owner line in `.github/CODEOWNERS` with area-scoped reviewer ownership, so each part of the tree routes to the people who actually own it (per the Contributor Ladder), instead of requiring the whole roster on every change. Highlights: - Architects (`@kvaps @lllamnyp`) are the default owner and own the API surface (`/api/`) and the VPC app. - CI / automation → `@myasnikovdaniil @sircthulhu`; backups → `@androndo @lllamnyp`; virtualization (kubevirt, vm-instance/disk) → `@scooby87 @lllamnyp @kvaps @myasnikovdaniil`. - `cozystack-api` / controller / lineage webhook → `@IvanHunters @kvaps @lllamnyp`. - General packages / apps / CLI ownership → `@lexfrei @IvanHunters @myasnikovdaniil` (+ `@scooby87` for apps), with `kubernetes` and `extra` scoped separately. - Flux → `@kingdonb`; GPU (hami, gpu-operator, GPU dashboards) → `@lexfrei`; metrics-server and seaweedfs → `@nbykov0`. - Governance / licensing files → `@tym83 @kvaps`. Ordering follows "last match wins" (broad → specific). The API review gate block is kept last so it cannot be overridden. This is review-routing only — no runtime/functional change — and only binds once "Require review from Code Owners" is enabled in branch protection. This file is what the CNCF Incubation review reads as "code ownership", so it is paired with #3462, which documents the same people in `MAINTAINERS.md` and introduces the Reviewers table the ladder already defined. Merging one without the other leaves the two describing different projects, which is the gap both PRs exist to close. Three of the people named here — `@kingdonb`, `@matthieu-robin`, `@mattia-eleuteri` — hold only read access today, and a CODEOWNERS entry for someone without write is silently ignored by GitHub, so the org-side permission changes have to land alongside. ### Screenshots N/A — governance/config change, no user-facing or UI impact. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff: no row matches. The change touches only `.github/CODEOWNERS` — no package, no `values.schema.json`, no `ApplicationDefinition`, no platform variant, no release asset, no node prerequisite. Every path named here was checked against current `main`; all 65 exist. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated CODEOWNERS from a single catch-all to an ordered, path-scoped reviewer mapping. * Clarified “last matching pattern wins” behavior and ensured the API review gate rules remain last to avoid overrides. * Expanded explicit ownership coverage across dashboards/examples, packages and Go components, apps, platform/infrastructure modules, CI/automation, and governance/licensing documentation (including SECURITY). <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
2330d6f3ca |
fix(ci): overlay images from the PR base branch, and publish per-line artifacts
Supersedes the interim guard in this branch`s first commit, which skipped the overlay entirely for release-line PRs. Skipping fixed the wrong images but left those PRs testing their line`s last release: for any package the PR did not rebuild, the committed ref is the released digest, so a component changed by an earlier backport was exercised as its pre-backport binary until the next rc. The overlay now reads `cozystack-packages:<base branch>` instead of always `:main`, and build-release.yaml publishes that artifact for every maintained `release-<major>.<minor>` branch the way build-main.yaml does for main: images tagged with the branch, and the whole packages tree pushed with each reference digest-pinned to what the run just built. Each base branch therefore has its own generation to overlay from, which is what the original bug was really about — #3437 failed install because main`s cozystack-controller served an aggregated OpenAPI release-1.6`s charts could not validate against. Three deliberate choices: * The trigger matches line branches only (`release-[0-9]+.[0-9]+`). The per-release and rc staging branches promote-rc.yaml and tags.yaml create (release-1.6.1, release-1.6.0-rc.4) must not trigger a full rebuild — their images come from the tag build, and rebuilding them would be waste. * WRITE_CACHE stays 0. CACHE_REGISTRY/<img>:buildcache is a single ref per image and build-main.yaml is deliberately its only, serialized writer so concurrent builds cannot race on the cache manifest (the 409 class #2711 fixed for image tags). A line build can overlap a main build, so writing here would reintroduce that race. Line builds read the cache. * A missing artifact still degrades to committed refs, but on a release line it now emits a ::warning:: naming the branch. Silent degradation is indistinguishable from a working overlay, which is how a mis-specified branch filter would hide for a whole release cycle. Cost: one `make build` per push to a maintained line, i.e. per merged backport. hack/overlay-main-images_test.bats pins the artifact tag to the base branch, rejects a hardcoded :main in either overlay step, and pins build-release.yaml`s branch filter, image tag and WRITE_CACHE=0. Mutation-checked: restoring :main, setting WRITE_CACHE=1, and broadening the filter to release-* each fail a test. 13/13 green; actionlint and zizmor clean. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
cc44f6c981 |
ci(release): move release validation to rc time — e2e lane, changelog button, promote gate, docs at promote (#3449)
## What this PR does Moves release validation from promote time to rc time, making promotion itself a pure retag operation. This is the follow-up to the v1.6.0 retrospective: the promote flow's checks ran on the promote PR (coupled to promotion, skippable by never promoting), the changelog and website docs were produced only after the release went public, and the finalize tag push silently failed to trigger `tags.yaml` at all (persisted-credential bug, root-caused below). The release lifecycle after this PR: 1. `cut-prerelease.yaml` cuts an rc; `tags.yaml` builds, drafts, pushes the staging branch — and now runs a **mandatory full e2e** for every `-rc.N` tag via the new reusable `e2e-tag.yaml` lane. Alpha/beta (and any published tag) get the same validation as a **manual button** (`workflow_dispatch`). 2. Optionally, a maintainer presses the **`changelog-rc.yaml` button**: the release changelog is generated from the rc tag (reusable `changelog-generate.yaml`) and committed to the rc staging branch, ready for review long before promotion. 3. `promote-rc.yaml` dispatch is **gated on green rc e2e** (evidence: the rc's `tags.yaml` run containing a successful `rc-e2e` job, or a successful manual `e2e-tag` dispatch for the tag; `skip_e2e_gate` input for emergencies, always surfaced in the PR body as ✅/⚠️). Promote stages the release, picks up the rc-time changelog (or generates one), and **opens the website docs PR** on cozystack/website from the staging branch — parked with an explicit DO-NOT-MERGE-until-finalize contract. 4. The promote PR **no longer gets `full-e2e` automatically** — its tree is the rc's already-validated digests with only tag strings rewritten; a maintainer can still add the label manually to force the suite. 5. Finalize performs the irreversible side effects (tag, publish, retag, chart) — and with `persist-credentials: false` its tag push now actually authenticates as the CI app, so the `tags.yaml` backstops (`generate-changelog`, `update-website-docs`) fire for promoted tags. This is the one-line fix for the v1.6.0 incident where the tag triggered nothing: the checkout's persisted `GITHUB_TOKEN` `http.extraheader` silently defeats the app token injected via `git remote set-url`, and `GITHUB_TOKEN` pushes create no workflow runs. Everything is pinned by executable contract tests (`hack/promote-gate-contract.bats`, extended `hack/release-changelog-contract.bats` / `release-changelog-behaviour.bats` — 48 tests total, including behaviour tests that execute the new `hack/parse-rc-tag.sh` and `hack/select-changelog-source.sh` for real), and `docs/release.md` is rewritten to describe the new lifecycle end-to-end. ### Commit map (review commit-by-commit) - `75b5192f1`…`dd308644c` + `6fc970851` — the pre-existing changelog-before-publication rework (changelog generated at promote, committed to the staging branch before the PR opens, finalize publishes it as the release body; tags.yaml job becomes a verbatim-port backstop), rebased onto main. - `5aec70b3a` — `e2e-tag.yaml`: reusable published-tag e2e lane (prerelease → staging branch, stable → tag; draft releases fail fast; dispatch-only SSH `debug` breakpoint; tag-suffixed artifacts). - `025564d96` — `tags.yaml::rc-e2e`: mandatory full e2e for freshly cut rc tags (also on `release_exists` re-runs); fixes the `cut-prerelease.yaml` comment that falsely claimed e2e already existed. - `2dfbeea77` — `changelog-generate.yaml` (reusable core) + `changelog-rc.yaml` (rc-time button committing to the staging branch); promote-rc gains a top-level `parse` job with a strict no-whitespace tag policy (`hack/parse-rc-tag.sh`); source selection is validate-then-fallthrough (`hack/select-changelog-source.sh`) so an invalid higher-priority changelog can never silently win. - `ff7ddd3c9` — the promote e2e gate + `skip_e2e_gate` override + removal of the automatic `full-e2e` label (pull-requests.yaml release-path e2e now requires an explicit label; a post-open label now actually starts the run). - `936190d4b` — `website-docs` job at promote time behind a FETCH_REF-support guard (see dependency below), with the DO-NOT-MERGE-until-finalize contract and a pin-freshness pass before `release-next`; `tags.yaml::update-website-docs` stays as the backstop. - `01e1e7188` — the `persist-credentials: false` finalize fix (v1.6.0 root cause). - `717c8860f` — `docs/release.md` coherence pass over the whole new lifecycle. ### Dependency The promote-time website docs job needs cozystack/website#631 (`FETCH_REF` + fail-loud fetch). Until that merges, the job's guard detects the missing support and degrades to a ⚠️ line in the promote PR body — promotion is unaffected and the tag-time backstop backfills, so this PR is safe to merge first. ### Merge instructions Please **merge-commit, do not squash**: the 13 commits are deliberately atomic (the credential root-cause fix is separable from the feature work) and squashing would collapse ~3k lines into one commit, hurting bisectability. ### First live run The e2e lane is verified by lint, contract tests and dry-traces, but has not yet executed against a real tag; the intended first live run is a manual `e2e-tag.yaml` dispatch against an existing published rc tag before the gate is relied on for a real promotion. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added reusable changelog generation with validation, “present” signaling, and safe artifact upload behavior (including RC-time support). * Added an E2E release-tag workflow with tag parsing, deterministic sandboxing, and comprehensive report collection. * Improved RC promotion with a stricter parse front-door, verified E2E green gate (optional bypass), and earlier parallel changelog/asset staging. * **Documentation** * Refreshed release-process docs to match the updated RC-to-stable, changelog, and website-docs timing rules. * **Bug Fixes** * Improved PR finalization to source release bodies from verified changelogs and warn/fallback on missing or empty content. * Tightened label gating so E2E runs only under the intended conditions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
78fc3f9178 |
refactor(ci): drop the vestigial /tmp workspace from e2e (both workflows)
The E2E job once ran as several state-sharing self-hosted-runner jobs that rendezvoused through a sha256(REPO:WORKFLOW:REF) key and a per-run /tmp/$SANDBOX_NAME copy of the checkout. Since the move to the ephemeral pool it is a single job: there is no cross-job rendezvous and the checkout is never reused, so copying it into /tmp and removing it afterwards is dead weight. Run every step in $GITHUB_WORKSPACE instead: drop the Prepare/Remove workspace steps and the `cd /tmp/$SANDBOX_NAME` prefixes, and point the upload paths at _out/. This deletes the unguarded `rm -rf /tmp/$SANDBOX_NAME` rather than guarding it: when an upstream step fails or the job is cancelled, the unconditional `Set sandbox ID` is skipped, SANDBOX_NAME stays empty, and the `if: always()` steps interpolate it — the removal expanding to `rm -rf /tmp/` and failing the cleanup step. `Set sandbox ID` stays, since the container still needs a name. Apply the same treatment to nightly.yaml, and drop the Makefile dead code the single-runner retirement left behind: the world-writable mkdir/chmod of /tmp/$SANDBOX_NAME (backed a bind mount removed long ago) and the `-e SANDBOX_NAME` container env (nothing in the e2e-sandbox image reads it). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
23f1b926c9 |
chore(codeowners): say that the rules bind today
The header told the reader to enable "Require review from Code Owners" for the rules to take effect. It has been enabled on main all along, so the file reads as a proposal when it is in fact load-bearing from the commit that lands it. Spell out the consequence that follows: because GitHub ignores a handle without write access, scoping a path to someone who lacks it is worse than naming the wrong person. A wrong owner is visible in review; an unhonoured one carves the path out from under the catch-all and gates it on nobody, reporting nothing. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
bf8c5b3fbc |
chore(codeowners): rework ownership from contribution data
Review on #3228 measured authorship and reviews per path and found the rules named people who had not touched the code while leaving out those who had. Rework accordingly. Close the largest fall-throughs to the catch-all: dashboards, examples (with the backups subtree routed to its controllers' owners) and tests. Widen where the data says the list was too short: @lexfrei on the Go source, cozystack-api, backups and extra; @myasnikovdaniil and @lexfrei on managed Kubernetes, which also grows from one directory to the whole area — node pools, resource definitions, kamaji and the CAPI providers resolved to three different owner sets for one feature. Narrow where it was too wide: drop @myasnikovdaniil from kubevirt and kubevirt-cdi at his own request, since he has no activity there. Drop rules that route nothing: /tools/ and /cmd/cozypkg/ have no human activity, and the metrics-server rule named a maintainer moving to emeritus in #3462 with no active owner to replace him -- all three fall back to a broader rule that already covers them. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
c13c6ee2ca |
chore(codeowners): add flux, GPU and storage-metrics areas
Three areas had a clear owner by contribution history and no entry here, so changes to them routed to the catch-all instead of to the person who actually maintains them. kingdonb is the flux owner and holds no write access today, which is the same governance gap the roster PR fixes; the entry lands with it. GPU work (hami, gpu-operator, the GPU dashboards) went to lexfrei when he picked it up. metrics-server and seaweedfs follow nbykov0, with sircthulhu on the extra/ variant he has been carrying. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Myasnikov Daniil
|
fea40d6314 |
fix(ci): only overlay current-main images on main-based PRs
The PR finalize job pulls cozystack-packages:main and repoints every package the PR did not rebuild at current-main images. On a main-based PR that is the whole point: it stops e2e from testing last-release images for everything outside the build matrix. On a release-line PR it is wrong — the committed refs there are not stale, they are that line`s released digests, and the charts are written against exactly those. So a release-line PR was installing main`s binaries onto its own charts, and the mismatch grows with every commit main gains. #3437 is the demonstration: a one-line change on release-1.6 that deactivates an app failed install deterministically, twice, with SchemaError(...core/v1alpha1.Option.spec): unknown model in reference: "...core~1v1alpha1.OptionSpec" from main`s cozystack-controller serving an aggregated OpenAPI that branch`s charts cannot validate against. Nothing in the PR was broken; the lane was. Left alone this makes every 1.6 backport look red, which is when release-branch PRs are busiest. Both overlay steps are now gated on `github.base_ref == main`, so a release-line PR keeps its committed digests — the behaviour that predates the overlay. Retargeting the overlay at a per-line artifact would be better but is not possible today: build-main.yaml publishes only cozystack-packages:main, and the registry carries no release-* equivalent (verified against the packages repo`s tag list: `main` plus per-PR tags, nothing else). hack/overlay-main-images_test.bats pins the wiring per step, so adding a third overlay step without the guard fails the suite. Mutation-checked by removing one guard. 12/12 green; actionlint clean. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
a2667c58ca |
fix(ci): grant rc-e2e the checks:write ceiling e2e-tag.yaml declares
A caller job's permissions are the ceiling for every job in the workflow it calls, and GitHub validates that ceiling when it CREATES the run — before the calling job's `if:` is evaluated, let alone before a step runs. e2e-tag.yaml's e2e job declares `checks: write` for its workflow_dispatch-only breakpoint, so granting only `contents: read` here made the run fail validation with "requesting 'checks: write', but is only allowed 'checks: none'". A startup failure takes the whole run with it, and an `if:` cannot gate it, so this was not limited to rc cuts: EVERY tag push — rc and stable, including the one finalize pushes to publish a release — would have produced no build, no draft release and no staging branch. The comment claimed the opposite: that because workflow_call can never reach the breakpoint branch, the grant was unnecessary. Runtime reachability has no bearing on a static check. e2e-tag.yaml now states the contract for its callers, next to the permission block that was being read as "contents: read is all this needs". hack/promote-gate-contract.bats pins both halves of the pair, and separately pins the `E2E <tag> (full suite)` job name the gate correlates evidence by. Neither is reachable from any PR lane — tags.yaml runs only on tag pushes — so a regression here would otherwise surface at the next release. Mutation-checked: dropping the grant fails the first pin, renaming the e2e job fails the second. changelog-rc.yaml's reusable-workflow call now pins `contents: read` explicitly, matching promote-rc.yaml's identical call. The outcome is the same either way, but the docs describe an omitted block as the repository default rather than the workflow's own top-level block, and this is the file class where that ambiguity just cost a release lane. Reported by CodeRabbit on #3449; blast radius and the missing pins found by an adversarial re-check. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
01e1e7188e |
ci(release): stop finalize persisting credentials so the tag push triggers tags.yaml
pull-requests-release.yaml's finalize job checked out the repo with the default persist-credentials, leaving GITHUB_TOKEN embedded as http.extraheader. Every later tag/branch push re-injects the CI app token via `git remote set-url`, but a persisted extraheader silently wins over the URL credential — so those pushes authenticated as GITHUB_TOKEN. A GITHUB_TOKEN-authenticated push creates no workflow run (GitHub anti-recursion), which is exactly why v1.6.0's stable tag never triggered tags.yaml: the generate-changelog and update-website-docs backstops never ran. Set persist-credentials: false on the finalize checkout, so the set-url app token actually authenticates the pushes — the same handling promote-rc.yaml already uses. Audited every git/gh step in the job: both tag steps set-url the app token before pushing, and Ensure-maintenance-branch / Publish-draft-release / the retag+chart steps use the app token via github-script or a separate registry login — none depends on the persisted GITHUB_TOKEN extraheader. Pin it in hack/promote-gate-contract.bats. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
936190d4b6 |
ci(release): open the website docs PR at promote time
The cozystack/website 'update managed apps reference' PR used to be opened only at tag time by tags.yaml::update-website-docs — after finalize pushed the stable tag, i.e. after the release was already published (too late to review) — and it fetched README content from that stable tag (BRANCH=RELEASE_TAG), which finalize only creates post-merge. Add a promote-time website-docs job to promote-rc.yaml (needs [parse, promote]) that opens the same update-docs-vX.Y.Z PR from the release-X.Y.Z staging branch via the website Makefile's FETCH_REF variable, so the docs are reviewable alongside the promote PR while the stable tag is still unborn. It refreshes the trunk pins from the staging ref before release-next so a new minor inherits fresh Talos/cozystack pins (the v1.6 talos-pin incident class), then runs update-all with FETCH_REF pinned to the staging branch. The job is non-blocking: a FETCH_REF-support guard fails loudly (warn + exit 1) when cozystack/website main predates the FETCH_REF Makefile variable — otherwise make would silently ignore the unknown FETCH_REF override, fall back to the not-yet-created stable tag, and update_apps.sh would warn-and-continue into STUB docs. Any failure surfaces as a website-docs warning line in the promote PR body (plumbed through open-pr via needs.website-docs.result) and never blocks promotion; the tag-time job stays as the backstop and no-ops when promote already did the work. MERGE-TIMING CONTRACT: the website PR must NOT be merged before finalize publishes the release — doing so flips the site's latest-version pointer to a version whose docs and openapi are not on GitHub yet, 404ing every version-pin. The website PR body and the promote PR body both say so in bold. Pin the contract in hack/promote-gate-contract.bats (needs shape, FETCH_REF invocation, the guard, non-persisted checkout credentials, the DO-NOT-MERGE body wording, the promote-PR status line, and the tags.yaml backstop comment) and relax the release-changelog-contract needs pin to tolerate the appended website-docs dependency. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
ff7ddd3c9a |
ci(release): gate promotion on a green rc e2e and drop the automatic full-e2e label
Promotion now assumes checks passed at rc time instead of re-running them. At dispatch, before any staging happens, promote-rc verifies the rc tag has a green full e2e via either evidence path: the rc's tags.yaml push run containing a successful 'rc-e2e / E2E <tag> (full suite)' workflow_call job, or a successful manual e2e-tag.yaml dispatch correlated by its exact tag-bearing job name (immutable rc digests make past evidence valid). No evidence fails the dispatch with both paths and the remediation named. A skip_e2e_gate boolean input bypasses the gate loudly for emergencies; the promote PR body always carries the verification state (verified or bypassed), and 'verified' can only originate in the gate's green branch. The promote PR keeps the release label but no longer gets full-e2e automatically: its tree is the rc's already-validated digests with only tag strings rewritten, so pull-requests.yaml now runs e2e on the release path only when a maintainer opts in by adding full-e2e. The regular-PR path and the ordinary full-e2e forcing are untouched, and the new labeled trigger makes a post-open label actually start the run. A new promote-gate contract suite pins the gate's DAG position, the override default, the label set, the body lines, and the label-gate condition with comment-proof executable-line counts. Assisted-By: GPT-5 <noreply@openai.com> Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
2dfbeea776 |
ci(release): dispatch changelog generation at rc time
Extract the promote-time changelog generation into a reusable workflow (changelog-generate.yaml, workflow_call) and add a dispatchable wrapper (changelog-rc.yaml) so a maintainer can generate the release changelog the moment an rc goes green, while there is still time to review it. The reusable core keeps promote-rc's observable behaviour: detached rc-tag checkout, read-only app token for the AI step, the validate-changelog.sh gate, non-blocking AI failure, and the same changelog-vX.Y.Z artifact. .release-tooling is overlaid from github.ref_name, which under workflow_call is the caller's dispatch ref (and coincides with the reusable workflow's own ref for a local uses:), so "current tooling against an old rc tag" is preserved. Secrets are declared explicitly rather than inherited, so a caller that forgets one fails at parse time. The rc tag is parsed by a shared hack/parse-rc-tag.sh that REJECTS whitespace outright (a stripped 'v1. 6.0-rc.2' would silently become a different tag). promote-rc parses it ONCE in a cheap `parse` job that both `promote` and `changelog` depend on — so they still run in parallel with each other while consuming one validated tag, and a whitespace tag fails the whole promotion rather than letting promote succeed while the changelog job rejects the same input and the PR opens empty. The reusable core mirrors the reject-whitespace policy inline because its parse must run before any checkout. changelog-rc.yaml verifies the rc staging branch release-X.Y.Z-rc.N exists UP FRONT (in parse, which generate depends on), so a missing branch fails in seconds rather than after a 30-minute AI run, then commits the result there as docs/changelogs/vX.Y.Z.md idempotently (compare-before-push). promote-rc's changelog job is a thin uses: call. Source selection — validate-then-fall-through across the rc tag tree, the rc staging branch (the rc-time pickup), then AI generation, discarding any invalid source with a loud warning so it never suppresses a valid lower-priority one — lives in hack/select-changelog-source.sh, executed after the tooling overlay. open-pr carries the picked-up file onto release-X.Y.Z and finalize publishes it. Contract and behaviour bats are repointed and extended: the shared parser, the source-selection script, and promote's parse policy are all exercised by execution on adversarial inputs; the remaining grep pins match executable lines with exact counts so a comment or dead block cannot satisfy them. docs/release.md lists the new workflow and the rc-time entry point. actionlint and zizmor (default and pedantic) are clean. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
025564d961 |
ci(release): make full e2e mandatory for freshly cut rc tags
Wire the published-tag e2e lane into tags.yaml as a leaf rc-e2e job: needs prepare-release, fires only for -rc. tags (alpha/beta stay button-only via e2e-tag.yaml dispatch), and calls the reusable lane with the pushed tag. The staging branch and published rc release both exist before the job starts because needs waits for the whole prepare-release job (Publish rc release + Create release branch). rc-e2e deliberately also runs when prepare-release short-circuits via release_exists: that flag means skip-rebuild, not skip-validation, and revalidating a published rc is idempotent. The caller grants only contents: read — the lane's checks: write is needed solely by its dispatch-only breakpoint, unreachable under workflow_call. Also correct the cut-prerelease handoff comment, which claimed tags.yaml already ran e2e (it never did). Assisted-By: GPT-5 <noreply@openai.com> Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
5aec70b3ae |
ci(release): add published-tag e2e validation lane
The release bits currently get their only e2e on the promote PR, which couples validation to promotion and leaves alpha/beta tags with no e2e path at all. This lane decouples them: a workflow_call + dispatch workflow that validates any PUBLISHED release tag — the digest-pinned tree (staging branch for prereleases, the tag itself for stable), the release's nocloud disk asset, and the full Chainsaw suite, mirroring the proven release-path e2e from pull-requests.yaml (no TIA, no PR context). Draft releases fail fast with a clear message: checks in the new release model happen at rc time, not at promote time. A dispatch-only debug input exposes the same SSH breakpoint as the PR lane; workflow_call runs can never trip it. Artifacts are tag-suffixed. Wiring it as the mandatory post-rc-cut validation lands separately in tags.yaml; for alpha/beta/stable this workflow is the manual button. Assisted-By: GPT-5 <noreply@openai.com> Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
dd308644c8 |
ci(release): execute the release-changelog logic in tests, not just grep it
The existing suite asserts on workflow YAML. That catches a rename or a deleted `if:` and nothing else: it cannot tell whether the shell inside a `run:` block does what the step name claims, and it stayed green while the logic underneath was wrong — twice, in the two preceding rounds. Add hack/release-changelog-behaviour.bats, which runs things for real. The GitHub Actions semantics the whole non-blocking design rests on are now verified by executing them. hack/testdata/needs-semantics.yaml mirrors the promote / changelog / open-pr shapes with a fatal (non-AI) step in the changelog job, and `act` runs it. Result: open-pr DOES run when a job it needs has failed, `needs.changelog.result` is `failure`, and — the part that could not be read off the YAML — a failed job's outputs come back EMPTY, while a successful job's propagate. That empty-output behaviour is precisely why open-pr re-establishes changelog presence from disk instead of trusting needs.changelog.outputs. Deleting the `if:` from the fixture makes open-pr get skipped, so the gate is demonstrably load-bearing. The git logic moves into hack/changelog-preserve.sh and is exercised against real bare remotes: a branch that exists, one that does not, one with no changelog, one with an empty changelog, and the full preserve/checkout -B/restore/force-push cycle promote actually performs. The last of these directly reproduces the regression the preserve step exists to prevent. An empty changelog is deliberately NOT preserved — it would suppress generation and then publish as blank notes. finalize's release-body selection runs under node across all three inputs (good, whitespace-only, absent) and the resulting decision table is compared as a whole. Replace the shell overlay with a sparse second actions/checkout of the dispatch ref at .release-tooling. The overlay had a bootstrap problem I introduced: the script that fetches missing tooling was itself missing from the rc tree, so it could never have worked for exactly the trees that need it. actions/checkout comes from the runner, not the repo, so it has no such circularity. .release-tooling is gitignored because promote runs `git add -A`, and the contract suite pins that. All 21 tests mutation-checked. Breaking changelog-preserve.sh, making it return an empty file, removing the fixture's `if:`, dropping the tooling checkout, and renaming its path each fail the suite and pass on restore. Not covered, and stated in the suite header rather than implied: the AI generation step, the real GitHub API calls, and whether the sparse checkout resolves on GitHub's runners. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
34e45295fe |
ci(release): fix second-round review findings
Two more review passes (Claude + Codex) on the previous commit. Codex found the one that would have broken the very next promotion. The agent reads its own instructions from the working tree, which is the rc tag. Every rc cut before this change carries the old §2 of docs/agents/changelog.md — two valid configurations, "stop" on anything else — and a detached rc-tag checkout is exactly that "anything else". So promoting v1.6.0-rc.4 would have presented a compliant agent with instructions telling it to refuse, silently, producing no changelog on every promotion of an existing rc. Verified against the tag. The same applies to hack/validate-changelog.sh, absent from those trees entirely, which would have failed the Verify step with "command not found" and, in open-pr, taken the promote PR down with it. Both are now overlaid from the dispatch ref, best-effort, in both jobs. The structural validation added last round rejected every PATCH changelog. It asserted a `# Cozystack vX.Y.Z` header and a 20-line floor; patch releases use `# vX.Y.Z (date)` and v1.5.1 is a complete 19-line release. Good AI output would have been discarded and the promote PR mislabelled "generation failed". Measured across the tree: the header form holds for 23 of 115 changelogs, so it was never an invariant. The predicates now live in hack/validate-changelog.sh, tested directly against real changelogs rather than through YAML greps — the reason the defect shipped is that the suite pinned finalize's guard but nothing on the promote-side validator, and stayed green when the validator was mutated away. It accepts both header conventions, has no line floor, and checks the version in the header, the release-link comment and the compare link, so a changelog generated for the wrong version is caught. All 16 current-era changelogs pass; whitespace, empty, truncated and wrong-version inputs are rejected. Also fixed: - A fatal artifact download in open-pr would abort the job before the PR step — the same wedge as a skipped job, one step lower. Download is now best-effort and the commit and PR body key off what is on disk, not an upstream job output that says nothing about whether the transfer worked. - A pre-existing changelog (hand-written, or preserved across the force-push) reached the PR without ever being validated, was advertised with ✅ and published verbatim. It now goes through the same validator, as does the tags.yaml backstop. - `gh pr view <branch>` matches closed PRs, so re-dispatching after closing an abandoned promote PR edited the dead PR and opened nothing. Branch on state explicitly. - `gh pr edit --body` unconditionally overwrote the PR body, destroying the upstream breaking-change notes docs/release.md instructs reviewers to paste there. Refresh only while the body is still the generated one. - open-pr consumed raw dispatch input while its siblings used the validated parse; the changelog job's bash parse did not trim, so an input with a stray space passed promote and failed there. Both fixed. - The prompt and §2 said "trailing release-link comment"; it is the first thing in every changelog. Tests grow to 16, each mutation-checked: dropping the overlay, making the download fatal, reverting the existing-changelog check to `-s`, and removing the backstop validator each fail the suite. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
f49d54a689 |
ci(release): fix review findings; keep website docs at tag time
Dual-model review (Claude + Codex) of the previous two commits found seven substantive defects, two of them destructive. Both models independently reached three of them; the split is noted per item. Descope the website half (Claude, verified by reproduction). The docs job was moved to promote time, but the website Makefile pins the fetch ref with `override BRANCH := $(RELEASE_TAG)`, which defeats the command-line BRANCH assignment. `make -p` resolves BRANCH to v1.6.0 — a tag that cannot exist at promote time, since promote's own validate step hard-fails if it does. update_apps.sh logs the failed fetch and exits 0 AFTER truncating each destination file, so every app doc would become a stub, `changed=true` would open the PR, and the new auto-merge would land it unreviewed. Not fixable from this repo. update-website-docs returns to tags.yaml where RELEASE_TAG resolves to a real tag, and the finalize auto-merge is dropped. Opening it earlier needs a cozystack/website change first and is left as a follow-up. This also removes the unordered-sibling race Codex found, where the release PR could merge before the docs job opened its PR, leaving finalize nothing to merge and no retry. Fix the non-blocking guarantee (both models). Only the AI step carried continue-on-error; the changelog JOB did not, so a flaky `npm i -g @github/copilot` failed it and `needs` skipped open-pr entirely — staging branch pushed, release drafted, no promote PR. Gate open-pr on promote alone. Generate from a checkout the contract permits (both models). The job checked out the rc staging branch, which docs/agents/changelog.md §2 rejects — a compliant agent stops and writes nothing, so the normal path produced no changelog at all. Check out the rc tag detached and document that configuration in §2. The checkout ref now comes from the validated parse output rather than raw dispatch input. Port, don't regenerate, in the backstop (both models). For a patch release promoted onto release-X.Y the changelog never reaches main, so the backstop fired and ran a second AI pass; once its PR merged, update-releasenotes.yaml overwrote the published body with unreviewed text. Detect the file at the tag and port it verbatim; the AI runs only when it is absent from both main and the tag. Survive re-dispatch (Codex). promote force-pushes the staging branch, erasing a hand-written changelog — the documented recovery when generation fails. Preserve an existing one across the rebuild, and stop open-pr overwriting it with generated output. Validate structurally, not by size (Codex). A truncated or whitespace-only AI run passed `[ -s ]`, was committed, advertised as valid in the PR body, and published verbatim as release notes. Assert the header, the compare link and a plausible line count; finalize additionally refuses whitespace-only content rather than blanking the notes. Add hack/release-changelog-contract.bats — nine tests pinning each link in the chain, mutation-checked: removing open-pr's if:, breaking finalize's body read, and restoring paths-ignore each fail the suite. Update docs/release.md (~10 sites) and docs/agents/changelog.md §2, which described the old flow throughout. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
ba67fea7df |
ci(release): drop paths-ignore from the release finalize trigger
The promote PR now carries docs/changelogs/vX.Y.Z.md, and Prepare stable branch explicitly tolerates producing no tag-string changes when the digests are already stable. Together those can yield a docs-only promote PR, which `paths-ignore: docs/**/*` would filter out — finalize never runs, so the stable tag is never cut and the release is never published, with no error raised anywhere. The `if:` gate (merged + `release` label + release-bot author) is what actually decides whether this workflow does anything, so let it do the filtering. The cost is a skipped job on unrelated closed PRs. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
75b5192f1b |
ci(release): produce changelog and docs before the release goes public
Changelog generation and the website-docs PR both lived in tags.yaml, gated on the stable tag push. Under the promote flow that tag is created by pull-requests-release.yaml finalize, which also publishes the release — so both jobs could only ever run once the release was already public. A reader hitting the release in that window saw an empty body, and the site lagged until someone merged the docs PR. Move both to promote time: - promote-rc.yaml gains a `changelog` job that runs in parallel with `promote` and hands the file to a new `open-pr` join job, which commits it onto the release-X.Y.Z branch so it rides the promote PR. Generation is AI-driven and slow while `promote` spends its wall clock on the Talos assets; neither needs the other, so both start at once. - The website-docs job moves to promote-rc.yaml too, depending only on `promote`. Different repo, so it cannot ride the PR — it is opened early and finalize merges it after the release publishes. - finalize reads docs/changelogs/vX.Y.Z.md from the merge commit and uses it as the published release body. update-releasenotes.yaml is no longer on the critical path: it triggers on the same push and would race this job, and if it won it would find no published release, skip, and never fire again. It also only watches main, so a patch release promoted onto release-X.Y was never synced at all. - tags.yaml keeps generate-changelog as a backstop only. It self-skips when the changelog is already on main, and covers the two cases where promotion does not put it there: generation failed (non-blocking), or the promote PR targeted a maintenance line. Generation stays non-blocking, so a flaky AI step cannot wedge a promotion. When it fails the promote PR body says so explicitly, and the escape hatch is to commit docs/changelogs/vX.Y.Z.md to the branch — which also makes a hand-written changelog first-class: if the file is already in the rc tree, generation is skipped entirely. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
baf0158d81 |
fix(release): reuse only an OPEN promote PR, not one in any state
The promote step guarded PR creation with `gh pr view <branch>`, which resolves a pull request by head branch in any state. A closed one satisfied the guard, so creation was skipped while the step still exited 0 — a promotion that reports success and leaves nothing to merge, the same silent-skip shape as the enumeration bug fixed in #3404. STABLE_BRANCH is derived from the stable version, so every rc promoted to that version shares it. Both re-dispatch paths this workflow documents as supported are therefore wedged once any promote PR for that version has been closed: re-dispatching the same rc, and promoting a newer rc to the same version. Neither recovers by reopening the old PR, because the step's own force-push makes its head unreachable and GitHub refuses with "state cannot be changed. The <branch> branch was force-pushed or recreated." Hit live while promoting v1.6.0-rc.4 after #3397 had been closed: the run went green and opened no PR. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
d0c0f399f8 |
fix(ci): cover every image-ref storage shape in promote, retag and mirror (#3404)
## The bug Promotion rewrites the rc version substring to the stable version across the vendored image references. That rewrite lived inline in `promote-rc.yaml` and globbed the depth-2 package `values.yaml` plus `packages/apps/kubernetes/images/*.tag` **only**, on the premise that the kubernetes app was the only one whose `.tag` files carry the cozystack version. Nine other `.tag` files do, and `system/multus` seds its reference straight into a vendored upstream daemonset manifest. #3397 (promote `v1.6.0-rc.4` → `v1.6.0`) is therefore staged with **33 of 55 first-party references still reading `v1.6.0-rc.4`**. `hack/promote-retag.sh` and `hack/nightly-mirror.sh` had the same blind spot from the other direction — both scanned the depth-2 `values.yaml` alone, so those images were never retagged to the stable version (**30 references selected before, 42 after**) and never mirrored to the public registry for a nightly. A retag miss leaves an image short of a tag, since the digest still resolves inside one registry. A mirror miss is worse: the host rewrite walks the same file list, so a reference that is neither mirrored nor rewritten leaves the published tree pointing at the private build registry. `v1.6.0` is the first release to go through promotion — every earlier release was a full rebuild from the tag, which stamped every file uniformly. The glob has always been wrong; the old path just made it unobservable. ## Why it could not be caught before The rewrite was workflow-inline, so there was nothing to unit test. The only way to observe the miss was to cut a release. It is a script now — `hack/promote-rewrite-tags.sh` — and `hack/promote-rewrite-tags_test.bats` round-trips it against the real tree under `make unit-tests`, with no cluster, registry or release involved. ## The fix The three call sites each carried their own idea of where a reference can live, which is how they drifted apart. That knowledge moves into `hack/lib/image-refs.sh` as the single enumeration — `image_ref_files()` for the files, `collect_image_refs()` for the references inside them — and all three source it, so a new storage location is declared once and reaches every consumer. The rewrite also asserts its own postcondition: it scans **wider** than it rewrites and fails the promotion when a reference survives in a location the enumeration does not know about, rather than shipping it quietly. The wide scan is shape-filtered to image-reference position, because `X.Y.Z-rc.N` is an ordinary version string that other things legitimately contain — `kubevirt-csi-driver/go.sum` pins `github.com/golang/protobuf v1.4.0-rc.2`, and `v1.4.0-rc.2` is a cozystack rc that was actually cut. Narrowing it to `${RC}@sha256:` instead would have been wrong: `system/cilium/values.yaml` keeps `tag:` and `digest:` under separate keys. `docs/agents/image-refs.md` records the contract — three tag classes, three storage shapes, the invariants, and which consumer does what. ## Not a bug: kamaji `cluster-api-control-plane-provider-kamaji` stays at `v0.19.0-cozystack.0` across the promotion. It is a first-party rebuild of an upstream component, versioned by that component rather than by the cozystack version line, so no rewrite is owed. A test pins this so a future fix here cannot over-reach. ## Documented known gaps, deliberately not fixed here Both are pre-existing and in the **host** dimension; the version dimension this PR owns is complete, because the version always lives in a `tag` key regardless of where the host sits. - **The nightly host rewrite cannot reach a split host.** It is a literal `<src-registry>/` substring replace, so `keycloak-operator` (host in a sibling `registry:` key) and `kubeovn` (`global.registry.address`, no trailing slash) are mirrored but keep pointing at the build registry in the published tree. Fixing it needs structure-aware rewriting. - **A reference inside a gzip is invisible.** `capi-providers-cpprovider` ships `files/components.gz`, and the promote postcondition cannot catch this class at all since `grep -rIl` skips binaries. Nothing is owed today — kamaji is component-versioned and the image is still mirrored and retagged via its `.tag` — so the residual is that a nightly's kamaji ConfigMap keeps the build-registry host. ## Testing - 17 tests in the new suite; `make bats-unit-tests` green (312 tests, exit 0); `shellcheck -x` clean; `zizmor` clean on the workflow. - Verified against four real rc versions (`1.4.0-rc.2`, `1.4.0-rc.4`, `1.5.0-rc.2`, `1.6.0-rc.4`) in both directions — no false positives, no false negatives. - Every test is mutation-proven. Reverting the enumeration to the old glob, deleting any of the five YAML shapes, corrupting emitted digests, reverting extras to parse-only, dropping the unreadable-file guard, broadening shape 4 beyond `global.images`, or planting an unenumerated first-party reference in any storage shape each fail a named test. ## Reviewer note, out of scope `platform-migrations` is pinned at two different digests — `core/platform/values.yaml` at `v1.5.0` and `backupstrategy-controller/values.yaml` at `v1.4.0-rc.2` (an unowned pin; no Makefile writes that key, tracked in #3143). A `promote-retag.sh` dry-run emits **two copies to the same destination tag**, so the write-once guard will fail the promotion. This is independent of this PR and needs a decision about the backup-client image version, so it is left alone here — but it blocks the 1.6.0 promote and wants fixing before the re-cut. ## Release note ```release-note Promotion, retagging and nightly mirroring now cover every image reference the build stamps, including those stored in `images/*.tag` files and in vendored manifests. Previously a promoted release could ship images still tagged with the release-candidate version, and those images received no stable tag in the registry. ``` Fixes the promote half of #3143. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Image promotion and nightly mirroring now detect digest-pinned image references across values files, standalone tag files, and supported templates. - Stable release preparation supports RC→stable tag rewriting via a dedicated promotion script, with broader verification to ensure no RC references remain. - **Bug Fixes** - Improved completeness of ref discovery and scanning, including previously missed template/tag locations. - **Documentation** - Added and linked guidance documenting image-reference invariants and how tooling enumerates supported storage shapes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
09323fff13 |
fix(ci): cover every image-ref storage shape in promote, retag and mirror
Promotion rewrote the rc version substring across the depth-2 package values.yaml plus packages/apps/kubernetes/images/*.tag alone, on the premise that the kubernetes app was the only one whose .tag files carry the cozystack version. Nine other .tag files do, and system/multus seds its ref straight into a vendored upstream manifest. v1.6.0 was therefore staged with 33 of 55 first-party refs still reading v1.6.0-rc.4. promote-retag.sh and nightly-mirror.sh had the same blind spot from the other direction: both scanned only the depth-2 values.yaml, so those images were never retagged to the stable version (30 refs selected before, 42 after) and never mirrored to the public registry for a nightly. A retag miss merely leaves an image short of a tag, since the digest still resolves inside one registry; a mirror miss is worse, because the host rewrite walks the same file list and a ref that is neither mirrored nor rewritten leaves the published tree pointing at the private build registry. The three call sites each carried their own idea of where a ref can live, which is how they drifted apart. Move that knowledge into hack/lib/image-refs.sh as the single enumeration -- image_ref_files() for the files, collect_image_refs() for the refs inside them -- and have all three source it, so a new storage location is declared once and reaches every consumer. The rewrite itself moves out of promote-rc.yaml into hack/promote-rewrite-tags.sh. Being workflow-inline is why this was only observable by cutting a release: there was nothing to unit test. It now also asserts its own postcondition, scanning wider than it rewrites and failing the promotion when any ref survives in a location the enumeration does not know about, rather than shipping it quietly. Component-versioned images (kamaji's v0.19.0-cozystack.N) and third-party pass-through images do not ride the cozystack version line and are deliberately left alone; a test pins that so a future fix here cannot over-reach. capi-providers-cpprovider is a documented known gap: its ref is also embedded in a gzipped copy that the chart actually ships, and rewriting only the readable copy would diverge the two. docs/agents/image-refs.md records the contract -- three tag classes, three storage shapes, the invariants, and which consumer does what. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
792951b9c6 |
fix(ci): make workflow cancellation effective
Assisted-By: GPT-5 <noreply@openai.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
8020656725 |
fix(ci): resolve nightly packages digest without image-specific skopeo ops (#3332)
## What this PR does
The Nightly workflow has **never completed a run**. Every scheduled run
since `NIGHTLY_ENABLED` was set has failed in `mirror` at "Resolve
source commit and version", ~80s in, with:
```
level=fatal msg="unsupported image-specific operation on artifact with type \"application/vnd.cncf.flux.config.v1+json\""
```
Nothing has ever been published — `cozy-installer:nightly` and
`cozystack-packages:nightly` both 404 on GHCR, and `cozystack-nocloud`
does not exist.
This fixes that failure and the next two behind it.
### 1. The digest lookup could never have worked
`cozystack-packages` is a **Flux OCI artifact** (config mediaType
`application/vnd.cncf.flux.config.v1+json`), not a container image. A
plain `skopeo inspect` parses the config blob as an image config and
fatals on it, so `skopeo inspect --format '{{.Digest}}'` was never
viable against this ref — the `--raw` call on the very next line already
acknowledged this.
Replaced with `skopeo inspect --raw` + `skopeo manifest-digest`, which
hashes the manifest bytes — exactly what the registry addresses the
artifact by, with no image-specific parsing.
This also collapses two registry fetches into one: the digest and the
`org.opencontainers.image.revision` annotation now come from the **same
bytes**, which strengthens the race the original comment describes —
there is no longer any window between pinning the tag and reading it.
### 2. `flux pull artifact` rejects a non-existent output path (×2)
`flux pull artifact --output _baked` fails with `invalid output path
"_baked": no such file or directory`. `_baked` is neither tracked nor
created by anything. `pull-requests.yaml` does `rm -rf` + `mkdir -p`
first; `nightly.yaml` omitted it. Since the workflow always died at step
1, this was never reached.
Same bug for `_rw` in the e2e staging step. Both now create the
directory first.
## Verification
- Reproduced the original fatal against the live artifact with skopeo
1.22.2.
- Ran the fixed resolve step end-to-end against the real registry:
returns the digest the registry reports, and a `sha` that matches the
current `main` HEAD.
- Reproduced the `_baked` failure and confirmed the fix extracts the
tree.
- `actionlint` clean; `zizmor --offline` clean; `shellcheck` clean on
changed lines.
Also checked and **not** changed, each disproven rather than assumed:
- `skopeo copy --multi-arch all` on a Flux artifact — works; `copy` is
mediaType-agnostic.
- The `awk -F@ '/artifact successfully pushed/'` digest capture —
matches current flux CLI output, and is the same idiom
`packages/core/installer/Makefile` already uses in production.
- `nightly-mirror.sh`'s `inspect --format Digest` — all mirrored refs
there are genuine image manifests, so it is correct as written.
- `make -C packages/core/talos talos-nocloud` — exists and is CI-proven
via `tags.yaml`.
## Honest scope
This fixes the `mirror` job's blockers. It does **not** promise a green
nightly on the first run: every step after `mirror` is still unproven,
having never executed. The most likely next wall is `build-disk` —
`cozystack-nocloud` is a brand-new package, so the first `oras push`
creates it, and the follow-up `oras tag` can hit `403 permission_denied`
until the package is linked (the risk the workflow already documents
in-line). The first-ever full e2e run may well surface its own issues.
Reviewing that follow-on separately seems better than growing this PR.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved nightly artifact mirroring and staging reliability.
* Prevented artifact pull failures by ensuring required output
directories are created before use.
* Enhanced retrieval of artifact version and source revision metadata.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
0f4a2715ba |
ci(labeler): map cert-manager, external-secrets and metrics-server scopes
PRs scoped to these three components fell through to area/uncategorized because the scope map had no entry for them. metrics-server serves the monitoring pipeline; cert-manager and external-secrets are platform services. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
90254efa6d |
fix(ci): resolve nightly packages digest without image-specific skopeo ops
The Nightly workflow has never completed a run. Every scheduled run since
NIGHTLY_ENABLED was set has failed in `mirror` at "Resolve source commit
and version", ~80s in:
level=fatal msg="unsupported image-specific operation on artifact with
type \"application/vnd.cncf.flux.config.v1+json\""
cozystack-packages is a Flux OCI artifact, not a container image: its
config mediaType is application/vnd.cncf.flux.config.v1+json. Plain
`skopeo inspect` parses the config blob as an image config and fatals on
any non-image artifact, so resolving the digest with
`inspect --format '{{.Digest}}'` could never work.
Fetch the :main manifest once with `inspect --raw` and derive both the
digest (`skopeo manifest-digest`, which just hashes the manifest bytes —
no image-specific parsing) and the revision annotation from those exact
bytes. This also tightens the race the original comment describes: the
digest and the annotation now come from a single fetch, so there is no
window at all between pinning the tag and reading its revision, and one
registry round-trip is saved.
Because the workflow never ran past this step, the rest was unproven.
Also fix both `flux pull artifact` calls, which passed an --output
directory that does not exist and would have failed with "invalid output
path": create it first, matching the established idiom in
pull-requests.yaml ("Pull current-main packages tree").
Verified against the live OCIR artifact: the fixed step resolves
packages_ref to the same digest the registry reports and yields main's
current HEAD sha; the fixed pull extracts the tree. skopeo copy on Flux
artifacts, the flux push digest awk parse, the cozy-installer chart name
and nightly-mirror.sh's per-image digest verification were each checked
and are correct as-is.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Aleksei Sviridkin
|
90342063c0 |
chore(github): add area/keycloak, area/tenant, area/kamaji labels
Three scopes recur across PR history with no fitting area/* label (keycloak, tenant, kamaji). Add the labels and map their scopes in the auto-labeler, and fill two gaps in existing mappings (mongodb -> area/database, csi -> area/storage) so these stop landing in area/uncategorized. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4cbfb31348 |
chore(github): auto-label new issues triage/needs-triage
Every issue created through a template now gets triage/needs-triage so the untriaged backlog is visible at a glance and the stale bot's 60-day clock has a clear signal for maintainers to act on. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
myasnikovdaniil
|
a269eb3965 |
chore(deps): update actions/checkout action to v6.0.3 (#3133)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [actions/checkout](https://redirect.github.com/actions/checkout) | action | patch | `v6.0.2` → `v6.0.3` | --- ### Release Notes <details> <summary>actions/checkout (actions/checkout)</summary> ### [`v6.0.3`](https://redirect.github.com/actions/checkout/blob/HEAD/CHANGELOG.md#v603) [Compare Source](https://redirect.github.com/actions/checkout/compare/v6.0.2...v6.0.3) - Fix checkout init for SHA-256 repositories by [@​yaananth](https://redirect.github.com/yaananth) in [#​2439](https://redirect.github.com/actions/checkout/pull/2439) - fix: expand merge commit SHA regex and add SHA-256 test cases by [@​yaananth](https://redirect.github.com/yaananth) in [#​2414](https://redirect.github.com/actions/checkout/pull/2414) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cozystack/cozystack). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNDIuMiIsInVwZGF0ZWRJblZlciI6IjQzLjI0Mi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJhdXRvbWF0ZWQiLCJkby1ub3QtbWVyZ2UvaG9sZCJdfQ==--> |
||
|
Aleksei Sviridkin
|
44507d2cd0 |
docs(agents): add downstream repository checklist to the PR template
Cozystack is upstream for repositories that nothing keeps in sync with it, and no CI job compares the two sides, so a change here breaks them in silence. Only two narrow paths are automated, and both fire on a release rather than on a PR. It has already happened. The website docs generator works from an app list hardcoded in its own Makefile, and packages/apps/opensearch was never added to it, so an application we ship has no reference page at all. The Terraform provider still offers a Kubernetes version this repo removed. The plugin repo still sends operators to hack/ scripts that were deleted, one of them rewritten as a Go command. Add a Downstream Repositories trigger map to the contributing guide, naming per repository which change here forces a change over there and which file to touch, and a checklist in the PR template that links to it. The template addresses AI agents directly: they author a large share of PRs and will otherwise tick a box without opening the follow-up it claims. The coupling is wider than the obvious ones, and mostly invisible. talm restates the node contract, overlapping ours but already diverged, so it must be diffed rather than overwritten. cozyhr reads an annotation the operator writes and silently merges the wrong values if it is renamed. cozy-proxy is vendored in from its own repo, yet consumes labels produced here and quietly stops handling external VM services if they move. The telemetry server queries metric names emitted here, and a rename just empties its dashboards. None of these fail loudly. Guard the map with a unit test, because a map about silent drift must not drift silently itself. It pins every in-repo path the map cites to a file that exists, pins the map and the checklist to the same repository list, and pins the workflow carve-out below to the map's location, to the job that reads the flag, and to the output that carries it: cutting any one of those three leaves the other two looking wired up. The header states plainly what the suite cannot check, so a green tick is not mistaken for a correct map: whether a coupling is real, and whether it still holds over there, stays a human's job. Editing the map alone is a docs-only PR, which skips the unit tests and would exempt the map from its own guard. Rather than widen the docs-only rule, which also gates the Talos image build, the plan step exports a separate flag that only the unit-test job reads: a docs PR now runs the tests and still builds nothing. It detects the map by its old path too, since a rename is reported as the new path alone. Document that gh pr create --body and --body-file replace the body wholesale, which would otherwise drop the checklist from every CLI-authored PR. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
renovate[bot]
|
8a5bc68b43 | chore(deps): update actions/checkout action to v6.0.3 | ||
|
myasnikovdaniil
|
99a28d82c2 |
test(e2e): migrate app suite from BATS to Chainsaw and wire into CI (#2826)
## What this PR does Migrates the E2E **app** test suite from BATS to [Kyverno Chainsaw](https://github.com/kyverno/chainsaw) — a declarative, Kubernetes-native E2E framework (CNCF, part of the Kyverno project; successor to KUTTL) — and wires it into CI as the replacement for the per-app BATS loop. This grows the original pilot (postgres + bucket) into the full suite. All 22 app suites are ported under `hack/e2e-chainsaw/` and the `hack/e2e-apps/*.bats` files are removed: `postgres`, `bucket`, `mariadb`, `mongodb`, `redis`, `qdrant`, `clickhouse`, `kafka`, `etcd`, `openbao`, `harbor`, `foundationdb`, `external-dns`, `kuberture`, `vminstance`, `gateway`, `kubernetes-latest`, `kubernetes-previous`, `kubernetes-oidc-system`, `kubernetes-oidc-customconfig`, `securitygroup`, `serviceexposure`. **Approach** - **Declarative** suites assert on `status.conditions` and concrete fields. The `timeout N sh -ec "until kubectl get ..."` + `kubectl wait` pair that appeared ~100 times collapses into a single `assert` that polls existence and state together, with a structured diff on failure and automatic `events`/`describe`/`podLogs` capture via `catch` (previously only `harbor.bats` did this, by hand). - **Imperative** suites keep their logic in `script` steps: `openbao` init/unseal, `kuberture` external-dns split-horizon probes, `vminstance`, the `gateway` admission/impersonation cases, and `kubernetes-latest`/`previous`, which wrap the relocated `hack/e2e-chainsaw/_lib/run-kubernetes.sh` verbatim (Kamaji bring-up, LB/NFS/ouroboros checks). - `gateway` tests derive the tenant apex from the namespace `namespace.cozystack.io/host` label at runtime, so they are host-independent. **CI** - The `e2e` job now runs `chainsaw test hack/e2e-chainsaw/` via a new `test-chainsaw` target and uploads the JUnit `chainsaw-report.xml`. - The `chainsaw` binary is added to the e2e-sandbox image. - `install-cozystack` and `test-openapi` stay BATS (cluster bootstrap + OpenAPI checks). - The per-app 3-retry loop is dropped — assertion polling replaces the fixed-timeout flakiness it papered over. **Validation** Ran against a development cluster: the DB/app, storage, and VM suites pass; the three suites that depend on platform features not present on that cluster (`gateway`, `kuberture`, `external-dns`) and the two heavyweight `kubernetes-*` suites are exercised by this PR's CI run on a freshly installed platform. Note for reviewers: service-port asserts use the `(ports[*].port)` projection form rather than a number-literal filter (`` ports[?port == `N`] ``), because Chainsaw v0.2.15 mis-evaluates JMESPath number-literal comparisons. **2026-07-09: reconciled with main (~520 commits of drift)** - etcd suite re-ported to the v1alpha2 operator contracts (#2859): `readyMembers=3` gate, pod-label/`/scale`/WorkloadMonitor/metrics/defrag contracts, plus the `ETCD_E2E_S3_ROUNDTRIP`-gated backup round-trip driving `examples/backups/etcd` - 12 new `ingress-hostname-policy` gateway cases ported (apex derived at runtime from the tenant-root namespace label) - post-branch bats drift folded in: `mariadb-single` webhook guard, kafka `c1.small` presets, bucket/harbor 2m BucketClaim fail-fast, bucket readonly-denial promoted to a hard fail (cosi-driver v0.3.1), qdrant PVC-reclaim guard (#3059), kuberture fail-loud negations, SC-fallback-default test (#2872 B1) - 4 suites that only exist on main since the branch was cut are ported: `kubernetes-oidc-system`, `kubernetes-oidc-customconfig`, `securitygroup`, `serviceexposure` - CI seams: `nightly.yaml` converted to the chainsaw invocation, `run-kubernetes.sh` reconciled (Talos/CABPT waits, tenant drain, LINSTOR pool wait, talos-image-cache under `_lib/`), `e2e-capture-dataplane.sh` replicated in the Chainsaw global catch ### Release note ```release-note fix(kubernetes): tenant Kubernetes teardown no longer hangs when the cluster has no working nodes — the pre-delete hook now bounds its wait for the in-tenant HelmReleases and force-clears their Flux finalizers on timeout (#3271) ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * End-to-end testing is now driven by a unified Chainsaw suite (platform apps, Kubernetes, networking, security groups, service exposure). * Test Impact Analysis selects affected suites, with an option to run the full suite. * CI produces a JUnit report artifact and collects suite-scoped diagnostics on failure. * **Bug Fixes** * CI E2E now fails fast and emits clearer Kubernetes/HelmRelease and sandbox bucket state diagnostics. * Improved teardown reliability with bounded wait + finalizer handling to avoid hangs. * **Documentation** * Updated E2E testing docs and local guidance to reflect the new Chainsaw-based workflow and suite paths. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
302a72f14a |
Merge origin/main into feat/e2e-chainsaw-example
Reconciles the talos-image-cache conflicts created when #3254 merged to main: keep the Chainsaw _lib/ port (hack/e2e-chainsaw/_lib/talos-image-cache.sh, a superset of #3254's logic + the node-join diagnose helper), drop the BATS-era hack/e2e-apps/talos-image-cache.sh, and keep the _lib/-path versions of the mirror manifest, install bats, and unit test. Also picks up main's zizmor action-pinning (#3223, unblocks pre-commit/zizmor) and Talos v1.13.6 (#3240). Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
renovate[bot]
|
5b9b420ffe | chore(deps): update container image & action digests | ||
|
Andrei Kvapil
|
814a47f23e |
ci: move remaining workflows off the persistent self-hosted runner
Finishes the runner decommission tracked in #2937. PR builds, the main cache warmer, nightly, promote-rc and pull-requests-release already run on the ephemeral pool; this moves the stragglers so no workflow depends on the persistent runner anymore: - tags.yaml prepare-release: large ephemeral shape (same as the cache warmer), flux CLI bootstrap, and read-only OCIR access so release builds warm-start from the shared mode=max build cache while still pushing images to GHCR. Without the cache wiring every tag build on an ephemeral runner would be 100% cold. - tags.yaml generate-changelog / update-website-docs, backport.yaml, update-releasenotes.yaml: GitHub-hosted runners — these jobs only need git and the GitHub API, no docker or repo toolchain. - pull-requests.yaml: the debug label no longer reroutes jobs to the self-hosted runner (that path would queue forever once the runner is gone). The label keeps gating the SSH breakpoint on e2e failure, which works from ephemeral runners too. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
4a2e43969b |
fix(ci): polish cut-prerelease per re-review nits (#3253)
Cosmetic follow-ups from the re-review (all fail-closed, no behavior change): - Write-access gate: drop the inert 'maintain' from the allowed list. getCollaboratorPermissionLevel's `permission` field returns only the legacy roles admin/write/read/none (maintain collapses to write), so admin+write is the full "can push" set; 'maintain' never matched. - Stale-tip guard: resolve the remote tip without a pipe masking ls-remote's exit code, mirroring the write-once query — a transport failure now reports a distinct error instead of "branch not found". - tags.yaml: the hand-pushed-stable rejection message now says stable tags are finalized by the promote flow at the merge commit (pull-requests-release.yaml), not "come from promote-rc.yaml" — promote-rc opens the PR; finalize cuts the tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
31cda39dcd |
fix(ci): address review findings on cut-prerelease (#3253)
Non-blocking review notes from #3253: - Require write access to dispatch. workflow_dispatch already needs write access to trigger, but add an explicit getCollaboratorPermissionLevel gate (admin/maintain/write) as auditable defense-in-depth that fails closed. - Constrain a `main` dispatch to a new minor's pre-release (patch Z == 0). A patch-line pre-release built from main content would produce a wrong-tree release-X.Y.Z-rc.N staging branch; it must be cut from its release-X.Y line. - Guard against a stale checkout tip. workflow_dispatch pins the dispatch-time SHA, not the push-time branch tip (the gap includes the concurrency-queue wait); if the branch advanced, the push would tag a stale commit and yield an empty base_ref, failing tags.yaml after burning the write-once name. Compare local HEAD to the remote branch tip and fail with a re-run hint instead. - Distinguish ls-remote "tag absent" (exit 2) from a transport/auth failure (any other non-zero), so an infra blip cannot read as "free to create". - Shallow checkout (fetch-depth: 1, no fetch-tags) instead of a full-history clone to push one ref; drop the redundant local tag pre-check. - Run pure-input validation (tag + dispatch branch) before minting the token or checking out, so a bad dispatch fails in seconds. actionlint + zizmor clean; run blocks bash -n checked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
bece285678 |
ci: pin action refs flagged by the zizmor gate (#3223)
The zizmor "Audit workflows" gate (added in #3223) is red on main: five actions/* references in the release workflows from #3017 are pinned to a floating tag instead of a commit SHA (blanket unpinned-uses policy, High severity): - nightly.yaml (x3) actions/checkout@v4 - promote-rc.yaml actions/checkout@v4 - retention.yaml actions/create-github-app-token@v1 Pin each to the SHA the rest of the repo already standardizes on: - actions/checkout -> 34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 - actions/create-github-app-token -> d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 `zizmor --offline .github/workflows/` now reports no findings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |