## What this PR does
Restores the two OpenSSF Scorecard checks that dropped the aggregate
score below our target, and adds a durable gate so they cannot silently
regress again.
**Root cause.** 15 of 17 workflows already declare a read-only top-level
token and pin their actions by SHA. Two did not — `build-main.yaml` and
`ui-test.yaml` (the latter added by the in-tree console move) — and that
was enough to zero out two checks:
- **Token-Permissions** is scored on the weakest workflow, so a single
workflow with an undeclared (default-write) top-level token drops the
whole check to `0`.
- **Pinned-Dependencies** counted the six tag-pinned action refs as
unpinned.
**Fixes**
- `build-main.yaml`, `ui-test.yaml`: add a top-level read-only
`permissions` block (job-level `packages: write` kept where the build
cache needs it) and pin every action to the SHA already used elsewhere
in the tree.
- `.github/zizmor.yml` + `.github/workflows/zizmor.yml` +
`.pre-commit-config.yaml`: add [zizmor](https://docs.zizmor.sh/) as a
gate. It runs on the PR diff (pre-commit) and tree-wide on every
workflow-touching PR, failing on any reintroduced unpinned action or
missing/overly-broad permissions block. Scope is intentionally limited
to the two Scorecard-relevant audits (`unpinned-uses`,
`excessive-permissions`), which are clean tree-wide as of this PR; the
remaining audits carry pre-existing debt and are disabled with a
documented TODO to burn them down and re-enable one at a time.
- `scorecard.yml`: wire an optional `SCORECARD_TOKEN` secret. The
default `GITHUB_TOKEN` cannot read branch-protection settings, so the
Branch-Protection check is currently scored on partial data. The token
is a no-op until the secret is added by a maintainer.
- `internal/fluxshardoperator/config_fuzz_test.go`: native Go fuzz
targets for the shard config parsers (round-trip and range invariants,
not just no-panic), giving the Scorecard Fuzzing check a real signal on
a hot-path parser.
**Verification**
- `zizmor --offline .github/workflows/` → *No findings to report* (gate
green).
- `go test ./internal/fluxshardoperator/ -run '^Fuzz'` passes; a 15s
live fuzz of `FuzzParsePinnedTenants` ran 10.5k execs with no crashers.
Follow-ups (tracked separately, not in this PR): merge Renovate's
digest-pinning PRs to lift Pinned-Dependencies further, tighten branch
protection on `main` and release branches, upgrade the OpenSSF Best
Practices badge, and sign releases.
### Release note
```release-note
ci: pin GitHub Actions by SHA and scope workflow token permissions to restore the OpenSSF Scorecard Token-Permissions and Pinned-Dependencies checks, and add a zizmor gate (pre-commit + PR workflow) that blocks future regressions
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automated security scanning for workflow changes via a dedicated
CI workflow, along with a matching pre-commit hook.
* **Bug Fixes**
* Hardened CI workflows with workflow-level read-only token permissions
and pinned GitHub Action versions for more reliable execution.
* Improved OpenSSF Scorecard configuration to better handle
branch-protection checks when a token secret is available.
* **Tests**
* Added Go fuzz tests to ensure shard parsing/formatting remains
non-panicking and preserves round-trip/contract behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Implements the immutable-tag + rc-promotion flow from #2677. A **stable
release becomes a renamed release-candidate**: the bytes shipped as
`vX.Y.Z` are bit-for-bit the bytes built and e2e-tested as
`vX.Y.Z-rc.N`. No tag is ever force-moved, and stable is never rebuilt —
it is *promoted* by retagging the rc's existing images.
Layered onto the build matrix from #2937/#2983 (this PR is **stacked on
`refactor/build-matrix-2937`** and must merge after it).
## The five force-retag sites, removed
| Site | Before | After |
|---|---|---|
| `tags.yaml` api/apps/v1alpha1 tag | `git tag -f` / `push -f` |
write-once (create-if-absent, fail if it would move) |
| `tags.yaml` release-X.Y.Z branch | `git branch -f` / `push -f` |
compare-before-force (no-op if unchanged; staging branch only) |
| `pull-requests-release.yaml` stable tag | `git tag -f` / `push -f` |
write-once at the PR merge commit (force impossible by construction) |
| `pull-requests-release.yaml` maintenance branch | `updateRef
force:true` | fast-forward-only |
| `auto-release.yaml` patch tags | delete-recreate (cron) | **workflow
deleted** — stable only via explicit promote |
## Version decoupling (the enabler)
The operator baked its version into the image at build time, so an rc
image self-reported the rc string — blocking retag-promotion. It now
reads `COZYSTACK_VERSION` from the environment (threaded via
`cozystackOperator.platformVersion` → Deployment env, stamped by `make
manifests`), falling back to the build-time value when unset. The same
image bits can report any release name. The only runtime reader is the
telemetry metric `cozy_cluster_info{cozystack_version=...}`.
## Promotion flow
`promote-rc.yaml` (`workflow_dispatch`, `rc_tag=vX.Y.Z-rc.N`):
1. Validate the rc release exists and the stable tag does not.
2. `hack/promote-retag.sh` reads the rc's digest-pinned image refs from
`packages/*/*/values.yaml` and `skopeo copy`s each — **by digest** — to
`:vX.Y.Z` and `:latest`, verifying with `skopeo inspect`.
3. Rewrite the cosmetic `-rc.N` substring in vendored tags to the stable
version (the `@sha256` wins regardless), restamp the version-stamped
assets (operator manifests, cozypkg, openapi; the heavy Talos assets are
copied verbatim from the rc draft), open a `release-X.Y.Z` PR.
4. Merging the PR reuses the existing release-PR e2e and
`pull-requests-release.yaml` finalize to cut the **write-once** stable
tag at the merge commit and publish the release. Squash is disallowed
(the tag needs a real merge commit).
`nightly.yaml` cuts write-once `*-nightly.<date>` tags (gated by
`NIGHTLY_ENABLED`); `retention.yaml` keeps the newest 14 per line
(dry-run by default).
## Validation
**Local end-to-end — the core claim is proven, not asserted.** The whole
design rests on "copy-by-digest to a new tag preserves the digest, so
stable == the e2e-tested rc, bit-for-bit." This was exercised against a
real registry (`ttl.sh`) with real `skopeo`:
- Pushed a multi-arch image under a `:v1.4.0-rc.2` tag, built a temp
tree exercising all three `values.yaml` digest shapes (single `image:`
string, split `repository`/`tag`/`digest` map, and the
`platformSourceRef` OCI artifact), then ran the actual
`hack/promote-retag.sh v1.4.0`.
- **Result: `:v1.4.0-rc.2`, `:v1.4.0`, and `:latest` all resolve to the
identical digest** (`sha256:fd8d9aa6…`), across all 17 platform
manifests. The script's own `skopeo inspect` post-check passed, and an
independent re-inspection confirmed it.
Local validation caught (and this PR fixes) two bugs that only surface
against a real registry — never in static lint:
1. `skopeo copy --multi-arch all --all` — mutually exclusive flags,
fatal error; every retag would have failed on CI. Now `--multi-arch
all`.
2. The same digest was retagged twice when an image appeared in two
value shapes; the ref set is now deduped on the canonical `repo@digest`.
**Other local checks:** `go test ./pkg/version` + `go build`/`vet`;
`helm template` renders `COZYSTACK_VERSION` in all 3 operator variants
(omitted when unset); `make manifests` stamps the version into the
install assets; `shellcheck` clean on `promote-retag.sh`; `actionlint`
clean and `act -l` resolves the job graph on all workflows; the rc-tag
parser and nightly version-math unit-tested (accept/reject +
minor-vs-patch bumps).
**Still CI-only (cannot be exercised offline):** the full workflow
runtime — github-script API calls, OCIR auth, the rc-release /
staging-branch preconditions, the nightly `base_ref` push mechanics, and
self-hosted runners. The *logic* inside the steps is unit-tested; the
*orchestration* is not. Treat it as unproven until the workflows run.
## Out of scope / required follow-up (needs repo admin — not doable from
a PR)
- Tag-protection rules: `v*` and `api/apps/v1alpha1/*` create-only by
the CI app, no delete/update; "limit branches/tags updated in a single
push" = 1.
- Enable `NIGHTLY_ENABLED` (and `RETENTION_APPLY` when ready) repo
variables.
- Registry-side pruning of `*-nightly.*` image tags (no OCIR delete
parity yet — tracked TODO in `retention.yaml`).
## Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added nightly publishing (mirror-by-digest, disk build, e2e
validation) and nightly retention pruning.
* Added RC-to-stable promotion via digest retagging (no rebuild).
* Operator/installer and console now expose version/platform metadata,
including runtime override via `COZYSTACK_VERSION`.
* **Bug Fixes**
* Enforced write-once tag behavior and fast-forward-only maintenance
updates across release workflows.
* Made nightly mirroring/selection and retention pruning more selective
and safer.
* **Tests**
* Added Bats coverage for nightly mirroring and RC retagging, plus new
Helm/unit checks.
* **Documentation**
* Updated the release model to center RC promotion and tag immutability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Pin the remaining floating version tags in the build-main and ui-test
workflows to immutable commit SHAs, preserving the original tag in a
trailing comment so Renovate can keep the pins up to date.
Supply-chain hardening against tag hijacking/mutation; no functional
change — each SHA is the commit the tag currently resolves to.
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Review feedback (non-blocking):
- .pre-commit-config.yaml pinned zizmor at v1.22.0 while the CI gate installs
1.26.1; the zizmorcore/zizmor-pre-commit rev is the zizmor version 1:1, so the
gap meant local and CI runs could differ. Align the hook to v1.26.1.
- ui-test.yaml's checkout did not set persist-credentials: false, unlike the
other two workflows hardened here (build-main.yaml, zizmor.yml). Set it so the
token is not left in .git/config for the later pnpm install step to read.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build-main.yaml and ui-test.yaml (the latter added by the in-tree console
move) landed without following the repo's workflow conventions: no top-level
permissions block and actions referenced by moving tags. Together they zero out
two OpenSSF Scorecard checks the rest of the tree already satisfies:
- Token-Permissions: scored on the weakest workflow, so one workflow with an
undeclared (default-write) top-level token drops the whole check to 0.
- Pinned-Dependencies: the six tag-pinned action refs count as unpinned.
Fix both files (top-level read-only token; job-level packages:write kept where
needed; all action refs pinned to the SHA already used elsewhere in the tree)
and add a durable gate so it cannot recur:
- .github/zizmor.yml enables the two Scorecard-relevant audits (unpinned-uses,
excessive-permissions), clean tree-wide as of this commit. The remaining
audits carry pre-existing debt and are disabled with a TODO to burn down and
re-enable one at a time.
- .pre-commit-config.yaml runs zizmor (offline) on changed workflows.
- .github/workflows/zizmor.yml runs the same audit tree-wide on every PR that
touches a workflow, failing on any reintroduced regression.
Also wire scorecard.yml to an optional SCORECARD_TOKEN secret: the default
GITHUB_TOKEN cannot read branch-protection settings, so the Branch-Protection
check is scored on partial data. The token is a no-op until the secret is added.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review feedback from lexfrei on .github/workflows/promote-rc.yaml:200.
The "Prepare stable branch" step built the release-X.Y.Z branch with
`git checkout -B` + `git commit -s` (a fresh, timestamp-dependent SHA every run)
and pushed it with a plain non-force push. On a re-dispatch — retrying after a
later step failed, or promoting a newer rc to the same version — the new commit
is a sibling of the already-pushed one (same parent, different SHA), so the push
is rejected non-fast-forward and promotion wedges until the branch is deleted by
hand. That contradicts both the validate step (which tolerates a leftover draft
precisely so a newer rc can be re-promoted) and the workflow header's
"overwritable branch, nothing to wedge a later re-promotion".
Reuse the compare-before-force pattern from tags.yaml's Create release branch
step: skip when the remote already matches, force-push + log when the branch
genuinely moves. It is a mutable staging branch, so overwriting it is correct.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
- nightly.yaml: pass the registry tokens to `skopeo login` over stdin
(--password-stdin) instead of `-p` on argv, matching promote-rc/finalize and
keeping the token out of the process list.
- dashboard configmap: require digits-then-dot (^v?[0-9]+\.) when deriving the
displayed platform version from the console image tag. The old ^v?[0-9] let a
tagless, port-carrying ref (ghcr.io:443/.../console, whose last ":"-segment is
"443/.../console") pass as a version. Verified: v1.5.0 / 1.5.0 still surface;
the port ref and :latest no longer do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The tag workflow assumed the old hand-pushed-stable flow in several places that
break (or misbehave) under rc-promotion:
- Drop PUBLISH_FLOATING from the Build step. This job now only ever builds a
prerelease (a stable tag skips via its pre-existing draft, or is rejected —
below), so moving :latest here would track the newest rc, and an rc on an old
line would drag :latest backwards. :latest belongs to promotion (finalize).
- Publish rc releases. tags.yaml created the rc release as a draft and nothing
ever published it, so the community could not see the rc page or download its
nocloud-amd64.raw.xz — defeating the rc's stated purpose. Publish it
(prerelease:true, make_latest:false) once assets are uploaded; rc tags are
write-once, nothing to protect with draft status.
- Reject hand-pushed stable tags. A stable vX.Y.Z with no pre-existing draft is
a hand-push down the legacy full-rebuild path (full rebuild, moved :latest,
rogue release PR). Fail fast with a message pointing at promote-rc.yaml. This
replaces two now-unreachable steps that shared the same gate: the submodule
Go-module tag (moved to finalize in the previous commit) and "Create pull
request if not exists" (promote-rc owns PR creation).
- Gate generate-changelog and update-website-docs on stable (ref_name has no
'-'), so each rc no longer opens a changelog-vX.Y.Z-rc.N PR that never merges
and an "update managed apps for vX.Y.Z-rc.N" website PR.
- Paginate the release lookups (check_release, draft-reuse): a promote-created
draft can sit past the default first 30 and be missed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Promotion mutated the registry at dispatch time — retagging ~37 images to
vX.Y.Z (+:latest), publishing the stable cozy-installer chart — before the
promote PR's e2e ran and before any merge. An abandoned promotion then left
stable-named bytes for a version that never shipped, and the (correct) image
write-once check wedged any later re-promotion to the same version.
Move every irreversible registry side effect into pull-requests-release.yaml
finalize, which runs only after the promote PR merged (= after its e2e passed),
making promotion atomic. promote-rc.yaml now only stages the release-X.Y.Z
branch, drafts the stable release, and opens the PR; an abandoned promotion
leaves nothing but an overwritable draft + branch.
Folded into the same restructure:
- Cut the api/apps/v1alpha1/vX.Y.Z Go-module tag in finalize, at the stable
tag's commit. It was gated in tags.yaml on `release_exists==false &&
is_stable`, unsatisfiable under promotion (the draft always pre-exists), so
promoted stables never got their submodule tag — a regression for Go
consumers. Finalize is where the stable tag is born, so it cuts the matching
submodule tag write-once (stable only).
- Gate :latest by max-semver. promote-retag.sh now moves :latest only when
MOVE_LATEST=1 (default off); finalize passes it the same make_latest decision
the release gets, so a patch on an older line moves neither the release's
`latest` nor the images' :latest. Added MOVE_LATEST bats coverage.
- Stamp platformVersion into the published stable chart's default values
(uncommitted, at package time) so `helm --install --version X.Y.Z` reports the
stable version in telemetry instead of the rc version baked into the operator
image. Not committed to the branch — that would leak platformVersion=vX.Y.Z
onto main and mis-stamp the next rc.
- promote-rc: serialize all promotions (static concurrency group, not per-rc);
label the PR full-e2e so the full suite runs; scope the rc->stable sed to the
depth-2 values.yaml glob plus the kubernetes .tag files (the build's own write
set); tolerate a leftover draft so re-promotion is not wedged.
- finalize: assert the PR author is cozystack-ci[bot], and run on an ephemeral
oracle-vm runner (needs skopeo/helm).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The first-ever `oras push` of cozystack-nocloud creates the GHCR package,
but without org.opencontainers.image.source it is created unlinked to the
repo, and the follow-up `oras tag … nightly` can 403 permission_denied:
write_package until someone links it by hand. Set the annotation on push so
the package is linked from the start (matches the --label the image builds
already carry).
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Addresses two blockers that stop a stable release from being cut by the
promote flow (both reproduced against a real registry in review).
B1 — wrong registry: the workflow forced env.REGISTRY to the OCIR CI
registry and logged in only to OCIR, but release-tag images (rc and
stable) are built and pushed to GHCR by tags.yaml (hack/common-envs.mk
default). promote-retag.sh filters refs by the $REGISTRY/ prefix, so with
REGISTRY=OCIR every GHCR ref was dropped, the selection went empty, and
the retag step exited 1 before doing anything. Point the job at
ghcr.io/cozystack/cozystack (matching the script default) and log in to
GHCR via --password-stdin (also resolves the token-on-argv nit).
B2 — missing chart: promote-retag.sh only retags image refs found in
values.yaml, and the cozy-installer OCI chart is a separate artifact in no
values.yaml. make build (which pushes the chart) is gated on a
not-yet-existing release, so a promoted stable never published
cozy-installer:X.Y.Z — the documented `helm --install … --version X.Y.Z`
404'd with manifest-unknown. Repackage the chart from the restamped stable
tree (no rebuild — it references the retagged images + packages artifact by
digest) and helm push it, mirroring the nightly flow. Adds helm to the job.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The PR Auto-Label workflow triggered on pull_request_target 'edited',
which fires on every PR title *and body* edit. Long-lived bot PRs such
as Renovate repeatedly edit their description on schedule, causing the
label job to re-run dozens of times against an unchanged commit (e.g.
~48 runs on #2998, all on the same head SHA).
Drop 'edited' so the job only runs on opened/reopened/synchronize.
Labels derived from the PR title are re-applied on the next push or on
reopen. Title-only edits between pushes no longer trigger a relabel,
which is an acceptable trade for eliminating the churn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Two issues flagged in review on promote-rc.yaml:
- The "Validate rc is promotable" github-script called core.setFailed()
but kept running: a missing rc release then hit `rcRel.assets` and a
missing disk asset hit `disk.id`, throwing a TypeError that masked the
intended failure message. Return immediately after each setFailed so the
clean diagnostic is what surfaces.
- "Create draft stable release and upload assets" ran `make upload_assets`
without COZYSTACK_VERSION. upload_assets depends on `manifests`, which
re-renders the operator manifests; with no version it blanks the
platformVersion, silently undoing the stable stamp the Restamp step just
applied. Pass COZYSTACK_VERSION=${STABLE_VERSION} so the re-render
re-stamps the stable version.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/tags.yaml: the
single `skip` output meant 'a release already exists' but was reused to gate
update-website-docs, so a promoted stable tag (whose draft promote-rc already
created) skipped the website-docs job, not just the rebuild. Rename the output
to release_exists to reflect its real meaning and drop it from the
update-website-docs gate so a promotion still publishes docs (the rebuild steps
inside prepare-release stay gated on it).
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on
.github/workflows/pull-requests-release.yaml: the fast-forward catch block
treated every updateRef failure as a recoverable warning, so an auth, API, or
malformed-ref error would silently leave the maintenance branch stale. Warn
only on the 422 non-fast-forward rejection and fail the job otherwise.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/promote-rc.yaml:
the rc staging checkout left the app token in .git/config even though the
Prepare stable branch step re-injects it via 'git remote set-url'. Set
persist-credentials: false to drop the redundant exposure.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/promote-rc.yaml:
the validate step only checked for an existing stable release, so a stray
refs/tags/vX.Y.Z (partial run or manual push) would slip through and fail later
in finalize — after promote-retag had already retagged images. Also fail early
when the stable git tag already exists.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/promote-rc.yaml:
actions/create-github-app-token was used at the floating @v1 and without a
repositories restriction, so the minted token carried owner-wide access. Pin it
to the SHA the rest of the repo uses and scope it to this repository.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/nightly.yaml:
actions/checkout writes the token into .git/config by default, and these jobs
copy the checkout into /tmp/$SANDBOX_NAME before running repo targets. Set
persist-credentials: false on all three source checkouts so the token is not
carried into the sandbox copies.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/nightly.yaml:
the version was 0.0.0-nightly.<date>, so a same-day re-dispatch or re-run
republished every image, chart and artifact under the same tag — the dated
nightly was not write-once. Append the run id and attempt so each build gets a
distinct, immutable tag, and widen retention.yaml's selector regex to match the
extra numeric components (still excluding the floating :nightly and any release
tag).
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/retention.yaml:
gh api --paginate with --jq runs the filter once per page and emits one JSON
array per page, so total/drop_ids miscount once a package's versions span more
than one page. Pipe the raw pages through 'jq -s add' to merge them into a
single array before counting and slicing.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/nightly.yaml:
the resolve step read cozystack-packages:main and the pull step re-fetched
the mutable :main tag, so a concurrent build-main push could check out commit
A while mirroring tree B. Resolve the :main digest once, derive the revision
sha from it, and pull the baked tree by that same digest.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Match the repo-wide SHA pin main applies to every github-script use, so
the new promote-rc workflow doesn't regress the Scorecard pinned-dependencies
check.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Nightlies are now GHCR OCI tags (0.0.0-nightly.<date>), not git tags or
GitHub releases, so:
- retention.yaml now prunes old GHCR container-package versions via the
Packages API instead of git tags/releases. Scoped to versions whose
tags are ALL 0.0.0-nightly.<YYYYMMDD>, so the floating :nightly tag,
release tags (vX.Y.Z, latest, rc, main) and untagged versions are
never touched. Keeps newest 14 per package; dry-run by default.
- nightly-rc.yaml is removed: it cut nightly *git tags* that drove a
release/e2e path we no longer use. The nightly.yaml workflow replaces
it with the mirror + full-e2e flow.
actionlint clean.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Scheduled nightly that publishes an installable nightly of main without
rebuilding it, then runs the full e2e suite against the published bits:
- mirror: resolves the exact commit cozystack-packages:main was built
from, runs nightly-mirror.sh to copy every cozystack-owned component
image OCIR->GHCR by digest, re-publishes the rewritten packages
artifact and the cozy-installer chart to GHCR (0.0.0-nightly.<date>
+ floating :nightly).
- build-disk: assembles the Talos nocloud disk from the upstream imager
(no cozystack-image dependency, no rebuild) and publishes it as a GHCR
OCI artifact (cozystack-nocloud) via oras, so a nightly is installable
on real hardware, not just e2e.
- e2e: stages the GHCR install closure (rewritten tree + chart pinned to
the GHCR packages artifact + the disk) and runs the FULL app suite (no
Test Impact Analysis - a nightly has no diff to scope against).
No GitHub release and no api/apps/v1alpha1 Go-module tag are created.
Gated by the NIGHTLY_ENABLED repo variable. actionlint clean.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The rc->stable substring rewrite ran `sed s/${RC_VERSION}/.../` with the dots
unescaped, so the rc version matched as a regex. Escape regex metacharacters so
the match is literal. Use `git checkout -B` for the stable staging branch so a
re-run resets the local branch instead of failing on "already exists".
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
nightly-rc.yaml's `release-[0-9]*.[0-9]*` glob also matched the per-tag staging
branches tags.yaml creates (release-1.4.0, release-1.4.0-rc.2,
release-1.5.0-nightly.<date>), cutting spurious nightly tags. Anchor the target
selection with a `^release-[0-9]+\.[0-9]+$` regex.
tags.yaml now exposes an is_stable output and gates the write-once
api/apps/v1alpha1 Go-module tag and the auto-opened release PR on it, so an
rc/beta/alpha (is_stable=false) or a nightly (is_stable unset) never pollutes the
module proxy or opens a release PR (#2677 goal 3).
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
- nightly-rc.yaml: daily cron that cuts a write-once
vX.Y.Z-nightly.<date> tag for each tracked target (main + active
release-X.Y branches) when its HEAD moved since the last nightly. The tag
drives the normal rc build/e2e path. Gated by the NIGHTLY_RC_ENABLED repo
variable so it can merge dark and be switched on deliberately.
- retention.yaml: daily cron that keeps the newest 14 nightlies per line and
deletes older nightly tags + their releases. Delete-only, scoped strictly
to the *-nightly.* namespace (rc/stable untouched). Dry-run by default;
flip the `apply` input or the RETENTION_APPLY repo variable to delete.
Registry image pruning is left as a tracked TODO (no OCIR delete parity).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Add the rc->stable promotion path. A stable release is a renamed
release-candidate: the bytes shipped as vX.Y.Z are bit-for-bit the bytes
built and e2e-tested as vX.Y.Z-rc.N.
- hack/promote-retag.sh: reads the digest-pinned image refs the rc baked
into packages/*/*/values.yaml (single-string, split repository/tag/digest,
and the cozystack-packages OCI artifact) and `skopeo copy`s each — by
digest — to the stable tag and :latest. Copying by digest guarantees the
stable image is identical to the rc; the result is verified with
`skopeo inspect`. Validated offline with `sh -n` and the yq extraction
against the current tree; the skopeo path is CI-validated only.
- promote-rc.yaml: workflow_dispatch(rc_tag). Validates the rc exists and
the stable tag does not, retags via the script, rewrites the cosmetic
-rc.N substring in the vendored tags to the stable version (the @sha256
wins regardless), restamps the version-stamped assets (operator manifests,
cozypkg, openapi; the heavy Talos assets are copied from the rc draft),
and opens a release-X.Y.Z PR. Merging it reuses the existing release-PR
e2e + pull-requests-release.yaml finalize to cut the write-once stable tag.
Squash is disallowed on the PR (the stable tag needs a real merge commit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replace the force-retag operations in the release flow with write-once /
compare-before-force guards, so published tags are never moved (the cache
poisoning on api/apps/v1alpha1 via the Go module proxy, plus SBOM/scanner
drift, all stem from moved tags).
- tags.yaml: the api/apps/v1alpha1 submodule tag is now write-once (create
if absent, no-op if unchanged, fail loudly if it would move). The
release-X.Y.Z staging branch uses a compare-before-force guard (skip when
unchanged; force+log only when the staging branch genuinely moves — it is
a mutable staging ref, not an artifact). The build is now skipped when a
release (draft OR published) already exists, so a stable tag prepared by
promotion is never rebuilt here.
- pull-requests-release.yaml: the stable tag is created at the PR merge
commit with a write-once guard instead of `git tag -f`/`git push -f` —
the merge commit did not exist before the PR opened, so force is
impossible by construction. The maintenance branch is fast-forward-only
(no `updateRef force:true`); a non-FF update warns and is left for a
maintainer.
- auto-release.yaml: removed. Stable patch tags were auto-bumped nightly via
delete-recreate; stable now appears only through an explicit rc promotion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The API Review Gate builds cmd/api-gate from the PR's merge-base checkout
so a PR cannot neuter the detector by editing its own code. When the merge
base predates the commit that introduced the gate, cmd/api-gate is absent
from both the merge base and the equally old head, so the head fallback
also fails to build and the required check errors out before evaluating
any API content. This blocks every open PR whose merge base is older than
the gate, and neither a re-run nor an approving review clears it.
Add the base branch tip as an intermediate build source: prefer the
merge-base checkout, fall back to the base tip when the merge base lacks
the tool, and bootstrap from head only while the tool exists nowhere on
the base branch yet. The base tip is a protected branch, not PR-authored,
so it preserves the "analyze head as data, not code" property. The API
surface is still diffed against the merge base; only where the detector
binary is compiled changes. This unblocks the backlog without forcing
every affected PR to rebase.
The selection stays inline in the workflow rather than in a helper script:
a helper would be read from the untrusted PR head, which both reintroduces
the failure for heads that predate the helper and lets a PR steer the
selection away from the trusted build.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
The Automatic Backport workflow passed `conflict_resolution:
draft_commit_conflicts` as a **top-level** input to
`korthout/backport-action`, but the action only reads
`conflict_resolution` from inside its `experimental` JSON input. Passed
top-level it was silently ignored — the action run logs warn `Unexpected
input(s) 'conflict_resolution'` — so it fell back to the `experimental`
default of `fail`. On a cherry-pick conflict the action then created
**no** pull request and only left a "Backport failed" comment on the
source PR (this bit #3121 and #3119).
This moves the key into the `experimental` input, so a conflicting
backport instead opens a **draft PR with the first conflict committed**
for maintainers to resolve in place.
No version bump: the pinned `v3.2.1` already supports
`draft_commit_conflicts` — the bug was purely the input placement.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Backport pull requests now handle conflicts more reliably and open as
draft PRs when conflicts are detected.
* Improved configuration handling so the expected conflict behavior is
applied consistently during backport automation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Scope the gate's own files (cmd/api-gate, internal/apigate, the workflow) to the API owners in CODEOWNERS, and compile the detector from the trusted base checkout when present (head fallback only during bootstrap), so a PR cannot neuter the gate by editing its own detector code.
Detect two more breaking vectors: removing nullable:true (rejects null) and adding a oneOf/allOf/not composition constraint. Both covered by regression tests.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Fix schema-diff correctness: guard nil head on removed child schemas, flag adding a type to a previously-unconstrained field, stop reporting a head becoming unconstrained as narrowing, detect additionalProperties:false, and only recurse items/additionalProperties when both sides carry a schema.
Gate resource and group removal as new categories. Filter CRD versions to served ones so the deprecation flow surfaces as a removed version, and flag added x-kubernetes-validations (CEL) rules. Discover first-party cozystack.io CRDs by walking crds/ and definitions/ dirs so a relocated CRD (e.g. cozystack-scheduler) is covered, and error on an empty snapshot. Surface unmapped apiserver storage prefixes and allow hyphens in plurals; drop Markdown from text-mode reports.
Workflow: diff against the true merge base, count only each reviewer's latest review on the current head commit, and scope the token to the approval step with persist-credentials:false.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Add a deterministic api-gate tool (cmd/api-gate, internal/apigate) and a workflow that require an approving review from an API owner (lllamnyp or kvaps) when a PR introduces a new API group, a new resource, or a breaking schema change to an existing resource.
Detection diffs the checked-in OpenAPIv3 schemas (apps cozyrds + CRD manifests) and the apiserver storage registrations; no network or LLM calls.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
flux pull artifact --output requires the target directory to already
exist; it stats the path before any network call and aborts with
"invalid output path: stat _out/mainpkgs: no such file or directory"
when the dir is missing. The step deleted _out/mainpkgs with rm -rf but
never recreated it, so flux failed locally on every run and the ||
fallback misreported the failure as "artifact not published" — leaving
the current-main overlay a permanent no-op even though the artifact
exists and is anonymously pullable.
Add mkdir -p after the rm -rf so the directory exists before flux pull,
and reword the fallback message so a genuine future pull failure is no
longer mistaken for an absent artifact.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The current-main overlay skipped only rebuilt units (plus talos/installer),
so for a package the PR edited but did not rebuild — an upstream image bump
in a non-build-unit package such as keycloak or cert-manager — the committed
ref differed from the main artifact on image-reference lines only, the drift
check passed, and the overlay reverted the PR's edit back to current-main.
e2e and the installer artifact then ran against the stale image.
Pass the PR's edited package dirs (derived from the plan job's git diff) into
the overlay skip set so a PR's own image-ref edits win over the artifact. Add
tests pinning the preserve behaviour and covering the --image= arg form and
the file-absent-in-PR-tree branch.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Conditional PR builds (the per-package matrix in pull-requests.yaml) only
rebuild packages whose dir changed. Every other package falls back to the image
refs committed in the tree — the last release's digests (currently v1.5.0),
which go stale as main advances. So e2e and the installer's packages artifact
ran current-main configs against last-release images for everything outside the
PR's build matrix, and a PR installed via the cozy-installer chart was an
incoherent mix. This is the systemic gap in issue #3143 (its concrete bite: the
merged objectstorage-controller requeue fix#3034 never ran in e2e, flaking the
harbor BucketClaim check).
In finalize, pull the cozystack-packages:main OCI artifact that build-main
already publishes (the whole current-main packages tree, refs digest-pinned to
the images build-main pushed) and walk every ref-bearing file in it
(values.yaml + images/*.tag), overlaying the repo's copy. This is driven by the
artifact, not the build matrix, so it covers the ENTIRE first-party tree —
apps/system/core AND extra/, library/ — and a merged fix to any first-party
image (e.g. the objectstorage-sidecar ref under extra/seaweedfs) takes effect.
For a file build-main did not rebuild the artifact copy is byte-identical and
is skipped. Skipped explicitly: units the PR itself rebuilt (their pr-<N>-<sha>
refs win), packages/core/talos and packages/core/installer (owned by their
dedicated build-talos / finalize-installer jobs), and vendored charts/ subtrees.
The copy is self-validating: a file is overlaid only when every differing line
is image-reference-bearing (a full @sha256: ref, or a split
image/repository/registry/tag/digest key, or a --…-image= arg); otherwise it
keeps its committed ref. A missing artifact is a no-op (prior behaviour) and the
step is best-effort so it never blocks a PR.
No build-main change is needed: cozystack-packages:main is already published by
`make build`. Runs before the installer build so the flux-pushed packages
artifact and the pr.patch e2e applies both reflect the overlay.
Result: a PR is tested and installed as current-main + its own changes, coherent
across config and image, instead of current-main configs running on last-release
images.
Covered by hack/overlay-main-images_test.bats (run by `make unit-tests` in the
checks job): single-line and split-form ref overlay, extra/* coverage,
skip-rebuilt, talos/installer skip, vendored-charts prune, drift
self-validation, and missing-artifact no-op.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The backport workflow passed conflict_resolution as a top-level input to
korthout/backport-action, but the action only reads it from inside the
experimental JSON object. Passed top-level it was silently ignored (the
run logs warn "Unexpected input(s) 'conflict_resolution'"), so the action
used the experimental default of "fail": on a cherry-pick conflict it
created no pull request and only posted a comment.
Move the key into the experimental input so a conflicting backport opens a
draft PR with the first conflict committed, giving maintainers something to
resolve in place instead of a dead-end comment.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What
Make Renovate PRs land with consistent, conventional labels and clear a
recurring Dependency Dashboard warning.
## Why
Renovate's `:semanticCommits` preset picks `fix(deps)` for stable
(`>=1.0`) production modules and `chore(deps)` for `0.x` modules. The
result:
- `fix(deps)` bumps were auto-labeled **`kind/bug`** by the PR
auto-labeler → routine dependency updates polluted the bug list.
- The auto-labeler has no `deps → area/*` mapping, so **every** Renovate
PR fell through to **`area/uncategorized`**.
- `kubevirt.io/csi-driver` is pinned to an untagged pseudo-version
(`v0.0.0-2026…`) with no release tags, producing a permanent `no-result`
lookup failure on the Dependency Dashboard.
## Changes
- **`.github/renovate.json`**
- Pin `semanticCommitType: chore` + `semanticCommitScope: deps` → every
dependency PR is now `chore(deps): …` (maps to `kind/cleanup`, no more
bogus `kind/bug`).
- Disable Renovate for `kubevirt.io/csi-driver` (commit-pinned,
unbumpable) to clear the dashboard lookup warning.
- **`.github/labels.yml`** — add `area/dependencies` (justified: 25+
dependency PRs queued, well over the 3-issue threshold for a new area).
- **`.github/workflows/pr-labeler.yaml`** — map scope `deps →
area/dependencies`.
## Result
Future Renovate PRs: `chore(deps): …` → `automated` + `kind/cleanup` +
`area/dependencies` + `do-not-merge/hold`.
## Notes
- `renovate.json` / `labels.yml` take effect once merged; the labeler
change applies to PRs opened or edited after merge.
- The `sigs.k8s.io/yaml` lookup warning on the dashboard is left
untouched — it's a normal module and reads as a transient datasource
hiccup, not a config issue.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a new `area/dependencies` label to consistently tag
dependency-related pull requests.
* Updated the PR auto-labeler to map commit/title `deps` scopes to
`area/dependencies`.
* Standardized Renovate dependency commit messages to use `chore(deps)`,
including special handling for Go module updates, and disabled Renovate
updates for the `kubevirt.io/csi-driver` Go module.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Renovate's :semanticCommits preset picks fix(deps) for stable (>=1.0)
production modules and chore(deps) for 0.x modules, so routine bumps
landed inconsistently — fix(deps) bumps were auto-labeled kind/bug,
polluting the bug list. Pin semanticCommitType to chore (and the scope
to deps) so every dependency PR is chore(deps): … and maps to
kind/cleanup.
The PR auto-labeler had no deps -> area/* mapping, so every Renovate PR
fell through to area/uncategorized. Add an area/dependencies label and
map the deps scope to it.
Also disable Renovate for kubevirt.io/csi-driver: it is pinned to an
untagged pseudo-version with no release tags, producing a permanent
no-result lookup failure on the Dependency Dashboard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The cozystack-ui SPA was vendored into the monorepo (#2963) and now lives under packages/system/dashboard. Map its likely PR title scopes to the existing area/dashboard so UI PRs no longer fall through to area/uncategorized.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Completes #2937. Replaces the single serial `Build` job — one ephemeral
VM running `make build` (~26 images one after another) that wedged on
the shared embedded buildkit under concurrent multi-PR load — with a
parallel, diff-scoped matrix.
**New job graph (regular PR path):**
```
plan changed files -> hack/build-matrix.sh -> scoped matrix + docs gate
checks make unit-tests + test-controllers (gates finalize -> e2e)
build one package per job, own docker-container buildx,
uploads a per-package digest patch fragment [parallel]
build-talos talos image + nocloud disk (its own leg; e2e always needs the disk)
finalize download+apply all fragments, then build the installer LAST
(its flux-push bundles the whole digest-patched tree),
emit the unified pr.patch
e2e consumes talos-image + pr.patch unchanged
```
**Why finalize is special:** `packages/core/installer`'s `image` target
runs `flux push artifact --path=packages`, bundling the *entire*
packages tree into the OCI artifact the operator pulls — so it must run
after every other package's digest edit is applied, never as a peer
matrix job. `talos` is likewise pulled out of the matrix (its image and
nocloud disk share heavy `_out/assets` and build once).
**Gating:** `finalize` (and therefore `e2e`, which needs it) depends on
`checks`, so a red unit/controller test skips the expensive install+e2e
path instead of running it for ~1h — the gating the old in-build-job
tests gave, without re-coupling the cheap parallel builds.
**Change-scoping:** `hack/build-matrix.sh` parses the root Makefile
`build:` target (single source of truth) and emits only the packages a
PR touches — most PRs build ~0–1 images — escalating to the full set on
a shared-dependency change (`cozy-lib`, the build macros, `go.mod`, the
build workflows). It fans `seaweedfs` out to `objectstorage-controller`
(cross-package sidecar-tag dependency). 16 bats cases cover the
selector.
**SBOM:** `SBOM=1` wires buildx `--sbom` through `common-envs.mk` (off
by default). It stays off in the matrix until an OCIR
referrers/attestation probe confirms the registry accepts attestations.
**Decommission:** the cache warmer (`build-main.yaml`) moves to the
ephemeral pool, so no build path depends on the dedicated self-hosted
runner. The runner remains referenced only by the e2e `debug` breakpoint
path; fully de-registering it is a separate infra step.
### Validation
- Local: `hack/build-matrix_test.bats` 16/16; `actionlint` clean (custom
runner labels registered in `.github/actionlint.yaml`); `make -n`
confirms `--sbom` is off by default and on with `SBOM=1`.
- CI: this PR edits `pull-requests.yaml`, so `build-matrix.sh` forces a
full matrix — the PR exercises the new pipeline on itself. First run was
green through `plan → 25 parallel builds → build-talos → finalize →
install → 21/22 app tests`; the only red was the pre-existing
`bucket`/BucketAccess flake (install and the other 21 suites passed on
the matrix-built tree), which validates the decomposition end-to-end.
### Follow-ups (tracked on #2937)
- Probe OCIR referrers/attestation support; enable `SBOM=1` in the
matrix if supported.
- Infra: de-register the self-hosted build runner (the e2e `debug`
breakpoint path is its only remaining user).
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Refined CI to a staged, artifact-based pipeline with gated checks,
per-package parallel builds, and centralized patch assembly before
building final images
* Improved ephemeral runner performance and reliability (runner
selection, longer timeout, build toolchain setup)
* Added build-matrix auto-selection with targeted fan-out rules and
“full rebuild” triggers
* Updated CI lint configuration to recognize ephemeral runner labels
* Introduced an optional SBOM toggle for build artifacts
* **Documentation**
* Updated e2e testing docs to better explain Test-Impact Analysis scope
and gating semantics
* **Tests**
* Added Bats tests validating build-matrix selection and edge cases
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Cleans up four GitHub Actions workflows that are redundant, broken, or
noisier than they need to be. Each change is its own commit so they can
be reviewed (or reverted) independently.
- **`retest.yaml` — removed.** The `/retest` command re-ran a job named
`Prepare environment`, but the TIA / job-split refactor of
`pull-requests.yaml` collapsed that into a *step* inside the `E2E Tests`
job, so the job lookup can never match. Every real `/retest` invocation
since fails with `Could not find "Prepare environment" job` — **10
failed runs spanning 2026-01-08 → 2026-06-24, none that ever completed a
retest**. (The 41 runs GitHub records as `success` are `issue_comment`
events where the job-level `if`-guard was false, so no steps ran.) `gh
run rerun --failed` covers the need.
- **`codeql.yml` — switch Go to manual build; keep the per-PR run.** The
Go `autobuild` ran `make`, whose included `hack/common-envs.mk` does a
`$(shell git fetch upstream --tags)` at parse time — a network call to a
non-origin remote that has nothing to do with the analysis and is a
flake vector. Replace it with `build-mode: manual` that compiles each
first-party module explicitly (the root module, `api/apps/v1alpha1`, and
the three nested image modules; the gitignored `_repos/*` upstream
clones drop out for free on a clean checkout). Same coverage, no
upstream fetch. The `pull_request` trigger is **kept on purpose**:
per-PR analysis is the only mode that diffs *new* alerts against the
base branch and annotates them on the changed lines — the basis for
promoting CodeQL to a required pre-merge check (follow-up: add `CodeQL /
Analyze (go)` to branch protection and set the code-scanning failure
threshold to errors-only). No `paths` filter — a required check that is
path-skipped never reports and would block PRs that touch no Go.
- **`scorecard.yml` — badge-only; stop polluting code-scanning.**
Scorecard uploads one code-scanning alert per check, which floods
Security → Code scanning and buries CodeQL's first-party findings. Drop
the `upload-sarif` step and the now-unneeded `security-events: write`;
keep `publish_results` (the public scorecard.dev badge is the canonical
view). Also drop the redundant `push:[main]` trigger — the weekly cron +
`branch_protection_rule` already cover it.
- **`update-releasenotes.yaml` — path-gate to changelog changes + manual
hatch.** The release body is synced from `docs/changelogs/v*.md`, which
only lands on main when the changelog PR merges or a maintainer edits an
existing changelog. Gate the `push` trigger to `docs/changelogs/v*.md`
(the `v*` glob skips `template.md` / `patch-template.md`) so the
self-hosted job stops firing on every unrelated main push, and add
`workflow_dispatch` as a manual re-sync hatch.
- **`SECURITY.md` — document the security scanners.** Add an "Automated
security analysis" subsection describing the CodeQL (SAST) and OpenSSF
Scorecard (supply-chain) setup, including the manual Go build, the
new-alert-on-diff behaviour, the intent to make CodeQL a required
pre-merge check, and how findings are dismissed — so the posture is
documented, not just encoded in YAML.
Not touched: `release-e2e.yaml` — already removed by #2978.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated automation workflows for security checks and release-note
generation.
* Removed an unused retry workflow and tightened when some checks run.
* **Documentation**
* Expanded security guidance to better explain automated analysis, alert
handling, and reporting.
* Clarified when release notes updates are generated.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolve the .github/workflows/pull-requests.yaml conflict: keep this branch's
per-package build matrix and carry main's `git diff --binary` patch fix
(#e566992) onto every fragment and pr.patch capture, so provider bundles that
package image targets regenerate (components*.gz) survive fragment reassembly
in finalize and reach the patch e2e applies.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
> **Supersedes #2610.** Same work, rebased cleanly onto current `main`
so it now sits on top of #2867 (the `kube-controller-manager` VAP
type-checker fix). That fix is what was killing the base install in
#2610's E2E *before* the `kubernetes` app test ever ran, leaving the
Talos worker-bootstrap rewrite verified only by code reading. #2610's
67-commit branch is squashed here into 8 logical commits with original
authorship preserved (@kvaps and @IvanHunters as authors); the tree is
**byte-identical** to #2610's head — no code changes, just the rebase
onto `main` (to pick up #2867) plus history cleanup. The detailed review
history lives on #2610.
Phase 1 of the Kubernetes-app split design (see cozystack/community#8):
replace the Ubuntu+kubeadm worker bootstrap path with Talos, driven by
`cluster-api-bootstrap-provider-talos` (CABPT) and a
`clastix/talos-csr-signer` sidecar embedded in the Kamaji control-plane
pod. Existing tenants keep working — old machines roll out and get
replaced by Talos workers without manual intervention.
Highlights:
- Add CABPT (v0.6.12) as a second `BootstrapProvider` alongside the
existing kubeadm one in `capi-providers-bootstrap`.
- Bump the Kamaji control-plane provider with an upstream-bound patch
that exposes `KamajiControlPlane.spec.network.additionalServicePorts`,
used to publish trustd (50001/TCP) on the apiserver service.
- Generate Talos PKI (Ed25519 CA + trustd TLS) via cert-manager and
stable random Talos secrets (`token`, `clusterId`, `clusterSecret`,
`bootstrapToken`) using the helm lookup-and-reuse pattern.
- Materialise a kubeadm-format `bootstrap-token-<id>` Secret inside the
tenant `kube-system` via a Helm post-install/upgrade Job.
- Render a `TalosConfigTemplate` (worker machineconfig, `generateType:
none`) and switch the `MachineDeployment.bootstrap` reference from
`KubeadmConfigTemplate` to `TalosConfigTemplate`, gated on Talos secrets
and the Kamaji apiserver service being ready.
- Boot workers from the Talos openstack image via a CDI `DataVolume`
(`source.http.url` → `factory.talos.dev`), expose the system disk as
virtio-blk with `blockSize.custom: logical=512, physical=4096` so
4Ki-native block storage backends (LINSTOR/DRBD) play nicely with QEMU's
O_DIRECT writes and SeaBIOS still boots, drop the separate kubelet disk
(Talos lays out EPHEMERAL itself).
- Pin pod/service CIDRs in the worker machineconfig to the tenant ranges
(`10.243.0.0/16` / `10.95.0.0/16`) to avoid Talos's address-overlap
diagnostic against the host pod CIDR.
- Set `cluster.controlPlane.endpoint` and `cilium.k8sServiceHost` to the
Kamaji apiserver Service ClusterIP (looked up at render time), so
bootstrap survives the chicken-and-egg moment where tenant DNS does not
exist yet and the host CoreDNS does not serve the tenant zone.
End-to-end on a dev cluster: a fresh tenant brings up a Talos worker,
kubelet CSRs get approved (apiserver-client by Kamaji, kubelet-serving
by the talos-csr-signer sidecar), Cilium initialises with hostNetwork
against the apiserver ClusterIP, CoreDNS comes up, and the node
transitions to `Ready`.
### Release note
```release-note
feat(kubernetes): bootstrap tenant workers with Talos Linux instead of Ubuntu+kubeadm. Existing clusters roll over to Talos workers automatically; new tenants come up on Talos from the start. Powered by cluster-api-bootstrap-provider-talos and the talos-csr-signer sidecar in the Kamaji control-plane pod. Note: the tenant `kubernetes` HelmRelease now reports Ready as soon as Helm completes (Install/Upgrade DisableWait) and no longer blocks on worker/addon readiness — worker-rollout health is tracked by WorkloadMonitor, not the HelmRelease.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
**New Features**
* Added Talos worker configuration for `talos.version` and
`talos.schematicID`
* Introduced `nodeHealthCheck` (`maxUnhealthy`, `nodeStartupTimeout`)
for worker remediation tuning
* Added image overrides for `kubectl` and `talosCsrSigner`
* Exposed additional tenant control-plane Service ports support
**Breaking Changes**
* Removed Kubernetes v1.30; supported versions start at v1.31
* Renamed `nodeGroups.ephemeralStorage` → `nodeGroups.diskSize` with
consolidated Talos system-disk semantics
**Removals**
* Removed Ubuntu container-disk images and related containerd patch
mechanisms
* Air-gapped tenant workers temporarily unsupported during the Talos
transition
**Documentation**
* Updated chart docs, parameters, and GitOps upgrade guidance for
version bumps
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Moves the `cozystack-ui` console SPA into this monorepo so the UI
versions and ships in lockstep with the platform — removing the
cross-repo digest-bump PRs and the version drift between the two
repositories.
The source is vendored at `packages/system/dashboard/images/console/`
via `git subtree`, preserving full upstream history. This mirrors the
existing in-tree `images/token-proxy/` pattern, and `.helmignore`
already excludes `images/*/*` so the source never enters the Helm chart.
**Changes**
- **Source**: `cozystack-ui` `main` merged under
`packages/system/dashboard/images/console/` (history preserved as a
merge parent).
- **Build**: `packages/system/dashboard/Makefile` builds the console
image from the in-tree context instead of cloning the external repo at
build time. Dropped `clone-console`/`clean-console` and the
`CONSOLE_REPO`/`CONSOLE_BRANCH` variables. The built digest is still
pinned into `values.yaml` by `update-values-console` — now in the same
commit as the source change rather than a separate follow-up PR.
- **CI**: new `.github/workflows/ui-test.yaml` runs pnpm typecheck +
unit tests, path-gated to `images/console/**`. The container image
continues to build via `make build`.
**Verification**
- `pnpm install --frozen-lockfile` → `pnpm typecheck` clean → production
build produces `dist/`.
- 313 unit tests pass.
- `helm template` renders; console + token-proxy image references
unchanged.
**Follow-ups (after merge)**
- Archive the standalone `cozystack-ui` repository and redirect
contributions here.
- Update remaining documentation pointers that reference the old repo
path.
### Screenshots
N/A — no UI behavior change. This relocates the UI source and rewires
the build; the shipped image and digest are unchanged.
### Release note
```release-note
build(dashboard): vendor the cozystack-ui console into the monorepo and build it from in-tree source; the shipped image and digest are unchanged for cluster operators
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The /retest command re-ran a job named 'Prepare environment', but the
TIA/job-split refactor of pull-requests.yaml collapsed that into a step
inside the E2E Tests job, so the job lookup can never match. Every real
/retest invocation since fails with 'Could not find Prepare environment job'
— 10 failed runs spanning 2026-01-08 to 2026-06-24, none that ever completed
a retest. (The 41 runs GitHub records as 'success' are issue_comment events
where the job-level if-guard was false, so no steps ran.) `gh run rerun
--failed` covers the need.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Scorecard posts one code-scanning alert per check, which floods Security >
Code scanning and buries CodeQL's first-party findings. The public
scorecard.dev badge (publish_results) is the canonical view, so drop the
upload-sarif step and the now-unneeded security-events: write. Also drop the
push:[main] trigger — the weekly cron and branch_protection_rule already
cover it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
