The app installation token lists package versions but its DELETE of a
listed version is answered with 404, while the repository's
GITHUB_TOKEN with packages: write deleted the same version without any
change to package settings. Deletes therefore go through GITHUB_TOKEN;
listing keeps the app token.
That deletion was shown on a package linked to this repository. A
package linked elsewhere is not attempted: it is skipped before its
versions are read and named in a warning, so it cannot turn every
apply run red. A failed read of the link still fails the run, and so
does a sweep in which no resolved package is linked, since that sweep
prunes nothing.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A refused DELETE only logged a warning, so a token without delete
rights left an apply run green with nothing removed. Move the delete
into hack/retention-delete.sh, which reports each refusal as an error
carrying the API status, still tries the remaining versions, and exits
non-zero; the workflow turns that into a failed run after the sweep.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The org container-package listing answers 400 to the app installation
token, so the job has never reached its sweep. Take the names from the
published nightly tree via hack/retention-packages.sh and keep calling
only the per-package versions endpoints, which accept that token.
An unresolved name list now fails the run: the nightly publishes into
these packages daily, so an empty list cannot mean there is nothing to
prune. It skips only the nightly sweep. The promotion sweep reads one
fixed package and is kept safe by the protected set, which the name
list does not feed, so it still runs.
A versions read that fails, in either sweep, is reported with the API
error and fails the run once the rest is swept, instead of aborting
the step through set -e. A failed candidate read leaves no ids behind.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The nightly sweep's own listing shared one `|| true` with the grep
that filters it, and grep legitimately exits 1 when no cozystack/*
package matches. A failed /orgs/.../packages read therefore arrived
as an empty list, took the "No cozystack/* container packages found"
branch and exited 0 -- below the nightly sweep but ABOVE the deferred
guard, so a protection set that could not be resolved reported
success having swept nothing. That bypasses the guard rather than
varying it: replaying the step against a stub gh, a failed /pulls
combined with a legitimately empty org listing exited 0 after
printing its own ::error::.
List and filter as two statements so one `|| true` stops meaning two
things, handle the listing failure like the other two (::error::,
blocked flag, emptied list), and consult the flag before the early
exit. That exit is non-zero, matching the other two listing
failures: no sweep is left to carry the run down to the guard, so
this is the last place the flag can be honoured, and an unresolved
protection set is exactly what fail-closed means. The legitimate
empty-org case still prints the same line and exits 0.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Both candidate-protection sources open with a `gh api --paginate`
listing and neither handled a failure. Under `set -euo pipefail` a
transient /pulls or /branches blip therefore exited the step before
the unrelated nightly GHCR sweep had run -- the exact coupling the
deferred guard at the bottom exists to remove, and which
hack/promotion-retention-contract.bats already asserts positionally.
Route both listings into that guard instead: raise the blocked flag
and empty the list, so the promotion sweep is skipped and the job
still fails while the nightly sweep finishes. Emptying is not
cosmetic -- a half-paginated list left in place is a smaller
protected set, and a smaller protected set is the one shape that can
authorise a delete.
Replaying the step's real shell against a stub gh: with /pulls
failing it exited 5 and never reached the nightly line; it now exits
1 with the nightly sweep done and the promotion sweep skipped.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Retention protected a packages candidate only while an OPEN bot release
PR pinned its digest. The digest stops being eligible for deletion only
when finalize's retag adds a stable tag to the manifest, because that is
what makes the `all(tags match promotion-*)` predicate stop matching —
and finalize runs after the merge that closes the PR.
The window between them is where a half-finished release sits. Finalize
creates the write-once stable git tag, then dies before the retag: a
lost runner, a 5xx publishing the draft, a non-fast-forward on the
maintenance branch. What is left pins the candidate from a merge commit
and a tag nobody can move, with no open PR and only a temporary tag on
the manifest. Thirty days later retention deletes it, after which
re-running finalize 404s at the candidate check and re-dispatching
promote is refused because the stable tag exists. The release can then
only be finished by hand — the exact class of manual cleanup this design
exists to remove.
Protect the tips of the branches a promote PR can merge into as well.
The two sources are read differently on purpose. An open promote PR's
head is supposed to carry exactly one candidate pin, so anything else is
an anomaly and blocks. A branch tip is an additional source: every
digest pin found is protected and a branch without one contributes
nothing, because a maintenance line cut before this pipeline existed
cannot pin a candidate and must not wedge retention forever.
Blocking now stops the promotion sweep alone. It used to `exit 1` inside
the resolution loop, which also killed the unrelated nightly prune
further down the same step — so a WIP commit adding a trailing comment
to a pin on a release branch would silently stop GHCR nightly pruning.
The job still fails; it fails after the nightly sweep has run.
Verified by running the step's script against a stubbed `gh`: a digest
pinned by main's tip and referenced by no open PR is now excluded while
a genuinely abandoned one is still listed; a manifest that also carries
a stable tag stays excluded by the existing predicate; and a malformed
pin on a PR head leaves the nightly sweep running, skips the promotion
sweep and exits 1.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The zizmor "Audit workflows" gate (added in #3223) is red on main: five
actions/* references in the release workflows from #3017 are pinned to a
floating tag instead of a commit SHA (blanket unpinned-uses policy, High
severity):
- nightly.yaml (x3) actions/checkout@v4
- promote-rc.yaml actions/checkout@v4
- retention.yaml actions/create-github-app-token@v1
Pin each to the SHA the rest of the repo already standardizes on:
- actions/checkout -> 34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- actions/create-github-app-token -> d72941d797fd3113feb6b93fd0dec494b13a2547 # v1
`zizmor --offline .github/workflows/` now reports no findings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/nightly.yaml:
the version was 0.0.0-nightly.<date>, so a same-day re-dispatch or re-run
republished every image, chart and artifact under the same tag — the dated
nightly was not write-once. Append the run id and attempt so each build gets a
distinct, immutable tag, and widen retention.yaml's selector regex to match the
extra numeric components (still excluding the floating :nightly and any release
tag).
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on .github/workflows/retention.yaml:
gh api --paginate with --jq runs the filter once per page and emits one JSON
array per page, so total/drop_ids miscount once a package's versions span more
than one page. Pipe the raw pages through 'jq -s add' to merge them into a
single array before counting and slicing.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Nightlies are now GHCR OCI tags (0.0.0-nightly.<date>), not git tags or
GitHub releases, so:
- retention.yaml now prunes old GHCR container-package versions via the
Packages API instead of git tags/releases. Scoped to versions whose
tags are ALL 0.0.0-nightly.<YYYYMMDD>, so the floating :nightly tag,
release tags (vX.Y.Z, latest, rc, main) and untagged versions are
never touched. Keeps newest 14 per package; dry-run by default.
- nightly-rc.yaml is removed: it cut nightly *git tags* that drove a
release/e2e path we no longer use. The nightly.yaml workflow replaces
it with the mirror + full-e2e flow.
actionlint clean.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
- nightly-rc.yaml: daily cron that cuts a write-once
vX.Y.Z-nightly.<date> tag for each tracked target (main + active
release-X.Y branches) when its HEAD moved since the last nightly. The tag
drives the normal rc build/e2e path. Gated by the NIGHTLY_RC_ENABLED repo
variable so it can merge dark and be switched on deliberately.
- retention.yaml: daily cron that keeps the newest 14 nightlies per line and
deletes older nightly tags + their releases. Delete-only, scoped strictly
to the *-nightly.* namespace (rc/stable untouched). Dry-run by default;
flip the `apply` input or the RETENTION_APPLY repo variable to delete.
Registry image pruning is left as a tracked TODO (no OCIR delete parity).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>