mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-08 21:14:56 +09:00
Aleksei Sviridkin
da1f1cc387
The app installation token lists package versions but its DELETE of a listed version is answered with 404, while the repository's GITHUB_TOKEN with packages: write deleted the same version without any change to package settings. Deletes therefore go through GITHUB_TOKEN; listing keeps the app token. That deletion was shown on a package linked to this repository. A package linked elsewhere is not attempted: it is skipped before its versions are read and named in a warning, so it cannot turn every apply run red. A failed read of the link still fails the run, and so does a sweep in which no resolved package is linked, since that sweep prunes nothing. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>
329 lines
17 KiB
YAML
329 lines
17 KiB
YAML
name: Artifact Retention
|
|
|
|
# Prune old nightly artifacts from GHCR: keep the newest KEEP dated versions of
|
|
# each cozystack package the current nightly publishes, delete older ones.
|
|
# Nightlies are GHCR OCI tags (0.0.0-nightly.<date>.<run-id>.<attempt> on every
|
|
# component image plus cozystack-packages, cozy-installer and cozystack-nocloud)
|
|
# — not git tags or releases — so this prunes container-package versions via
|
|
# the GitHub Packages API.
|
|
#
|
|
# Strictly scoped: only versions whose tags are ALL of the form
|
|
# 0.0.0-nightly.<YYYYMMDD>.<run-id>.<attempt> are eligible. A version that also
|
|
# carries the floating `nightly` tag (the newest) or any release tag (vX.Y.Z,
|
|
# latest, rc, main) is never touched, so stable/rc/main artifacts can never be
|
|
# pruned.
|
|
#
|
|
# Promotion also creates one temporary cozystack-packages tag of the form
|
|
# promotion-vX.Y.Z-from-vX.Y.Z-rc.N-run-ID-ATTEMPT. Every dispatch gets a unique
|
|
# tag, so a retry never moves one and leaves an untagged manifest behind. A
|
|
# successful promotion adds a stable tag to the same manifest, which makes it
|
|
# ineligible below. A candidate stays protected while an open bot-authored
|
|
# release PR, or the tip of main or a release-X.Y branch, pins its digest;
|
|
# otherwise it becomes eligible after PROMOTION_TTL_DAYS only when ALL its tags
|
|
# are still temporary.
|
|
#
|
|
# Dry-run by default; flip the input or the RETENTION_APPLY repo variable to
|
|
# actually delete. Deletion is irreversible.
|
|
#
|
|
# NOTE (admin): the cozystack-ci app installation token cannot delete here: it
|
|
# lists package versions, yet DELETE of a listed version answered "404 Package
|
|
# not found". This repository's GITHUB_TOKEN, with the job's packages: write
|
|
# and no change to package settings, deleted the same version from a package
|
|
# linked to this repository, so deletes use it. A package linked to another
|
|
# repository is skipped with a warning.
|
|
|
|
on:
|
|
schedule:
|
|
- cron: '30 4 * * *' # daily 04:30 UTC, after the nightly has published
|
|
workflow_dispatch:
|
|
inputs:
|
|
apply:
|
|
description: "Actually delete (false = dry-run, the default)"
|
|
required: false
|
|
default: false
|
|
type: boolean
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
env:
|
|
ORG: cozystack
|
|
KEEP: '14'
|
|
PROMOTION_TTL_DAYS: '30'
|
|
|
|
concurrency:
|
|
group: artifact-retention
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
prune:
|
|
name: Prune old GHCR temporary artifacts
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
pull-requests: read
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Set up flux
|
|
uses: fluxcd/flux2/action@04acaec6161ac4fb1a82ffafa88901c03271d34f # v2.8.6
|
|
with:
|
|
version: "2.8.6"
|
|
|
|
- name: Generate GitHub App token
|
|
id: app-token
|
|
uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1
|
|
with:
|
|
app-id: ${{ secrets.COZYSTACK_CI_APP_ID }}
|
|
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
|
|
owner: cozystack
|
|
|
|
- name: Prune
|
|
env:
|
|
GH_TOKEN: ${{ steps.app-token.outputs.token }}
|
|
# workflow_dispatch input wins; otherwise the repo variable; else dry-run.
|
|
APPLY: ${{ github.event.inputs.apply || vars.RETENTION_APPLY || 'false' }}
|
|
# See the admin NOTE above for why deletes do not use the app token.
|
|
DELETE_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$APPLY" = "true" ]; then echo "MODE: APPLY (deleting)"; else echo "MODE: dry-run"; fi
|
|
|
|
# Protect every immutable packages digest a live promotion still
|
|
# depends on. Age alone does not imply abandonment: an approved
|
|
# release PR can legitimately stay open longer than the TTL.
|
|
#
|
|
# A candidate becomes permanently safe only when finalize's retag adds
|
|
# the stable tag to its manifest, because that is what makes the
|
|
# all(promotion-*) predicate at the bottom stop matching it. Finalize
|
|
# runs post-merge, so an open-PR-only protection has a hole exactly
|
|
# where a half-finished release sits: finalize creates the write-once
|
|
# stable git tag, then dies before the retag, and the digest is now
|
|
# pinned by a merge commit and a tag nobody can move, referenced by no
|
|
# open PR, on a manifest carrying only its temporary tag. Deleting it
|
|
# leaves a release that can be finished only by hand.
|
|
#
|
|
# Two sources, therefore, read differently on purpose. A failure in
|
|
# either blocks the promotion sweep at the bottom and nothing else —
|
|
# the nightly sweep in between shares no state with it and must not
|
|
# be held hostage to a bad pin on a release branch.
|
|
promotion_retention_blocked=0
|
|
protected_promotion_digests=""
|
|
protect_digest() {
|
|
protected_promotion_digests="${protected_promotion_digests:+${protected_promotion_digests} }$1"
|
|
}
|
|
|
|
# Source 1 — open, trusted, same-repository promote PRs opened by the
|
|
# release bot. Their head is SUPPOSED to carry exactly one candidate
|
|
# pin, so anything else is an anomaly: block rather than skip.
|
|
#
|
|
# The listing that feeds the loop defers the same way instead of
|
|
# aborting the step. Under `set -euo pipefail` an unhandled failure
|
|
# here exits before the nightly sweep below has run, which is the
|
|
# coupling the deferred guard exists to remove — and the handler must
|
|
# empty the list rather than leave a partially paginated one, because
|
|
# a shorter list is a smaller protected set. Blocked plus empty is the
|
|
# fail-closed reading: nothing can be called abandoned while the
|
|
# protected set is unknown.
|
|
release_pr_shas="$(gh api --paginate "/repos/${GITHUB_REPOSITORY}/pulls?state=open&per_page=100" \
|
|
| jq -s -r --arg repo "$GITHUB_REPOSITORY" 'add[]
|
|
| select(.user.login == "cozystack-ci[bot]")
|
|
| select(.head.repo.full_name == $repo)
|
|
| select(.head.ref | test("^release-[0-9]+\\.[0-9]+\\.[0-9]+$"))
|
|
| select(any(.labels[]?; .name == "release"))
|
|
| .head.sha')" || {
|
|
echo "::error::Could not list open pull requests; refusing retention of promotion candidates."
|
|
promotion_retention_blocked=1
|
|
release_pr_shas=""
|
|
}
|
|
for sha in $release_pr_shas; do
|
|
values="$(gh api \
|
|
-H 'Accept: application/vnd.github.raw+json' \
|
|
"/repos/${GITHUB_REPOSITORY}/contents/packages/core/installer/values.yaml?ref=${sha}")" || {
|
|
echo "::error::Open release PR at ${sha} has unreadable installer values; refusing retention of promotion candidates."
|
|
promotion_retention_blocked=1
|
|
continue
|
|
}
|
|
candidate_ref="$(printf '%s\n' "$values" | awk '
|
|
/^[[:space:]]*platformSourceRef:[[:space:]]*/ {
|
|
sub(/^[[:space:]]*platformSourceRef:[[:space:]]*/, "")
|
|
gsub(/^[\047"]|[\047"][[:space:]]*$/, "")
|
|
print
|
|
}')"
|
|
ref_lines="$(printf '%s\n' "$candidate_ref" | wc -l | tr -d ' ')"
|
|
if ! printf '%s\n' "$candidate_ref" | grep -Eq '^digest=sha256:[0-9a-f]{64}$' \
|
|
|| [ "$ref_lines" -ne 1 ]; then
|
|
echo "::error::Open release PR at ${sha} has no single immutable platformSourceRef; refusing retention of promotion candidates."
|
|
promotion_retention_blocked=1
|
|
continue
|
|
fi
|
|
protect_digest "${candidate_ref#digest=}"
|
|
done
|
|
|
|
# Source 2 — the tips of the branches a promote PR can merge into.
|
|
# This is what covers the window between merge and retag. Read
|
|
# permissively, the opposite of source 1: protect every digest pin
|
|
# found and treat a branch without one as contributing nothing. A
|
|
# maintenance line cut before this pipeline existed cannot pin a
|
|
# promotion candidate, because none existed when it was cut, so its
|
|
# missing or tag-form pin is not an anomaly and must not wedge the
|
|
# sweep forever. Reading permissively can only ever protect more.
|
|
#
|
|
# Permissive is about what a branch CONTAINS, not about whether the
|
|
# branches were listed at all: a failed listing enumerates no branches
|
|
# and so silently protects nothing, which is the one way this source
|
|
# can narrow the protected set. It therefore blocks and empties, like
|
|
# source 1, and for the same reason does it here rather than by
|
|
# letting `set -e` take the nightly sweep with it.
|
|
base_branches="$(gh api --paginate "/repos/${GITHUB_REPOSITORY}/branches?per_page=100" \
|
|
| jq -s -r 'add[] | .name
|
|
| select(. == "main" or test("^release-[0-9]+\\.[0-9]+$"))')" || {
|
|
echo "::error::Could not list base branches; refusing retention of promotion candidates."
|
|
promotion_retention_blocked=1
|
|
base_branches=""
|
|
}
|
|
for branch in $base_branches; do
|
|
values="$(gh api \
|
|
-H 'Accept: application/vnd.github.raw+json' \
|
|
"/repos/${GITHUB_REPOSITORY}/contents/packages/core/installer/values.yaml?ref=${branch}" 2>/dev/null)" || {
|
|
echo " ${branch}: no readable installer values; nothing to protect"
|
|
continue
|
|
}
|
|
branch_digests="$(printf '%s\n' "$values" \
|
|
| grep -oE 'digest=sha256:[0-9a-f]{64}' | sed 's/^digest=//' | sort -u || true)"
|
|
for digest in $branch_digests; do
|
|
protect_digest "$digest"
|
|
done
|
|
done
|
|
|
|
protected_count="$(printf '%s\n' "$protected_promotion_digests" | awk '{print NF}')"
|
|
echo "Protecting ${protected_count} promotion candidate digest(s) named by an open release PR or a base-branch tip."
|
|
|
|
# The cozystack/* packages to sweep. GitHub refuses to list org
|
|
# container packages to an app installation token (400 Invalid
|
|
# argument), so the names come from the published nightly tree; see
|
|
# hack/retention-packages.sh for why that tree and what it covers.
|
|
#
|
|
# An empty list is never "nothing to do": the nightly publishes into
|
|
# these packages every day, so empty means the names could not be
|
|
# resolved. That skips the nightly sweep and fails the run, but not
|
|
# before the promotion sweep below: it reads one fixed package, and
|
|
# what keeps a candidate safe is the protected set, which no part of
|
|
# this list feeds.
|
|
#
|
|
# A package whose versions cannot be read, or a version that cannot
|
|
# be deleted, is likewise reported and skipped so the rest is still
|
|
# swept, then fails the run at the end. The loop is fed by a
|
|
# here-string rather than a pipe so the flag survives it.
|
|
sweep_failed=0
|
|
unlinked=""
|
|
linked_count=0
|
|
gh_err="$(mktemp)"
|
|
tree="$(mktemp -d)"
|
|
pkgs="$(flux pull artifact "oci://ghcr.io/${ORG}/cozystack/cozystack-packages:nightly" --output "$tree" >/dev/null \
|
|
&& ./hack/retention-packages.sh "$tree")" || {
|
|
echo "::error::Could not resolve the ${ORG} container packages to sweep."
|
|
pkgs=""
|
|
}
|
|
if [ -z "$pkgs" ]; then
|
|
echo "::error::No package names resolved; skipping the nightly sweep."
|
|
sweep_failed=1
|
|
else
|
|
echo "Sweeping $(printf '%s\n' "$pkgs" | wc -l | tr -d ' ') package(s):"
|
|
printf '%s\n' "$pkgs" | sed 's/^/ /'
|
|
fi
|
|
while IFS= read -r pkg; do
|
|
[ -n "$pkg" ] || continue
|
|
enc="$(jq -rn --arg p "$pkg" '$p|@uri')"
|
|
|
|
# DELETE_TOKEN was shown to delete from a package linked to this
|
|
# repository; one linked elsewhere is not attempted, only named in
|
|
# a warning, so it cannot fail every apply run.
|
|
linked="$(gh api "/orgs/${ORG}/packages/container/${enc}" --jq '.repository.full_name // ""' 2>"$gh_err")" || {
|
|
echo "::error::Could not read ${pkg}; not swept: $(cat "$gh_err")"
|
|
sweep_failed=1
|
|
continue
|
|
}
|
|
if [ "$linked" != "$GITHUB_REPOSITORY" ]; then
|
|
unlinked="${unlinked:+${unlinked} }${pkg}"
|
|
continue
|
|
fi
|
|
linked_count=$((linked_count + 1))
|
|
|
|
# Version ids whose tags are non-empty AND every tag is a dated
|
|
# nightly (so the floating `nightly` tag and any release tag exclude
|
|
# the version). API returns versions newest-first, so the dated
|
|
# nightlies are already in newest->oldest order.
|
|
# `gh api --paginate --jq` runs the filter per page and emits one
|
|
# array per page, so length/slicing break once a package spans
|
|
# multiple pages. Slurp the raw pages and merge with `jq -s add`
|
|
# first, then count and slice the combined array.
|
|
ids="$(gh api --paginate "/orgs/${ORG}/packages/container/${enc}/versions" 2>"$gh_err" \
|
|
| jq -s '[add[]
|
|
| select((.metadata.container.tags | length) > 0)
|
|
| select(.metadata.container.tags | all(test("^0\\.0\\.0-nightly\\.[0-9]{8}(\\.[0-9]+)*$")))
|
|
| .id]')" || {
|
|
echo "::error::Could not read ${pkg} versions; not swept: $(cat "$gh_err")"
|
|
sweep_failed=1
|
|
continue
|
|
}
|
|
|
|
total="$(printf '%s' "$ids" | jq 'length')"
|
|
[ "$total" -gt "$KEEP" ] || { echo "── ${pkg}: ${total} nightlies (<= ${KEEP}); nothing to prune"; continue; }
|
|
drop_ids="$(printf '%s' "$ids" | jq -r ".[${KEEP}:][]")"
|
|
echo "── ${pkg}: ${total} nightlies, keep ${KEEP}, drop $((total - KEEP))"
|
|
|
|
printf '%s\n' "$drop_ids" | xargs ./hack/retention-delete.sh "$pkg" version || sweep_failed=1
|
|
done <<< "$pkgs"
|
|
[ -z "$unlinked" ] || echo "::warning::not linked to ${GITHUB_REPOSITORY}, skipped: ${unlinked}"
|
|
# One unlinked package is expected; all of them means nothing was swept.
|
|
if [ -n "$pkgs" ] && [ "$linked_count" -eq 0 ]; then
|
|
echo "::error::None of the resolved packages is linked to ${GITHUB_REPOSITORY}; the nightly sweep pruned nothing."
|
|
sweep_failed=1
|
|
fi
|
|
|
|
# Anything unresolved above means the protected set is incomplete, so
|
|
# no candidate can be called abandoned. Skip this sweep entirely and
|
|
# fail the job. Deliberately here and not at the point of failure: the
|
|
# nightly sweep above is unrelated work that had no reason to be
|
|
# cancelled by a bad pin on a release branch, and it has already run.
|
|
if [ "$promotion_retention_blocked" -eq 1 ]; then
|
|
echo "::error::promotion candidate protection could not be resolved; skipping the promotion sweep."
|
|
exit 1
|
|
fi
|
|
|
|
# A promoted manifest acquires vX.Y.Z (and possibly latest) in addition
|
|
# to its temporary tag, so the all(tags match promotion-*) predicate is
|
|
# the safety boundary: published bytes are never eligible. Digests
|
|
# named by an open promote PR or by a base-branch tip are excluded
|
|
# separately, which is what covers a candidate between merge and
|
|
# retag. Only cozystack-packages receives these candidate tags.
|
|
pkg='cozystack/cozystack-packages'
|
|
enc="$(jq -rn --arg p "$pkg" '$p|@uri')"
|
|
cutoff="$(date -u -d "${PROMOTION_TTL_DAYS} days ago" '+%Y-%m-%dT%H:%M:%SZ')"
|
|
promotion_ids="$(gh api --paginate "/orgs/${ORG}/packages/container/${enc}/versions" 2>"$gh_err" \
|
|
| jq -s --arg cutoff "$cutoff" --arg protected "$protected_promotion_digests" '[add[]
|
|
| .name as $digest
|
|
| select((.metadata.container.tags | length) > 0)
|
|
| select(.metadata.container.tags | all(test("^promotion-v[0-9]+\\.[0-9]+\\.[0-9]+-from-v[0-9]+\\.[0-9]+\\.[0-9]+-rc\\.[0-9]+-run-[0-9]+-[0-9]+$")))
|
|
| select(.updated_at < $cutoff)
|
|
| select(($protected | split(" ") | index($digest)) == null)
|
|
| .id]')" || {
|
|
echo "::error::Could not read ${pkg} versions; promotion candidates not swept: $(cat "$gh_err")"
|
|
sweep_failed=1
|
|
promotion_ids='[]'
|
|
}
|
|
promotion_total="$(printf '%s' "$promotion_ids" | jq 'length')"
|
|
echo "── ${pkg}: ${promotion_total} abandoned promotion candidate(s) older than ${PROMOTION_TTL_DAYS} days"
|
|
printf '%s' "$promotion_ids" | jq -r '.[]' \
|
|
| xargs ./hack/retention-delete.sh "$pkg" "promotion candidate version" || sweep_failed=1
|
|
if [ "$sweep_failed" -eq 1 ]; then
|
|
echo "::error::some versions could not be read or deleted; see the errors above."
|
|
exit 1
|
|
fi
|
|
echo "Done."
|