ci(retention): derive swept package names from the published tree

GitHub refuses to list org container packages to an app installation
token, so the retention workflow cannot enumerate what to sweep. Name
the packages from the published nightly packages tree instead: it is
the tree the nightly mirror copied images for, so it covers every
component image that receives dated nightly tags, including those
digest-pinned only by the build and therefore absent from the
committed tree. A tree that does not reference the packages artifact
is refused rather than read as a shorter list.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>

AGENTS.md

@@ -20,7 +20,7 @@ This file provides structured guidance for AI coding assistants and agents worki
 
 - **Image references, tags and digests** (e.g., "why is this image tagged rc", "add an image to a package", "change how an image is stamped", "promotion/nightly missed an image", "where do image refs live")
   - Read: [`image-refs.md`](./docs/agents/image-refs.md)
-  - Action: Read the entire file. Image refs live in three storage shapes and carry three classes of tag; a tool that knows only one shape skips images silently rather than failing. `hack/lib/image-refs.sh` is the enumeration shared by the promote, retag, mirror and candidate-verify tooling — extend it there rather than teaching an individual script a new path. The one exception is `hack/overlay-main-images.sh`, which reads refs but walks the tree itself, so a newly declared file does not reach it; `hack/promote-packages-artifact.sh` sources nothing because it publishes the whole tree and consumes no refs at all — the doc carries both cases
+  - Action: Read the entire file. Image refs live in three storage shapes and carry three classes of tag; a tool that knows only one shape skips images silently rather than failing. `hack/lib/image-refs.sh` is the enumeration shared by the promote, retag, mirror, candidate-verify, multi-arch stitch and verify, and retention tooling — extend it there rather than teaching an individual script a new path. The one exception is `hack/overlay-main-images.sh`, which reads refs but walks the tree itself, so a newly declared file does not reach it; `hack/promote-packages-artifact.sh` sources nothing because it publishes the whole tree and consumes no refs at all — the doc carries both cases
 
 - **Retiring an application** (e.g., "remove the X app", "deprecate X", "drop X from the catalog", "what happens to instances people already created")
   - Read: [`deprecating-applications.md`](./docs/deprecating-applications.md)

docs/agents/image-refs.md

@@ -59,6 +59,7 @@ If a package stamps a reference somewhere the two globs do not cover, add that f
 | `hack/nightly-mirror.sh` | all three shapes | copy each digest to the public registry, then rewrite hosts in the same file set |
 | `hack/overlay-main-images.sh` | `values.yaml` + `*.tag` | overlay current-main references onto packages a PR did not rebuild |
 | `hack/stitch-multiarch.sh` | all three shapes | join each amd64 image and its `-arm64` twin into one index, repin the tree on it, then fail if any old digest survives |
+| `hack/retention-packages.sh` | all three shapes, in the published nightly tree | name the GHCR packages the retention workflow prunes; a ref it cannot see is a package whose nightlies are never pruned |
 | `hack/verify-multiarch.sh` | all three shapes, first- and third-party | fail a release unless every pinned image is an index with `linux/amd64` and `linux/arm64`, bar the repositories in `hack/multiarch-allowlist` |
 
 `hack/multiarch-allowlist` is the only bypass for that gate, and it is for images that cannot exist for arm64, not for images nobody has rebuilt yet. An entry is a repository with no tag or digest, so bumping an allowed image needs no edit, and it must carry a reason after `#` on the same line: the script refuses an entry without one. Allowed refs are still printed, under their own "allowed single-arch" heading, and in tree mode an entry that matches no ref that failed the check is reported as stale without changing the exit status, so it gets removed once the image is multi-arch or gone. `--refs-file` reports no stale entries, since a list of pulled images omits everything the run did not pull.

hack/lib/image-refs.sh

@@ -3,7 +3,8 @@
 #
 # Sourced by hack/promote-rewrite-tags.sh, hack/promote-retag.sh,
 # hack/nightly-mirror.sh, hack/verify-promoted-packages.sh,
-# hack/stitch-multiarch.sh and hack/verify-multiarch.sh. It exists because the
+# hack/stitch-multiarch.sh, hack/verify-multiarch.sh and
+# hack/retention-packages.sh. It exists because the
 # first three call sites each grew their own idea of where a ref can live, and
 # drifted: promote-retag and nightly-mirror scanned only the depth-2
 # values.yaml, while the promote workflow's tag rewrite scanned those plus

hack/retention-packages.sh

@@ -0,0 +1,52 @@
+#!/bin/sh
+# Print the GHCR container packages the retention workflow sweeps, one per line.
+#
+# Usage: retention-packages.sh <published-tree-dir>
+#
+# GET /orgs/{org}/packages?package_type=container answers "400 Invalid
+# argument" to the app installation token the workflow uses, while the
+# per-package versions endpoints accept it, so the org cannot be listed. The
+# names come instead from the published nightly packages tree: it is the tree
+# hack/nightly-mirror.sh copied images for, so it names every component image
+# that receives dated nightly tags. The committed tree is not a substitute — an
+# image whose ref is digest-pinned only by the build is mirrored nightly yet
+# invisible there.
+#
+# A wrong list fails quietly (a package left out is simply never pruned), so a
+# tree that does not even reference the packages artifact is refused rather than
+# read as "fewer packages". A component removed from the tree drops out too and
+# keeps whatever dated nightlies it had; it receives no new ones.
+set -eu
+
+TREE="${1:?usage: retention-packages.sh <published-tree-dir>}"
+DST_REGISTRY="ghcr.io/cozystack/cozystack"
+
+[ -d "$TREE" ] || { echo "retention-packages: '$TREE' is not a directory" >&2; exit 1; }
+yq --version 2>&1 | grep -q mikefarah || { echo "retention-packages: yq (mikefarah) is required" >&2; exit 1; }
+
+# shellcheck source=hack/lib/image-refs.sh
+. "$(dirname "$0")/lib/image-refs.sh"
+
+# Package names keep the namespace under the org: ghcr.io/cozystack/cozystack/x
+# is the package "cozystack/x".
+package_names() {
+  collect_image_refs "$TREE" | while IFS= read -r ref; do
+    repo="${ref%@*}"
+    case "$repo" in
+      "${DST_REGISTRY}/"*) ;;
+      *) continue ;;
+    esac
+    img="${repo##*/}"
+    repo="${repo%/*}/${img%:*}"
+    printf '%s\n' "${repo#"${DST_REGISTRY%/*}/"}"
+  done
+}
+names="$(package_names)"
+
+printf '%s\n' "$names" | grep -qx 'cozystack/cozystack-packages' || {
+  echo "retention-packages: '$TREE' does not reference ${DST_REGISTRY}/cozystack-packages; not a published tree" >&2
+  exit 1
+}
+
+# Published by the nightly workflow alongside the tree, never referenced by it.
+printf '%s\n%s\n%s\n' "$names" cozystack/cozy-installer cozystack/cozystack-nocloud | sort -u

hack/retention-packages_test.bats

@@ -0,0 +1,80 @@
+#!/usr/bin/env bats
+# Tests for hack/retention-packages.sh — the package-name source of the GHCR
+# retention workflow. GitHub refuses to list org container packages to an app
+# installation token, so the workflow sweeps exactly the packages this script
+# names. A name missing here is a package whose nightlies are never pruned, so
+# the tests pin what is included and that an unusable tree fails rather than
+# yielding a shorter list.
+#
+# Run with: bats hack/retention-packages_test.bats
+
+load test_helper
+
+SCRIPT="${BATS_TEST_DIRNAME}/retention-packages.sh"
+
+# A published nightly tree: refs already rewritten to the GHCR host.
+_make_tree() {
+  D="$(printf 'a%.0s' $(seq 1 64))"
+  t="$1"
+  mkdir -p "$t/core/installer" "$t/system/single" "$t/system/split" \
+           "$t/system/nested" "$t/system/third" "$t/system/leftover" \
+           "$t/system/tagfile/images" "$t/system/multus/templates"
+  {
+    echo 'cozystackOperator:'
+    echo '  platformSourceUrl: oci://ghcr.io/cozystack/cozystack/cozystack-packages'
+    printf '  platformSourceRef: digest=sha256:%s\n' "$D"
+  } > "$t/core/installer/values.yaml"
+  printf 'image: ghcr.io/cozystack/cozystack/single:v1.0.0@sha256:%s\n' "$D" \
+    > "$t/system/single/values.yaml"
+  {
+    echo 'image:'
+    echo '  repository: ghcr.io/cozystack/cozystack/split'
+    printf '  tag: v1.0.0@sha256:%s\n' "$D"
+  } > "$t/system/split/values.yaml"
+  printf 'image: ghcr.io/cozystack/cozystack/metallb/controller:v1@sha256:%s\n' "$D" \
+    > "$t/system/nested/values.yaml"
+  printf 'ghcr.io/cozystack/cozystack/tagfile:v1.0.0@sha256:%s\n' "$D" \
+    > "$t/system/tagfile/images/thing.tag"
+  printf '          image: ghcr.io/cozystack/cozystack/multus-cni:v1@sha256:%s\n' "$D" \
+    > "$t/system/multus/templates/multus-daemonset-thick.yml"
+  # Not ours to prune: a third-party image and a sibling GHCR namespace.
+  {
+    printf 'a: docker.io/library/busybox:1.37.0@sha256:%s\n' "$D"
+    printf 'b: ghcr.io/cozystack/ingress-nginx-with-protobuf-exporter/controller:v1@sha256:%s\n' "$D"
+  } > "$t/system/third/values.yaml"
+  # A ref the mirror failed to rewrite lives in the build registry, not GHCR.
+  printf 'image: iad.ocir.io/idyksih5sir9/cozystack/leftover:main@sha256:%s\n' "$D" \
+    > "$t/system/leftover/values.yaml"
+}
+
+@test "names every GHCR cozystack package in all storage shapes plus the published extras" {
+  tree="${BATS_TEST_TMPDIR}/tree"
+  _make_tree "$tree"
+  run "$SCRIPT" "$tree"
+  [ "$status" -eq 0 ]
+  expected="cozystack/cozy-installer
+cozystack/cozystack-nocloud
+cozystack/cozystack-packages
+cozystack/metallb/controller
+cozystack/multus-cni
+cozystack/single
+cozystack/split
+cozystack/tagfile"
+  [ "$output" = "$expected" ]
+}
+
+@test "fails on a missing tree" {
+  run "$SCRIPT" "${BATS_TEST_TMPDIR}/absent"
+  [ "$status" -ne 0 ]
+}
+
+@test "fails on a tree that does not reference the packages artifact" {
+  tree="${BATS_TEST_TMPDIR}/tree"
+  _make_tree "$tree"
+  rm "$tree/core/installer/values.yaml"
+  run "$SCRIPT" "$tree"
+  [ "$status" -ne 0 ]
+  case "$output" in
+    *cozystack/single*) echo "FAIL: names printed despite failure"; false ;;
+  esac
+}