Commit Graph
24 Commits
Author SHA1 Message Date
Myasnikov Daniil ce054baa34 feat(release): attach an SBOM to every release image
A release said nothing about what is inside its container images, so a
user could not tell which packages a cluster runs. SBOM=1 had been
plumbed for it and left off.

BuildKit attaches the SBOM as an attestation, which makes the pushed
image an index of the image and its attestation manifest. The stitch took
such an index for something other than one amd64 and one arm64 image,
skipped it and still exited 0, and the multi-arch gate then failed the
release, so the stitch now reads through the index and the stitched one
keeps both attestations. The release build ran on the default docker
driver, which cannot attach attestations, so it moves to the
docker-container builder with the docker.io mirror, as the arm64 leg
already did. The scanner is pinned by digest where --sbom=true pulls a
floating tag.

The Talos installer image is built by the imager and pushed by skopeo, so
it carries no attestation.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-20 16:40:06 +05:00
Aleksei Sviridkin 7c46ffff12 fix(backupstrategy-controller): run backup clients on a package-owned image (#4756)
## What this PR does

The Altinity (ClickHouse) and RabbitMQ backup strategy Pods ran
`platform-migrations`: a 349 MB image carrying `kubectl`, `helm`, `git`,
`cozyhr` and `etcd-migrate`, whose `ENTRYPOINT` is `run-migrations.sh`.
Both pins were stamped from `packages/core/platform/Makefile`, so any
migration change re-digested the backup clients too, and only the
`command:` override kept the Pods from running platform migrations
(#3420).

- New
`packages/system/backupstrategy-controller/images/backup-http-client/`:
`alpine:3.24` (digest-pinned, the same base the tree already uses) +
`curl` + `jq` + `ca-certificates`, `USER 65532:65532`, no `ENTRYPOINT`.
Everything is installed at build time, so the Pods still never run `apk
add` and keep working air-gapped.
- New `image-backup-http-client` target in the package's own `Makefile`,
chained after `image-redis-backup` like the existing ones so only one
`yq -i` writes `values.yaml` at a time. It stamps `chBackupClientImage`
and `rabbitmqBackupClientImage` with the same ref.
- Removed the cross-package stamps from
`packages/core/platform/Makefile`, the WORKAROUND/TODO comments in the
chart and the RabbitMQ example, and the stale "live violation" line in
`docs/agents/image-refs.md`.
- `hack/image-pin-consistency.bats`: dropped the
`platform-migrations`-specific test, whose subject no longer exists. The
general check (no repository pinned at more than one digest, computed
from `hack/promote-retag.sh --dry-run`) still covers the two keys
sharing one image.

**Why one image for both strategies rather than `ch-backup-client` as
the issue named it.** The RabbitMQ strategy had the same reuse and the
same cross-package stamp; leaving it would have kept the stamp alive.
Its script needs `curl --aws-sigv4` + `ca-certificates`, a subset of the
ClickHouse one, so one image serves both. The two values keys stay
separate so either strategy can move to its own image later.

**Initial pin.** The image does not exist in ghcr until CI builds it, so
both keys are committed as `backup-http-client:v1.6.0` without a digest,
the same way `redis-backup` was introduced. PR CI rebuilds touched
packages and overlays the fresh refs for e2e; release preparation's
`make image` stamps tag+digest.

Image size: 6.9 MB vs 349 MB. Verified locally:
- `docker buildx build` for `linux/amd64,linux/arm64`; in the image as
uid 65532: curl 8.22 with `--aws-sigv4`, jq 1.8.2, `date`, `seq`, writes
to `/tmp` (the RabbitMQ script's only write), TLS verification against
the bundled CA store.
- `helm unittest` (23/23), `helm template` renders both strategies with
`backup-http-client` and no `platform-migrations` reference.
- `hack/cozytest.sh hack/image-pin-consistency.bats` and
`hack/build-matrix_test.bats` pass.

Not yet run end to end on a cluster; the e2e backup round-trips will
exercise it.

Fixes #3420

### Screenshots

N/A — no UI changes.

### Downstream repositories

Walked the trigger map against the diff: it touches
`packages/system/backupstrategy-controller` (new image, Makefile target,
values comments/pins, template comments),
`packages/core/platform/Makefile` (stamp removal only, no `values.yaml`
key change), `hack/image-pin-consistency.bats`,
`docs/agents/image-refs.md` and a comment in
`examples/backups/rabbitmq`. No app package, schema, CRD, variant,
asset, platform values key or `hack/package.mk` change. The website does
not list first-party image repositories (searched for
`platform-migrations` and `redis-backup`), so no mirror list needs the
new one.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(backupstrategy-controller): the ClickHouse (Altinity) and RabbitMQ backup strategy Pods now run a dedicated `backup-http-client` image (alpine + curl + jq + ca-certificates, non-root, ~7 MB) instead of the 349 MB `platform-migrations` image. Air-gapped mirrors need the new `ghcr.io/cozystack/cozystack/backup-http-client` repository.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Improvements**
* Altinity and RabbitMQ backups now use a dedicated backup HTTP client
image, configured separately from the platform migrations image.
* The client image includes the tools required for backup operations and
runs with a non-root identity.
* Both backup configurations receive the same digest-pinned image
reference when the image is built.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-12 12:38:55 +03:00
Andrey Kolkov 770f515105 docs(backupstrategy-controller): state the first-commit pin exception
docs/agents/image-refs.md forbade committing a bare tag without saying
how a new image's first pin can carry a digest it does not have yet.
Record the exception the backup client keys rely on: the bare tag is a
placeholder the producing `make image` replaces, and an unbuilt tree
pulls a tag that does not exist. Drop the securitygroup-controller
example, which is pinned by digest now.

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-10-12 09:54:52 +04:00
Andrey Kolkov 647407545d fix(backupstrategy-controller): run backup clients on a package-owned image
The Altinity and RabbitMQ strategy Pods ran platform-migrations, a 349 MB
image carrying kubectl, helm and etcd-migrate whose ENTRYPOINT is the
migration runner, and whose pins were stamped from packages/core/platform
so every migration change moved the backup clients too.

Build images/backup-http-client (alpine + curl + jq + ca-certificates,
non-root) from this package's Makefile and stamp both client keys from
it. The cross-package stamp in packages/core/platform/Makefile and the
platform-migrations-specific check in hack/image-pin-consistency.bats go
away; the general one-digest-per-repository check still covers the two
keys sharing one image.

Fixes #3420

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-10-12 09:54:52 +04:00
Aleksei Sviridkin da1f1cc387 ci(retention): delete with the workflow token
The app installation token lists package versions but its DELETE of a
listed version is answered with 404, while the repository's
GITHUB_TOKEN with packages: write deleted the same version without any
change to package settings. Deletes therefore go through GITHUB_TOKEN;
listing keeps the app token.

That deletion was shown on a package linked to this repository. A
package linked elsewhere is not attempted: it is skipped before its
versions are read and named in a warning, so it cannot turn every
apply run red. A failed read of the link still fails the run, and so
does a sweep in which no resolved package is linked, since that sweep
prunes nothing.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-10 21:15:48 +03:00
Aleksei Sviridkin 957397ed42 ci(retention): derive swept package names from the published tree
GitHub refuses to list org container packages to an app installation
token, so the retention workflow cannot enumerate what to sweep. Name
the packages from the published nightly packages tree instead: it is
the tree the nightly mirror copied images for, so it covers every
component image that receives dated nightly tags, including those
digest-pinned only by the build and therefore absent from the
committed tree. A tree that does not reference the packages artifact
is refused rather than read as a shorter list.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-09 17:12:14 +03:00
Aleksei Sviridkin 43bf2768c2 fix(release): collect refs stored as registry/image/tag/digest keys
The shared image-ref enumeration had no rule for the upstream
ingress-nginx values layout, where the repository sits in `image` and
the digest in its own `digest` key with no `@`. Both ingress-nginx
refs in the tree (the controller and the default backend) were
therefore invisible to the promote retag, the nightly mirror, the
multi-arch gate and the promoted-packages verifier.

The new rule requires a complete sha256 digest, so upstream's unpinned
default (`digest: ""`) and digest-less image maps emit nothing.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 05:40:47 +03:00
Aleksei Sviridkin c740031453 Merge remote-tracking branch 'origin/main' into ci/build-arm64-leg
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-29 11:20:27 +03:00
Aleksei Sviridkin cb035f8eac docs(agents): stop calling the image-ref known gaps unfixed
Two of the gaps listed there are handled in the tree now, and each entry
already says whether it is. The heading sentence still claimed all of
them were open.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 22:17:07 +03:00
Aleksei Sviridkin 5924b3fd34 docs: describe how a pre-release is built and gated for arm64
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 01:32:37 +03:00
Aleksei Sviridkin 144d080342 feat(build): gate a release on every pinned image being multi-arch
A release pins one ref per image for clusters of either architecture.
A ref that resolves to a single-platform manifest, or to an index
missing amd64 or arm64, installs on one and fails to pull on the
other, and nothing before an arm64 install would notice.

hack/verify-multiarch.sh resolves every ref hack/lib/image-refs.sh
enumerates, third-party ones included since an arm64 node pulls them
too, and fails naming each ref that is not an index carrying both
linux/amd64 and linux/arm64. A ref that cannot be resolved fails as
well, rather than passing unchecked. A rate limit or a 5xx is the one
answer retried, a bounded number of times, and a ref still refused
fails as rate-limited rather than as single-arch. --report prints the
same list without failing, for a tree that is not being released.

Images that cannot be built for both architectures go in
hack/multiarch-allowlist by repository, each with a reason, so a bump
needs no edit; they are still listed, and an entry that matches no
single-arch ref is named as stale.

--refs-file checks a list of refs instead of a tree, a tag-only ref
resolved by its tag, with the same allowlist and output. It is for
images the tree does not pin by digest, such as the ones an e2e run's
nodes actually pulled.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 01:32:36 +03:00
Aleksei Sviridkin 7753394bfe build: rebuild every package that has an image target
flux-plunger, keycloak-operator, kilo and migration-controller have
image targets but were missing from the root build list, which the
build matrix, the main build and release preparation all iterate. No
CI path rebuilt them, so their pins stayed on images pushed by hand
while Dockerfile, patch and base-image changes never shipped, and the
multi-arch pre-release build would find single-arch pins it cannot
replace. migration-controller shipped a bare :latest with no digest.

Add the four packages, and a test that derives the set from the
package Makefiles so a new image target cannot be left out again.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 00:57:24 +03:00
Aleksei Sviridkin f02963a2c7 fix(keycloak-operator): keep the image host inside repository
The image target stamped only `tag`, leaving the committed
`registry: ghcr.io` in place, so a build against the CI registry would
render the public host with a digest that exists only in the build
registry. Stamping the build host into `registry` instead would split
the source registry across two keys, where the nightly mirror's
substring host rewrite cannot reach it and the published tree would
keep pointing at the private registry.

Set `registry: ""` and keep the whole host in `repository`, stamped
by the image target. The upstream chart joins the two keys, so the
rendered image string is unchanged, and the ref now has the same
shape as every other stamped reference the mirror and promotion
already handle.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 00:57:24 +03:00
Aleksei Sviridkin 769d3b3897 fix(build): push the kamaji provider image under the build's IMAGE_TAG
The capi-providers-cpprovider Makefile assigned IMAGE_TAG itself, which
beats the value CI puts in the environment. Every PR, main and release
line build therefore pushed the same component-version tag and
overwrote each other's, and its second tag was built from an undefined
TAG variable and ended in a bare "-".

The image now goes through image-tags like every other package: the
build-unique IMAGE_TAG always, the component version only under
PUBLISH_VERSIONED=1. The stamped refs keep the component version as
their tag, so the promote, retag and mirror tooling see the same refs
as before.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-27 07:57:31 +03:00
Aleksei Sviridkin 9a7094427e feat(build): stitch amd64 and arm64 builds into multi-arch indexes
The arm64 images are built on native arm64 runners, apart from the
amd64 build, so each build pushes a single-platform image. The stitch
joins every first-party image with its -arm64 twin into one index,
moves every tag this build pushed for it to that index, and repins
the tree on the index digest, so one ref serves both architectures.

Most refs in the tree name only the build's IMAGE_TAG, while under
PUBLISH_VERSIONED=1 the build also pushes the component version, so
the refs alone cannot say which tags to move. The image-tags macro
therefore appends every tag it pushes to PUSHED_TAGS_LOG when that is
set, and the stitch moves exactly those, plus IMAGE_TAG and the tag a
ref carries. It reads no other tag: another build's tag that sits on
the same amd64 bytes is not this build's to move, and scanning every
tag of every repository would cost thousands of registry reads per
release.

It reads refs through hack/lib/image-refs.sh like every other
consumer, rewrites only the files that enumeration names, and then
searches the whole tree outside charts/, gzip payloads included, for
any old digest it re-pointed. A ref whose digest is also pinned where
the rewrite cannot reach is skipped instead of half rewritten.

The kamaji control-plane provider's chart ships its ref only inside
files/components.gz, built from a plain copy with gzip -n, which no
text search sees. The stitch rewrites both copies, recompressing with
gzip -n so the result is what the package's image target would have
written, rather than leaving that image amd64 only.

An image with no arm64 twin (talos, the e2e sandbox), or a pair that
is not one amd64 and one arm64 image, is skipped and named. A
registry read answered with a rate limit or a 5xx is retried a
bounded number of times; any other registry error fails the run at
once rather than reading as a missing arm64 image.

On a rerun after a stitch that failed partway, the build tag can
already sit on an index of the pinned image and an arm64 image, not
necessarily this run's if the arm64 leg rebuilt it. The stitch repins
on that index instead of skipping the image, and copies it to any of
the build's other tags still left on the amd64 image. A tag moved to
anything else fails the run, since the tree then no longer matches
the registry.

MATRIX_ARCH=arm64 makes hack/build-matrix.sh leave out the e2e
sandbox, which pins linux/amd64: an arm64 leg building it would push
a second amd64 image under its -arm64 tag.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-27 07:57:30 +03:00
Myasnikov Daniil 1118b340a7 fix(ci): teach the overlay to repair an unstamped images/*.tag ref
hack/overlay-main-images.sh walks images/*.tag, but every branch of its
line classifier needs either a YAML key or an @sha256:. A .tag file holds
one bare reference and nothing else, so until a build stamps it the line
has neither, the drift guard reads it as a non-ref change, and the
committed reference survives.

For a package onboarded with the :v0.0.0 placeholder this repo already
uses for a new image, that surviving reference is one no registry
serves. The overlay is the only thing that would replace it for a PR
that does not rebuild the package, so once such a package is on main
every full-suite run that skips its build keeps the placeholder. The
same placeholder inside a values.yaml repairs itself, because there it
sits under an `image:` key the classifier recognises: the difference is
the file shape, not the placeholder.

The gap is latent rather than new. 17 of the 19 .tag files in the tree
carry @sha256: and pass on that branch alone; the only other digest-less
one, extra/seaweedfs/images/seaweedfs-cosi-driver.tag, is third-party,
so build-main never rebuilds it, the artifact copy is byte-identical and
the classifier is never reached. The first digest-less first-party .tag
is what turns it live.

hack/promote-rewrite-tags.sh already carries a second match branch for
this shape and states why. This is that branch, scoped to *.tag so an
unkeyed line in a values.yaml still counts as drift.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-21 14:42:41 +05:00
Aleksei Sviridkin d2fcf41fb8 docs(agents): drop redis-operator from the build-target violations
The root Makefile build: list now carries the redis-operator image, so
the package stamps a reference from inside the list every other producer
iterates, and the image-refs rule no longer names it as a live violation.
flux-plunger, keycloak-operator and kilo still are.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-02 17:56:06 +03:00
Myasnikov Daniil a37a3178e5 docs(release): name all four sourcers of the image-ref enumeration
The library header still claimed three call sites. There are four:
this PR made hack/verify-promoted-packages.sh source it to compare
the candidate's refs against the rc artifact's. The historical
drift story stays with the original three, since the verifier has
only ever read the shared enumeration.

The consumer table gained hack/promote-packages-artifact.sh under a
bullet requiring every consumer of references to read all three
shapes through the library, while the exception below it still named
only hack/overlay-main-images.sh. The publisher sources nothing and
needs nothing: it hands the whole tree to `flux push artifact` and
touches one reference, the installer's own self-pin, by key with yq.
Say that, rather than implying a gap it does not have -- the real
exception remains overlay-main-images.sh, which does read references
and does walk the tree itself.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-14 18:37:59 +05:00
Myasnikov Daniil f21d277eba fix(release): retain live promotion candidates
Protect package candidates referenced by open release PRs while pruning abandoned run-specific promotion artifacts. Document the stable-candidate lifecycle and maintenance-branch rollout requirement.

Assisted-By: Claude <noreply@anthropic.com>

Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-14 18:37:57 +05:00
Aleksei Sviridkin 7739d1e294 feat(multus)!: stage reference CNI plugins from the multus image
Multus-backed NetworkAttachmentDefinitions delegate to the upstream
containernetworking reference plugins (bridge, macvlan, ipvlan, ...),
and nothing guarantees a node has them in the directory Multus
searches.  On k3s they live under /var/lib/rancher/k3s/data/cni while
Multus and Cilium install into /opt/cni/bin, so a bridge-type NAD fails
there with `failed to find plugin "bridge" in path [/opt/cni/bin]`.

Bundle 14 of the release's plugins into the multus image and add an
install-cni-plugins init container that installs them into the host
/opt/cni/bin, mirroring the existing install-multus-binary container.
loopback, dhcp, tap and dummy are left out; the Dockerfile records why.

Gate it on a new chart value, stageCniPlugins, resolved per bundle and
on for both.  Talos bundles only bridge, firewall, flannel, host-local,
loopback and portmap, so the other ten are missing there exactly as on
generic Linux; the four that overlap are rewritten with the same
upstream version, v1.9.1 on both sides, rather than an older one.  A
marker in the image Dockerfile records the Talos release that pin was
compared against, and a test holds it to the release the platform
ships.  multus therefore moves out of system.common-packages into the
two variant branches, the way linstor already does for talos.enabled.
networking.stageCniPlugins overrides the variant default in either
direction.

Where staging is on the install is unconditional: a plugin is replaced
on every pod recreation, not only when the destination is missing.  A
destination that is already a directory is reported and left alone,
since replacing it is not something this can do safely.  Turning the
value off afterwards does not undo it; nothing here removes or
restores a plugin.

The container is not privileged and takes no mount propagation.  It
runs in the spc_t SELinux domain, without which an enforcing node
refuses the write.  Each plugin is installed under a temporary name and
swapped in with a rename: /opt/cni/bin is shared with cilium and
kube-ovn.  The plugin tarball is checksummed before extraction.

A plugin that cannot be installed is reported on stderr and to
/dev/termination-log, and the container still exits 0: Multus stays the
node's primary CNI and nothing removes its config, so an init container
that never completes would leave the shim with no daemon behind it.

The manifest change lives in customize-deployment.patch so it survives
make update.

BREAKING CHANGE: enabling stageCniPlugins replaces reference CNI
plugins in /opt/cni/bin on existing nodes, Talos included -- four of the
names it writes are ones the Talos node image also carries.  Opt out
before upgrading: declining afterwards does not restore what was
replaced.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-08-05 14:19:47 +03:00
Andrei Kvapil 6cb579338b fix(nightly): rewrite an image host that is the whole scalar value
The nightly host rewrite is a literal "<src-registry>/" substring
replace, so it only reaches a host sitting contiguously in front of the
repository. kubeovn keeps its host in global.registry.address with the
bare repository in a sibling key: no trailing slash, no match. The
image was mirrored to the public registry and the tree kept pointing at
the private build registry.

The gap was dormant while kubeovn's image came from a separate
repository, because the baked ref carried the public host and the
ownership filter skipped it. Building the image in-tree stamps the
build-registry host into that key and turns it live, so fix it here.

Add an end-of-line-anchored expression that fires only when the source
registry is the complete value; a contiguous ref continues past the
host, so the two expressions cannot both fire on one reference.

keycloak-operator splits at a different boundary and stays unfixed --
neither of its keys holds the source registry as a whole. Documented,
and dormant while no CI path rebuilds it.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-28 18:07:34 +02:00
Myasnikov Daniil 1f302d0d78 docs(agents): correct issue ref, mirror claim and example in image-refs
The build-list violations bullet pointed at #3143, which tracks the
adjacent gap (packages in the list going stale between releases) and
never covers packages absent from the list; they are now tracked in
the newly filed #3416. The nightly split-host paragraph claimed both
affected images are mirrored while the published tree points at the
build registry — today neither ref carries the source-registry host
(keycloak-operator is never rebuilt by CI, kubeovn is built externally
by cozystack/kubeovn-chart), so the ownership filter skips them and
the gap is latent, not live; reworded to say exactly that. The opening
example paired a v1.6.0 tag with the real v1.5.0 grafana digest — a
ref that will never exist — and grafana is slated to stop being built;
replaced with the literal committed cozystack-api ref.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-22 12:23:29 +05:00
Myasnikov Daniil 9bd8c17971 fix(ci): shape-filter the promote postcondition and close test gaps
Review of the previous commit found the postcondition itself could abort
a release. It grepped the bare "X.Y.Z-rc.N" substring across the tree, and
three live files carry such strings without being image refs:
kubevirt-csi-driver's go.sum pins github.com/golang/protobuf v1.4.0-rc.2
-- a version cozystack actually cut, so promoting v1.4.0 would have
failed on a Go checksum line -- metallb_test.yaml names v1.5.0-rc.2 in a
comment, and the console pnpm-lock.yaml carries rolldown@1.0.0-rc.15. The
remediation the error message gave would have made things worse: declaring
go.sum as a ref source would sed a checksum file.

Match the version only in image-reference position now: after an
image/repository/tag key with no intervening '#', or immediately before an
@sha256 digest. Narrowing it to "${RC}@sha256:" alone would have been
wrong -- cilium keeps `tag: v1.5.0` and `digest:` under separate keys, so
the version is not always adjacent to the digest. Verified against four
real rc versions that previously aborted.

The suite also only tested the fail-closed direction, which is why that
defect survived, and the collector had no completeness guard at all: a
first-party ref added at an unenumerated path on a `latest@` tag was
mirrored by nothing and passed all eighteen tests. Since the mirror's
failure mode is the worst of the three -- the published nightly points at
the private build registry -- that gap mattered most where coverage was
thinnest.

Added: a fail-open test using the three real rc versions above; a
collector completeness diff that discovers refs by content and fails on
any the enumeration misses; coverage for the OCI-artifact shape, whose
deletion previously left every suite green; and an unreadable-file test,
since a read error was being treated as "no match" and silently skipped.

The component-versioned test was vacuous -- its rc version shared no
substring with the kamaji tag it claimed to protect, so it passed by
construction. It now uses an rc version sharing kamaji's 0.19.0 prefix,
which a loosely-anchored implementation would corrupt.

Round-trip fixture discovery matched a combined 'ghcr.io/cozystack/
cozystack/' prefix, which skips keycloak-operator (host in a sibling
registry: key) and kubeovn (host in global.registry.address). It now
matches 'cozystack/cozystack' and stamps split `tag:` keys too.

Also documented two pre-existing gaps rather than half-fixing them: the
nightly host rewrite is a contiguous-substring replace and cannot reach a
split host, and a ref inside a gzip is invisible to every tool including
the postcondition, which skips binaries. Both are host-dimension defects;
the version dimension this change owns is unaffected, because the version
always lives in a tag key regardless of where the host sits.

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 20:42:28 +05:00
Myasnikov Daniil 09323fff13 fix(ci): cover every image-ref storage shape in promote, retag and mirror
Promotion rewrote the rc version substring across the depth-2 package
values.yaml plus packages/apps/kubernetes/images/*.tag alone, on the
premise that the kubernetes app was the only one whose .tag files carry
the cozystack version. Nine other .tag files do, and system/multus seds
its ref straight into a vendored upstream manifest. v1.6.0 was therefore
staged with 33 of 55 first-party refs still reading v1.6.0-rc.4.

promote-retag.sh and nightly-mirror.sh had the same blind spot from the
other direction: both scanned only the depth-2 values.yaml, so those
images were never retagged to the stable version (30 refs selected
before, 42 after) and never mirrored to the public registry for a
nightly. A retag miss merely leaves an image short of a tag, since the
digest still resolves inside one registry; a mirror miss is worse,
because the host rewrite walks the same file list and a ref that is
neither mirrored nor rewritten leaves the published tree pointing at the
private build registry.

The three call sites each carried their own idea of where a ref can
live, which is how they drifted apart. Move that knowledge into
hack/lib/image-refs.sh as the single enumeration -- image_ref_files()
for the files, collect_image_refs() for the refs inside them -- and have
all three source it, so a new storage location is declared once and
reaches every consumer.

The rewrite itself moves out of promote-rc.yaml into
hack/promote-rewrite-tags.sh. Being workflow-inline is why this was only
observable by cutting a release: there was nothing to unit test. It now
also asserts its own postcondition, scanning wider than it rewrites and
failing the promotion when any ref survives in a location the
enumeration does not know about, rather than shipping it quietly.

Component-versioned images (kamaji's v0.19.0-cozystack.N) and
third-party pass-through images do not ride the cozystack version line
and are deliberately left alone; a test pins that so a future fix here
cannot over-reach. capi-providers-cpprovider is a documented known gap:
its ref is also embedded in a gzipped copy that the chart actually
ships, and rewriting only the readable copy would diverge the two.

docs/agents/image-refs.md records the contract -- three tag classes,
three storage shapes, the invariants, and which consumer does what.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 20:14:24 +05:00