The arm64 images are built on native arm64 runners, apart from the
amd64 build, so each build pushes a single-platform image. The stitch
joins every first-party image with its -arm64 twin into one index,
moves every tag this build pushed for it to that index, and repins
the tree on the index digest, so one ref serves both architectures.
Most refs in the tree name only the build's IMAGE_TAG, while under
PUBLISH_VERSIONED=1 the build also pushes the component version, so
the refs alone cannot say which tags to move. The image-tags macro
therefore appends every tag it pushes to PUSHED_TAGS_LOG when that is
set, and the stitch moves exactly those, plus IMAGE_TAG and the tag a
ref carries. It reads no other tag: another build's tag that sits on
the same amd64 bytes is not this build's to move, and scanning every
tag of every repository would cost thousands of registry reads per
release.
It reads refs through hack/lib/image-refs.sh like every other
consumer, rewrites only the files that enumeration names, and then
searches the whole tree outside charts/, gzip payloads included, for
any old digest it re-pointed. A ref whose digest is also pinned where
the rewrite cannot reach is skipped instead of half rewritten.
The kamaji control-plane provider's chart ships its ref only inside
files/components.gz, built from a plain copy with gzip -n, which no
text search sees. The stitch rewrites both copies, recompressing with
gzip -n so the result is what the package's image target would have
written, rather than leaving that image amd64 only.
An image with no arm64 twin (talos, the e2e sandbox), or a pair that
is not one amd64 and one arm64 image, is skipped and named. A
registry read answered with a rate limit or a 5xx is retried a
bounded number of times; any other registry error fails the run at
once rather than reading as a missing arm64 image.
On a rerun after a stitch that failed partway, the build tag can
already sit on an index of the pinned image and an arm64 image, not
necessarily this run's if the arm64 leg rebuilt it. The stitch repins
on that index instead of skipping the image, and copies it to any of
the build's other tags still left on the amd64 image. A tag moved to
anything else fails the run, since the tree then no longer matches
the registry.
MATRIX_ARCH=arm64 makes hack/build-matrix.sh leave out the e2e
sandbox, which pins linux/amd64: an arm64 leg building it would push
a second amd64 image under its -arm64 tag.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Classify the rename-blind changed-path list against direct Talos
sources and shared build inputs. This lets same-repository pull
requests skip the heavy build while removals, renames, and build
configuration changes still select it. Fork builds remain
unconditional.
Preserve the matchbox digest produced by a successful Talos build when
finalize overlays base-branch image references. When the job is
skipped, bootbox continues to receive the current base reference.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The matrix selector's full_rebuild_pattern omitted the root Go module dirs
(api/, cmd/, internal/, pkg/). Eight matrix image Dockerfiles COPY those dirs,
so a PR changing only root Go source emitted an empty matrix, skipped the build
job, and let e2e install stale digests — the Go change was never exercised. Add
the four dirs to the pattern and pin it with bats cases for api/- and pkg/-only
diffs forcing the full matrix.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replaces the single serial Build job (one ephemeral VM running `make
build`, ~26 images one after another, wedging on the shared buildkit
under concurrent load — #2937) with a fan-out:
plan changed files + hack/build-matrix.sh -> scoped matrix
checks unit + controller Go tests (decoupled from e2e)
build one package per job, own docker-container buildx,
uploads a per-package digest patch fragment
build-talos talos image + nocloud disk (its own leg; e2e needs it)
finalize apply all fragments, then build the installer LAST (its
flux-push bundles the whole patched tree) and emit the
unified pr.patch e2e consumes unchanged
build-matrix.sh scopes the matrix to the packages a PR touches (full set
on shared-dependency changes) and fans seaweedfs out to
objectstorage-controller. common-envs.mk gains an SBOM=1 flag wiring
buildx --sbom, kept OFF until the OCIR referrers probe confirms support.
.github/actionlint.yaml registers the custom ephemeral runner labels.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
hack/build-matrix.sh emits the CI build matrix (the list of package dirs
to build) from the root Makefile `build:` target — the single source of
truth — so the matrix can't drift from `make build`. It supports
diff-driven scoping: only units whose package dir changed are emitted,
unless a shared-dependency path (cozy-lib, the build macros, go.mod, the
build workflows) is touched, in which case the full matrix is emitted.
talos and installer are excluded from the parallel matrix: talos shares
heavy _out/assets between its image and the nocloud disk (built once in a
dedicated leg), and installer's `flux push artifact --path=packages`
bundles the whole digest-patched tree, so it must run in finalize after
all other fragments merge.
hack/build-matrix_test.bats covers full/scoped/empty selection, the
talos+installer exclusion, shared-dep escalation, and JSON/Makefile parity.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin