The dockerfile manager does not read Makefiles, so the digest-pinned
scanner would stay on the syft it was cut with, where the floating tag
it replaced moved on its own. A regex manager bumps the tag and the
digest together, and a test holds its regexp to the SBOM_SCANNER line,
so a reformatted pin cannot drop out of it unnoticed.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-by: LLM
A release said nothing about what is inside its container images, so a
user could not tell which packages a cluster runs. SBOM=1 had been
plumbed for it and left off.
BuildKit attaches the SBOM as an attestation, which makes the pushed
image an index of the image and its attestation manifest. The stitch took
such an index for something other than one amd64 and one arm64 image,
skipped it and still exited 0, and the multi-arch gate then failed the
release, so the stitch now reads through the index and the stitched one
keeps both attestations. The release build ran on the default docker
driver, which cannot attach attestations, so it moves to the
docker-container builder with the docker.io mirror, as the arm64 leg
already did. The scanner is pinned by digest where --sbom=true pulls a
floating tag.
The Talos installer image is built by the imager and pushed by skopeo, so
it carries no attestation.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.
Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
A local build that asks to load images into docker no longer pushes
them. `hack/common-envs.mk` set `PUSH` and `LOAD` with plain `:=`, and a
makefile assignment beats the environment, so `LOAD=1 PUSH=0 make image`
quietly pushed a multi-arch build. Both now take the environment value,
and an empty one falls back to the default. I also made `PUSH` default
to 0 when `LOAD=1`, so `LOAD=1` alone only loads. Before this, `make
image LOAD=1` pushed and loaded at once. An explicit `PUSH` from the
environment or the command line still wins, and `OCI_EXPORT_DIR` still
forces both off. Every buildx target gets the values through
`BUILDX_ARGS`, so the fix lives in that one file.
The Talos installer was the one image that ignored `PUSH`. It is
published with skopeo, not buildx, so it never saw `--push=$(PUSH)`. Its
registry copies now run only when `PUSH=1`.
A build that pushes nothing no longer writes `PUSHED_TAGS_LOG`. That log
tells the multi-arch stitch which tags to move, so it must hold only
tags that were really pushed.
The cache helper lost an argument nobody used. No caller passes the
second argument of `cache-args`, and an explicit tag there would bypass
`CACHE_TAG`, so the amd64 and arm64 builds of that image would share one
cache ref.
CI sets neither `PUSH` nor `LOAD`, so CI builds do the same thing as
before. Each fix has a `make --dry-run` test in
`hack/common-envs_test.bats`.
Closes#4499Closes#4504Closes#4551
### Screenshots
Not a UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
The ccp skills pass `PUSH=1` explicitly, which still works. The website
docs do not mention `PUSH`, `LOAD` or `cache-args`.
### Release note
```release-note
fix(build): `LOAD` and `PUSH` set in the environment are now honoured, `LOAD=1` no longer pushes unless `PUSH=1` is also set, and the Talos installer is no longer published when `PUSH=0`.
```
Six suites explain why they avoid `! cmd` by saying errexit is
suppressed for a "negated pipeline". The exemption is not about
pipelines: a bare `! true`, a negated `[ ... ]`, one inside a `for`
body or a brace group, one in a case branch, and one following `;`,
`&`, `&&` or `||` are all exempt too. Anyone reading the narrower
wording could conclude the other shapes are safe.
One of the six carries the sentence twice, and the second copy is
split across two physical lines -- `` `!` `` ending one and
`pipeline)` opening the next -- so a line-oriented search for the
phrase does not see it. Prose wraps; a grep for it has to fold the
lines first.
Comment text only; every one of these files already uses the working
form the comment recommends.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
PUSHED_TAGS_LOG fed every tag to hack/stitch-multiarch.sh whether or
not the build pushed it, so a PUSH=0 build, and now a plain LOAD=1
build, handed the stitch tags that were never published.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
image-talos publishes with skopeo instead of buildx, so it never saw
--push=$(PUSH) and a local PUSH=0 build still overwrote talos tags in
REGISTRY.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A plain := assignment beats an environment variable, so
`LOAD=1 PUSH=0 make image` pushed a multi-arch build instead of
loading one. An empty value from the environment falls back to the
default rather than reaching buildx as `--push=`. PUSH now also
defaults to 0 under LOAD=1, so a local load never publishes unless
PUSH=1 is asked for. The OCI_EXPORT_DIR block still overrides an
environment PUSH or LOAD.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
No caller passes the second argument any more, and an explicit tag
would bypass CACHE_TAG, so the amd64 and arm64 legs of such an image
would write the same mode=max cache ref and evict each other's layers.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The capi-providers-cpprovider Makefile assigned IMAGE_TAG itself, which
beats the value CI puts in the environment. Every PR, main and release
line build therefore pushed the same component-version tag and
overwrote each other's, and its second tag was built from an undefined
TAG variable and ended in a bare "-".
The image now goes through image-tags like every other package: the
build-unique IMAGE_TAG always, the component version only under
PUBLISH_VERSIONED=1. The stamped refs keep the component version as
their tag, so the promote, retag and mirror tooling see the same refs
as before.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The arm64 images are built on native arm64 runners, apart from the
amd64 build, so each build pushes a single-platform image. The stitch
joins every first-party image with its -arm64 twin into one index,
moves every tag this build pushed for it to that index, and repins
the tree on the index digest, so one ref serves both architectures.
Most refs in the tree name only the build's IMAGE_TAG, while under
PUBLISH_VERSIONED=1 the build also pushes the component version, so
the refs alone cannot say which tags to move. The image-tags macro
therefore appends every tag it pushes to PUSHED_TAGS_LOG when that is
set, and the stitch moves exactly those, plus IMAGE_TAG and the tag a
ref carries. It reads no other tag: another build's tag that sits on
the same amd64 bytes is not this build's to move, and scanning every
tag of every repository would cost thousands of registry reads per
release.
It reads refs through hack/lib/image-refs.sh like every other
consumer, rewrites only the files that enumeration names, and then
searches the whole tree outside charts/, gzip payloads included, for
any old digest it re-pointed. A ref whose digest is also pinned where
the rewrite cannot reach is skipped instead of half rewritten.
The kamaji control-plane provider's chart ships its ref only inside
files/components.gz, built from a plain copy with gzip -n, which no
text search sees. The stitch rewrites both copies, recompressing with
gzip -n so the result is what the package's image target would have
written, rather than leaving that image amd64 only.
An image with no arm64 twin (talos, the e2e sandbox), or a pair that
is not one amd64 and one arm64 image, is skipped and named. A
registry read answered with a rate limit or a 5xx is retried a
bounded number of times; any other registry error fails the run at
once rather than reading as a missing arm64 image.
On a rerun after a stitch that failed partway, the build tag can
already sit on an index of the pinned image and an arm64 image, not
necessarily this run's if the arm64 leg rebuilt it. The stitch repins
on that index instead of skipping the image, and copies it to any of
the build's other tags still left on the amd64 image. A tag moved to
anything else fails the run, since the tree then no longer matches
the registry.
MATRIX_ARCH=arm64 makes hack/build-matrix.sh leave out the e2e
sandbox, which pins linux/amd64: an arm64 leg building it would push
a second amd64 image under its -arm64 tag.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A native arm64 leg builds the same images next to the amd64 build and
must not overwrite the cache that build owns. CACHE_TAG moves the
default cache ref, so the leg can write buildcache-arm64 without every
write evicting the amd64 cache layers, and without a change to each
package Makefile. An explicit per-call cache tag still wins.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The package pinned linux/amd64 because its assets were amd64 only.
With arm64 profiles it follows PLATFORM like every other image.
The installer is published as one index under every tag. skopeo
cannot assemble an index from the imager's docker archives, so the
archives go into one OCI layout and a small script writes the index
there for skopeo to push. buildx could build it from OCI layout
sources, but the release job builds on the classic docker driver,
which has no such source.
matchbox carries the kernel and initramfs of both arches whatever
PLATFORM is, rather than those of TARGETARCH: the node matchbox runs
on is not the arch of the machines it boots, so a per-arch image would
hand an amd64 machine an arm64 kernel whenever the pod lands on an
arm64 node. The amd64 assets stay at their old paths, and a group
keyed on the arch iPXE reports sends arm64 machines to the arm64 ones.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Assisted-by: LLM
PLATFORM defaulted to empty, so buildx built for the host alone. On an
arm64 workstation `make image` published an arm64-only digest, which
amd64 nodes then fail to pull; the cilium pin shipped exactly that
until it was rebuilt by hand. A build now targets linux/amd64 and
linux/arm64 unless PLATFORM is set, and the digest a package stamps is
the index digest.
LOAD=1 keeps building for the host only. The classic docker image store
cannot load a multi-platform index, and loading is the documented local
workflow.
This changes local pushes: a `make image` on the default docker driver
without the containerd image store now fails with "Multi-platform build
is not supported for the docker driver", where it used to build for
the host. Such a build needs a docker-container builder (BUILDER), or
LOAD=1 or an explicit PLATFORM for a single architecture.
talos and testing pin linux/amd64 with `override`, placed before the
include because common-envs.mk expands BUILDX_ARGS immediately. The
Talos assets are amd64: matchbox bakes the kernel and initramfs into
its image, and the e2e sandbox boots the disk under qemu-system-x86_64.
An arm64 variant of either would still serve or boot amd64 files.
CI pins linux/amd64 on every image build. Nothing there sets up QEMU,
fork builds export a single-platform archive, and tags.yaml builds on
the default docker driver, so the default would break or emulate those
jobs. Native arm64 builds belong in a leg of their own.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from IvanHunters on hack/common-envs_test.bats:
every existing case set OCI_EXPORT_DIR, so the OFF path — the one every
same-repo PR, main and release build still takes — had no regression
test at all. A refactor of the export wiring could stop pushing entirely,
or strip the release tags, and the suite would stay green.
Add a case asserting that with OCI_EXPORT_DIR unset buildx still pushes
(--push=1) and writes no archive, image-tags expands all three release
tags, and talos' skopeo copies still target docker:// with the
`[ -z "$(OCI_EXPORT_DIR)" ]` release gates inert.
Mutation-checked: making the PUSH/LOAD override unconditional (so export
mode leaks into the default path) fails this case.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from IvanHunters on hack/common-envs_test.bats:70:
`grep -c` counts matching LINES, not occurrences. The image-tags macro
expands onto a single line, so every "exactly one tag survives under
OCI_EXPORT_DIR" assertion returned 1 no matter how many --tag flags were
actually emitted.
Proven vacuous by mutation: removing the OCI_EXPORT_DIR guard from
image-tags restores three --tag flags on that one line and the suite
stayed green. Switching to `grep -o … | wc -l` counts occurrences, and
the same mutation now fails the suite as it should.
Applied to the --output type=oci,dest= assertions too, which count
occurrences the same way and had the same blind spot.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Belt-and-suspenders for the B2-class trap lexfrei flagged as latent in
image-tags. PUBLISH_* are 0 on fork PRs, so today the export path never emits a
versioned/floating tag — but if it ever did:
- image-tags: multiple --tag under `--output type=oci` write >1 manifest into
the archive's index.json, and `skopeo copy oci-archive:<f>` then refuses it
("more than one image in oci"). Suppress the versioned/floating tags when
OCI_EXPORT_DIR is set (macro-level $(if …), so `make -n` prints only the
taken branch).
- packages/core/talos: gate its versioned/floating `skopeo copy docker://…`
release copies on an empty OCI_EXPORT_DIR too.
The release (non-export) path is unchanged — versioned + :latest still emitted.
Covered by a new common-envs_test.bats case that forces PUBLISH_*=1 under
export.
Reported by gemini-code-assist on #3262.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Add hack/common-envs_test.bats (runs under make bats-unit-tests). For a
representative slice of the root build: matrix — including the two recipes
that previously slipped through, capi-providers-cpprovider and core/talos —
it asserts that under OCI_EXPORT_DIR each built image emits exactly one
--output type=oci and no ungated docker:// push, and that capi emits a single
--tag. This would have caught the talos-push and capi-two-tag regressions.
Refs: #3257
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Myasnikov Daniil