Commit Graph
7 Commits
Author SHA1 Message Date
Myasnikov Daniil cc942fed68 test(tests): run unit suites with Bats and retain POSIX coverage
The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.

Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 11:04:00 +03:00
Aleksei Sviridkin 6df5d076b1 ci(workflows): pin flux, yq and helm-unittest installs
Every flux install fetched fluxcd.io/install.sh and ran it as root, so
whatever that host served at the moment executed on runners that later
hold registry credentials. Use the upstream setup action instead, pinned
by commit and given an explicit version: it checks the release tarball
against that release's checksums file and runs no script fetched at run
time.

nightly and the fork e2e lane still took yq from the floating latest
release. They now use the pinned, checksum-verified install the
promotion steps already have.

The helm-unittest plugin embeds its own Helm and was installed at
whatever version was newest, so a plugin release could change chart
test results on a PR that touched no chart. Pin it, and replace a
different version a persistent runner may carry.

A contract test scans every workflow for these three tools, so an
unpinned flux, yq or helm-unittest install copied from an older step
fails the unit lane.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 11:02:18 +03:00
Myasnikov Daniil b65b6aa3ae test(release): pin every requiredBaseFiles entry, not just the list
The promote gate refuses a target base that lacks any of five
[path, marker] pairs, and the contract test asserted the list as a
block: deleting the whole thing turned it red, deleting the
hack/lib/image-refs.sh entry alone did not. Four of five pairs were
therefore unpinned individually.

Pin the missing one. The verifier sources that library through the
same `dirname $0` as lib/promoted-packages.sh, so a base carrying
the script without it fails at exactly the same point -- after the
candidate, the staging branch and the draft release exist.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-14 18:37:59 +05:00
Myasnikov Daniil 05b2ce90cf fix(release): close the verifier's fail-open paths and cover them
Three gaps in the guards that decide whether the published candidate is
still the tree being tagged. All three passed silently rather than
failing, which is the worst shape for a check that runs immediately
before a write-once name is created.

`normalized_refs` piped `collect_image_refs` straight into its parsing
loop, so the pipeline reported `sort -u`'s status and the collector's
was discarded. POSIX sh has no `pipefail` and this script has to stay
POSIX, so `set -e` never saw it: a failed or truncated collection became
a smaller set and the rc-vs-candidate comparison matched it happily. The
verifier then reported that no container bytes changed across promotion
having proven nothing — on the guard that carries that entire claim.
Collect into a file first, so a nonzero collector aborts before anything
is compared. An empty collection is the same failure wearing a zero exit
status and needs its own check: two empty sets compare equal, so the
proof passes having examined nothing, and every packages tree carries
image references — none means the collector understood nothing it was
handed.

The rc-leftovers scan swallowed grep's stderr, turning "could not read
this file" into "this file is clean" — the silent-skip shape
hack/promote-rewrite-tags.sh refuses by name. The `|| true` has to stay,
because `-exec … +` reports the legitimate "nothing matched" as a
failure too, so the diagnostics are the only remaining signal that a
file went unscanned. Fail on any of them. Defence in depth rather than a
live hole: a surviving rc string would also fail the per-file compare,
since the release tree has none.

The per-file content comparison had no test at all. It is the only leg
that catches a file present on BOTH sides with different bytes — exactly
the threat the job exists for, a maintainer editing packages/ on the
release PR after promotion published the candidate. The file-set compare
sees the same names, the installer compare reads one file, and the
rc-refs compare never looks at the release tree. Replacing the
comparison with a no-op left all nine other tests green.

Red phases, all observed before the fixes. The content-compare case
changes one non-image line on one side, so no other leg can fire and
claim the credit; it fails with the comparison no-op'd and passes with
it restored. The two collector cases run the verifier from a copy whose
sibling lib/ holds a stub — which is how the workflows already invoke
it, `.release-tooling/hack/…` with libraries resolved next to the
script, so the failure is reached the way production would reach it.
Against the piped version both fail on `[ "$rc" -ne 0 ]`: the verifier
exits 0 and prints its success line whether the collector returns 3 or
returns nothing.

Also pins `e2e`'s dependency on the candidate gate. The contract pinned
`e2e-report`'s, so removing `verify-release-candidate` from `e2e`'s
`needs` together with the `success` clause on its release arm left all
six tests green — and running a full-e2e promotion suite against a
candidate that never verified is the worse of the two failures.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
2026-08-14 18:37:58 +05:00
Myasnikov Daniil 9fa8eeeda2 ci(release): pin the promotion toolchain and cap the candidate gate
Three review findings on the new promotion steps.

`yq` was fetched from `releases/latest/download` in all three steps this
pipeline installs it in — the promote dispatch, the promote PR gate and
finalize. That makes an unpinned moving dependency part of what decides
whether a stable release may be created, and part of what rewrites the
installer pin: two promotions of the same rc can be served different
binaries. Pin it the way FLUX_VERSION already is, version-gated so a
matching preinstalled yq is reused, on the version the e2e sandbox image
already tracks.

A version authenticates nothing about the bytes it names, though, and
these are the workflows that mint write-once stable names. yq is
therefore also pinned BY CONTENT: its SHA-256 is committed beside its
version and verified before install.

Scoped to yq deliberately. It is the one downloaded binary with write
authority over what a release contains — it stamps the candidate digest
into installer values at promote time and platformVersion into the
stable chart at finalize. skopeo, helm and oras only move bytes that are
already digest-addressed, so a substituted one is caught downstream; a
substituted yq writes the wrong pin and every later check verifies that
pin faithfully.

The Flux installer is left as it is, on purpose. It fetches
flux_${FLUX_VERSION}_checksums.txt from the pinned release and verifies
the binary against it, so the binary is not the exposure — the ~6 KB
bootstrap script from fluxcd.io is, and it is mutable by design and
fetched identically in ten other workflows here. Pinning its bytes in
these three files would break on the next regeneration of a script the
project does not control while leaving the same fetch unpinned
everywhere else. The comment names the residual risk instead.

No precedent existed either way: nothing under .github/workflows/ or
hack/ verified a downloaded tool before this, and every version-pinned
download in the tree (oras 1.2.0, cozyvalues-gen 1.6.0) is unverified.
This is a new convention rather than an application of one, confined to
the release path and justified by write authority, and the comment says
so where the next reader will be standing. The three unpinned fetches
outside this pipeline (nightly.yaml, e2e-fork.yaml) are left alone.

The checksum is trust-on-first-use, which is the honest description: it
fixes the bytes to what was checked when the version was chosen and
makes any later substitution loud. Provenance is the release's own
published checksums list cross-checked against the downloaded asset;
both agree on
fa52a4e758c63d38299163fbdd1edfb4c4963247918bf9c1c5d31d84789eded4, and
that binary self-reports v4.53.3.

Verified locally against the real asset: the version gate skips a
matching preinstalled yq and re-installs a mismatched one, the checksum
line reports OK for the genuine binary and exits 1 for a tampered copy,
which aborts the step under the runner's `bash -e`.

`verify-release-candidate` had no `timeout-minutes`, so it inherited the
6-hour default while `e2e-report` waits on it through `needs` — a
stalled `flux pull artifact` would hold the required `E2E Tests` status
pending for the whole of it. 30 minutes, matching `checks` and
`finalize`.

The contract pins that all three install steps carry both versions and
the checksum, run the checksum check, name no `latest/download`, and
agree on one yq version and one checksum. So the unpinned one-line idiom
cannot come back into a release step by copy-paste from the workflows
that still use it, and bumping the version in one file without its
checksum cannot pass. Both mutations are caught.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
2026-08-14 18:37:58 +05:00
Myasnikov Daniil 2d734fb229 fix(release): preflight the rc tree's own pin before publishing
hack/promote-packages-artifact.sh pushes the candidate BEFORE rewriting
installer values, so the artifact carries the rc tree's
platformSourceUrl/platformSourceRef verbatim — and
hack/verify-promoted-packages.sh reads exactly those two fields back out
of it as the candidate's rc baseline, requiring a trusted repository and
an immutable `digest=sha256:<64 hex>` pin.

The publisher checked neither. A tag-form ref or an untrusted URL in the
rc tree published a candidate and pushed two commits regardless; the
refusal arrived later, in the promote PR gate or in finalize, once the
staging branch and the draft release already existed. Check both at
dispatch time instead, while nothing has been written.

A copied regex would drift from the guard it is bringing forward and
quietly restore the late failure, so the trusted repository and the
digest pattern move into hack/lib/promoted-packages.sh and both halves
source it. That makes the library part of what the verifier needs at run
time, so promote's base-branch precondition gains it: the list exists
because the base's own workflows run after merge, and a base carrying
the verifier without its library fails at the same two late points.

Tests: both preflights are covered, each asserting the mock registry
client was never invoked — "exited non-zero" alone does not say nothing
was written. Both fail with the preflight removed. The publisher
fixtures now pin the trusted repository the way a real rc tree does, and
still send the candidate elsewhere via REGISTRY so the rewrite
assertions keep their teeth.

Also covers a malformed PROMOTION_ID. It is not attacker-influenced, but
it is the run-uniqueness suffix of the candidate tag and retention.yaml
deletes a candidate only when every tag on the manifest matches
`promotion-v…-run-[0-9]+-[0-9]+`; a value outside that shape publishes
an artifact no cleanup pass can ever collect.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-14 18:37:58 +05:00
Myasnikov Daniil c2e145709b fix(release): publish stable packages candidate
Serialize the tag-rewritten packages tree during RC promotion and pin its immutable digest in the installer. Verify the candidate against the prospective merge and the original RC artifact before creating stable names.

Assisted-By: Claude <noreply@anthropic.com>

Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-14 18:37:57 +05:00