The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.
Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Promotion needs the `release` label to start work, so this PR added it
to the set of labels whose `labeled` event is not discarded. That set is
stated in four places and was updated in three.
`gate-concurrency-contract.bats` caught the shape of the problem and
then became the first casualty of it. It asserted that the concurrency
key excludes exactly ONE label from the `-label` group, by counting
occurrences of `github.event.label.name != '`. A second publishing label
made the count 2, turning the suite red — and with it `checks`, then
`finalize`, then `e2e`, then the required `E2E Tests` status. Raising
the constant to 2 would re-hardcode the same brittleness one label later
and throw away the invariant the test exists for: the set of labels
excluded from the `-label` suffix is exactly the set `plan` runs for.
Both sets are now extracted from the workflow and compared, so the
assertion holds at any number of labels.
`resolve_assets` was the guard that got missed. It sits on `e2e`'s
release arm, so a label event `plan` runs for and it skips for leaves
`e2e` with both arms false — `finalize` is force-skipped for release
PRs — while `e2e-report` still runs. On a promote PR already carrying
`full-e2e`, re-applying `release` therefore superseded the green run and
posted `E2E Tests` = failure, because `full-e2e` bypasses the "e2e not
repeated" branch and `E2E_RESULT` is `skipped`. `unlabeled` is not a
trigger, so removing the label could not clear it; only a push could.
`release` must also not publish on the fork lane. e2e-fork.yaml's
`resolve` decides "this run built nothing, so it is not a verdict" by
asking whether `Plan build` skipped. Admitting `release` at `plan` made
that false: a fork run reaches `plan`, has `build`/`finalize` gated off
by its own label, succeeds having produced no pr-patch, and the fork
lane fails closed on the missing artifact — red on a suite that was
never going to run, on a PR whose only fault is a mislabel. `plan` and
`resolve_assets` now admit `release` for same-repo PRs only, and the
concurrency key carries the complementary term, so
NOT(A OR B OR (C AND NOT F)) stays exactly (NOT A) AND (NOT B) AND
(NOT C OR F). For a same-repo PR F is false and every one of them
reduces to what it was, so none of this changes non-fork behaviour.
The same shape still reaches the fork lane from a PUSH to a fork PR
somebody mislabelled `release` — `build` is gated on the label, not on
the event, so this predates promotion using the label. Left alone; the
comment in e2e-fork.yaml now records it instead of asserting an
invariant this PR would have made false.
The contract enumerated `plan` and `e2e-report` by name, which is why it
did not catch `resolve_assets`: a named-job loop cannot notice a job
nobody thought to name. It now derives the job list from the file and
requires every job that decides from a label name to decide from the
same set, and pins the fork term on both halves of the complement. The
file's existing properties are kept — extraction runs over
executable-only lines, so a commented-out key cannot satisfy the
contract, and every comparison asserts a non-empty input first, so an
empty extraction fails closed under both bats and cozytest.sh.
Verified by mutation, each caught: excluding a label `plan` skips for,
admitting one the key does not exclude, reverting `resolve_assets`,
deleting `e2e-report`'s guard, narrowing `verify-release-candidate`'s,
renaming the label in one guard only, commenting out the group key, and
dropping the fork term from either side alone. Deleting `plan`'s
step-level fork guard is caught too, by an assertion that had to stop
matching on the bare identifier once `plan`'s own `if:` named it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The check collected every comparison against the label name and required
them to agree on one string. Agreement is not presence: deleting
`e2e-report`'s guard outright leaves the other two agreeing with each
other, so the assertion stayed green while a discarded label event began
publishing again and clobbering the verdict of the run that had actually
tested the SHA. That clobber is the defect two earlier commits were
written to stop.
Require the guard inside each of the two jobs that must carry it, and
keep the agreement check for the rename it does catch. Deleting either
copy now fails.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The reference section claimed both lanes open the required context when
their run starts, and enumerated the triggers it protects against: a
re-run, a reopen, a `full-e2e` label. Every one of those starts a Pull
Request run, and on a fork that run cannot open the status at all. The
opener is guarded same-repo only, because a fork's `pull_request` token
is read-only, and the fork's opener lives in the privileged workflow,
which starts only once the unprivileged build has finished. So on the
lane the sentence was most likely to be read about, an already-green
head SHA keeps reading green for the whole build.
The gap is structural. A read-only token cannot post a status at any
point in the run, and that same read-only token is what makes publishing
the gate with the default token safe in the first place. It is bounded
by the build rather than open-ended. What was wrong was only the claim
that it does not exist.
Say so in all three places that carried the overstatement: the section
itself, the opener's own comment, and the contract test, which pins that
each lane has an opener and cannot express when it lands.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The invariant that a cancelled run publishes nothing was written down in
three places and held in two of them.
The fork lane's terminal reporter ran under `always()`, which includes
cancellation. When a second privileged run supersedes the first, the
dying one still posted `failure` on a head SHA it no longer owned, for a
suite that never finished. It self-corrected once the superseding run
reported, but that can be two hours later, and until then a legitimate
pull request reads red. The same-repo lane was moved off `always()` for
exactly this reason. Move this one too: a failed publish or e2e still
reports, since neither is a cancellation, and a cancelled run now leaves
the `pending` it opened standing until its successor concludes.
The same-repo guard's rationale cited a scenario that no longer exists.
It described a label event cancelling the run that opened the pull
request, which the concurrency split made impossible for every label the
workflow discards. The guard is still needed, for a push, for a
`full-e2e` label superseding the head run, and for a manual cancel, so
the comment now names those instead of an extinct one.
The reference section listed the pieces this invariant rests on and
omitted the reporters, which is the half a reader is most likely to
touch, and counted them, which was already wrong and would go wrong
again on the next addition. It now enumerates without counting.
The contract pins both reporters, because both lanes have had this
defect at different times in the same shape.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The two lanes answered "is this docs-only" from different sources and
could disagree. `plan` reads `git diff --name-only`, which applies rename
detection and reports a rename as its new path alone, so moving a code
file into docs/ read as docs-only. The fork lane reads the PR file list,
where a rename carries `previous_filename`, and reads the same change as
code.
Disagreement is not a near-miss here. The same-repo lane builds nothing
and uploads no patch artifact; the fork lane concludes e2e is required
and then fails closed because that artifact is missing, reporting that
the fork build did not complete. It completed — it correctly built
nothing. There is no push the contributor can make to clear it and no
label that overrides it, so a legitimate contribution is permanently
red.
Point the docs-only decision at the rename-blind list, which the job
already computes for the downstream-trigger-map flag and whose comment
names this exact trap. Both lanes now agree, and the same-repo side
stops shipping a removed code path untested — until now such a change
skipped every build and reported e2e as not required. The build matrix
keeps reading the rename-detected list: it picks packages to rebuild,
and a path that no longer exists has nothing to build.
The contract file gains the agreement as a pin, and grows a header to
match what it now covers. The fork half is pinned on the docs-only
expression rather than on `previous_filename`, which also appears in the
file list a few lines down and would have kept the assertion satisfied
with the check deleted.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The case asserting that both lanes open the required status before they
conclude it counted occurrences of the context string per file and
required two. That measured nothing on the fork lane: the two
occurrences there are the shared `setStatus` helper and the terminal
report, while the opener calls the helper and contributes no occurrence
of its own. Deleting the opener outright left all six tests green, under
a comment promising that dropping either half would be caught — and the
window it guards is the wider of the two, since a privileged run holds
the head SHA for up to three hours.
Scope every assertion to the job that owns it and pin the call rather
than a string that happens to appear twice: `plan` opens with `pending`
under a same-repo guard, `e2e-report` concludes, `resolve` opens through
the helper, `report` concludes. Removing any one of the four now fails
this test.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Moving the gate from a job to a commit status lost a property the job
had for free. A commit status is latest-wins per (sha, context) and
never expires, so a head SHA that already went green keeps reading green
from the moment a new run starts until `e2e-report` posts at the end.
Re-running all jobs, reopening, or adding full-e2e therefore leaves the
pull request mergeable for the whole of a suite that may be about to
fail. While the gate was a job, GitHub created a fresh non-success check
run each time and the window did not exist.
Claim the context at the start of `plan`, the way the fork lane already
claims it in `resolve`, and let `e2e-report` conclude it as before. The
call is same-repo only: a fork's `pull_request` token is read-only, so
posting from there would 403 and fail the job on every fork PR — and
that read-only token is what makes the whole commit-status design safe.
Fork PRs get their opening status from the privileged workflow instead.
The contract test grows a case pinning that each lane writes the context
from two points, one of them pending. Both assertions on the same-repo
opener are scoped to the `plan` job: the bare fork guard occurs on
several steps in other jobs, so a whole-file "at least one" passed with
the opener's guard deleted, pinning unrelated guards and reporting that
as coverage.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The header claims the third silent path is unreachable because every job
either carries the discarded-label guard or needs `plan`'s outputs. The
`e2e` job does neither literally: it gates on the results of `finalize`,
`build-talos` and `resolve_assets`, which are the jobs that read those
outputs. The conclusion survives — a skipped `plan` still skips it, one
hop further along — but transitivity is the part the claim rests on, and
a reader checking the stated reason against `e2e` finds it false and has
no way to tell whether the conclusion failed with it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The contract asserts that the conclusions routed into a per-run group
match the ones `resolve` posts nothing for, which reads as covering
every case where a run stays silent. It does not. `resolve` also
returns silently when the triggering run succeeded but its `Plan build`
job skipped, and such a run sits in the shared group where it can still
cancel the live publisher.
That path is unreachable today, because every job in pull-requests.yaml
either carries the discarded-label guard or needs `plan`'s outputs, so a
skipped `plan` yields a skipped run and the conclusion check catches it
first. Adding a job that runs independently of `plan` would reopen it,
and the extraction here matches on the conclusion constant and would not
notice. Say so in the header rather than leaving the next reader to
infer coverage the pin does not provide.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Six assertions ran `grep -c` against a variable holding a single line,
so `-eq 1` only asked whether the pattern appeared at all. A key naming
the same field twice — two run-id interpolations, two branch
components — satisfied the pin exactly as the correct key does.
Demonstrated on the run-id assertion: duplicating the interpolation
leaves `grep -c` reporting 1 and passing, while counting occurrences
reports 2 and fails.
Use `grep -o … | wc -l` for the single-line pins. The two whole-file
`cancel-in-progress` counts stay on `grep -c`, where counting matching
lines is the intended question. This is the same correction made once
already in the sibling contract suite; the form did not survive being
copied.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two statements that were not true of the code next to them.
The fork `publish` job accepts only archive names it can derive from the
base parent, so a fork cannot name an artifact into an arbitrary
registry repository. The consequence is that a fork PR adding a new
image fails on `image '<name>' is not in the base-tree allowlist` with
no way to pass, and nothing said so — including the E2E section that
AGENTS.md points a reader at when a fork run fails. Record the
constraint, the exact log line, and the two ways around it, so the
failure reads as the guard working rather than as a broken PR.
The concurrency contract's comment on `code_lines` claimed a real grep
error propagates out of the helper. It does under the bats binary, and
not under `hack/cozytest.sh`, whose translator appends `return 0` to
every line that is exactly `}` — file-level helpers included — so a
helper's last command never decides its status there. Both runners
still fail closed, because each test asserts its extraction is
non-empty before comparing, so a swallowed error surfaces as empty
input rather than as a pass. Say that instead.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Giving every `labeled` event its own concurrency group stopped a
discarded label from cancelling a live run, but it also stopped a
`full-e2e` label from doing so — and that label is not discarded. `plan`
proceeds for it and `e2e-report` publishes with it, so the labelled run
and the head run it was meant to widen both post the `E2E Tests`
context, on the same head SHA, with no ordering between them.
That was a deliberate trade at the time: commit statuses are latest-wins
per context, so the reasoning went, the full-suite verdict lands last
whenever its suite finishes last, and the label is an explicit opt-in to
a longer run. This commit reverses that decision, because the premise
only holds while the full run succeeds. The full suite is the superset,
so a full-e2e run that fails early — a build flake, a suite failing at
minute twenty — finishes before the narrower head run it was meant to
widen, and that run's later green erases the failure. The required
status then reads success for a SHA whose suite did not pass. One
publisher per head SHA is worth more than the minutes of runner time a
cancelled narrow run costs.
Move aside only the label events `plan` discards, by making the
concurrency condition the exact complement of `plan`'s guard. A
`full-e2e` run stays in the main group and supersedes the head run the
way a push does, so exactly one publisher of the context is alive per
head SHA. Opting into the full suite now costs one suite instead of two,
and a second label landing afterwards can no longer cancel the full-e2e
run, because the two are no longer grouped together.
The contract test gains the complement — the group key must exclude the
publishing label rather than every label — and a check that the label
name is one and the same string in all three guards that read it, since
renaming it in one place splits the decision without touching a line
that reads load-bearing.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The privileged fork e2e run groups by fork repo + branch with
cancel-in-progress, so every completed Pull Request run for that branch
lands in the same group -- including a run that built nothing.
Labelling an open fork PR produces exactly that: pull-requests.yaml
triggers on `labeled` and `plan` discards every label other than
full-e2e, so all jobs skip. The privileged run for that event cancels
the one that is mid-suite on the same head SHA, then `resolve`
recognises the discarded trigger and posts no status at all. The
required "E2E Tests" context is a commit status, so the head SHA is
left at "Expected" with nothing left to flip it -- the run that
replaced the owner is the very one that decided to stay silent -- and
the PR stays unmergeable until the contributor pushes again.
Key the group by the triggering run's id when that run concluded
`cancelled` or `skipped`, so a run that will post nothing can cancel
only itself. Real runs keep superseding each other on fork repo +
branch.
The contract test ties the conclusions in the key to the ones `resolve`
stays silent for. Those two lists sit in different languages in
different halves of the file, and a third conclusion added to the guard
alone would restore the wedge without touching a line a reviewer reads
as load-bearing.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>