The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.
Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Findings from two independent adversarial reviews of the preceding two
commits. Each was reproduced locally before being fixed.
The ordering inverted itself on a shape I had not probed. `finishedAt`
is a metav1.Time behind `json:",omitempty"`, which cannot omit a
struct, and its zero value marshals to `null`; the kubelet does
synthesize a terminated state with no time (ContainerStatusUnknown on a
pod deleted mid-run, which also bumps restartCount and so clears the
restart filter). Two guards deep left the leaf exposed, and kubectl
renders it as the literal `<no value>` -- which sorts ABOVE every digit,
so that row took the front of the queue from the most recent real
restart, and the header printed `previous instance ended <no value>`, a
claim about a time nothing observed. Verified with a text/template probe
over seven lastState shapes: `null` and a missing key both produced it.
Fixed at both layers, because they fail independently. The template now
guards the leaf, so the string cannot be produced. And the sort
normalises any field-6 value not shaped like a timestamp to empty, so
the ordering stays correct and the header stays quiet even if some other
producer emits one. Earlier verification against a live cluster showed
no `<no value>` and I read that as the guard being right; it only meant
no container on that cluster was in that state.
Three guards did not bite:
- both stability fixtures were pre-sorted ascending, which is exactly
what sort's last-resort whole-line compare reproduces with no -s at
all, so the tests written for -s passed without it. Reversed; they
now fail when -s is dropped.
- no test fed a six-field row through the whole script, so the
timestamp reached the dump header uncovered -- deleting the read
variable left the suite green at 39/39. One integration test now
asserts the header names the time.
- the e2e guard matched any mention of COZYTEST_TRACE, so documenting
the design over there would have turned it red. It now matches an
assignment, and tolerating a comment is verified too.
Also: a failed sort no longer discards a whole namespace group. The
caller compares row counts to report the cap, so losing a group silently
became "0 of N captured, N more not captured" and sent the reader after
a cap that never fired. It falls back to discovery order and says so.
LC_ALL=C on the sort, since the key is fixed-format ASCII.
And the claim itself was too strong. "Nothing diagnostic is lost" holds
only for a test that RETURNS: the fail handler runs after the pipeline,
and there is no INT/TERM trap, so a test killed rather than failed -- a
hang reaped by the job timeout -- never dumps its log, and under quiet
mode its trace is gone rather than already printed. The comments in
hack/cozytest.sh and the Makefile now say so and name the escape hatch.
A trap that flushed the tail of the log would close it; left out
deliberately rather than bundled in here.
Every fix is mutation-proven: reverting the leaf guard, removing the
key normalisation, dropping the read variable, dropping -s, and
silencing an e2e recipe are each caught. Unit lane 1513 passed, 1 failed
(hack/ghcr-mirror_test.bats, which reads a live cluster and fails the
same way under main's runner).
Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The quiet unit lane rests on one invariant: run_one tees the raw stream
to $log and the fail handler prints all of it for whichever test fails,
so suppressing the live stream drops the output of PASSING tests only.
That invariant fails silently. It is exercised only when a test fails,
so a green CI run proves nothing about it - a refactor of run_one's
reader loop that also broke the dump would ship, and would be found by
whoever next needed a red unit run diagnosed, with no trace and the run
already lost. Nothing in the tree covered it.
Seven tests, fixtures that fail on purpose, following the structure of
hack/cozytest-capture-gate.bats:
- quiet mode streams no trace for a passing suite
- the unset default stays verbose, which is what the e2e call sites in
packages/core/testing/Makefile rely on
- a failing test still dumps its whole trace under quiet mode, xtrace
included, which is the load-bearing one
- quiet mode still fails the run
- the per-test start/end markers survive, so a hang still names the
test that never returned
- the unit recipe asks for the quiet runner
- the e2e recipes do not silence theirs
Every one is mutation-proven: removing the fail-handler dump, flipping
the default, making the switch a no-op, dropping the variable from the
unit recipe, adding it to an e2e recipe, and swallowing the exit code
are each caught by exactly one test. The last of those is worth a note -
it disables the mechanism the outer runner uses to report failure, so
the run exits 0 while the failure is in its output. A probe keyed on the
exit code alone reads that as uncaught.
Two traps are recorded in the file's header rather than left to be
rediscovered. Fixtures are built with printf because the awk transform
harvests a heredoc's own @test line into the suite that contains it. And
the default is asserted under `env -u COZYTEST_TRACE`, because the
Makefile passes the variable as a command prefix, which exports it - a
nested runner started from a suite that `make unit-tests` is running
would otherwise inherit 0 and the default would never be under test.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Myasnikov Daniil