The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.
Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
An assertion written `! cmd` cannot fail a test. POSIX exempts a
command whose return value is inverted with `!` from errexit, so the
status is dropped where it is produced. hack/cozytest.sh then closes
each test function with `return 0`, and the bats binary's ERR trap
carries the same exemption -- it reports only a negated assertion
standing last in the body, where the inverted status happens to be
what the function returns.
Every absence these suites assert was written that way: a command that
must not reach the log, an image that must not be copied, a flag that
must not be passed, an object that must not exist. On that half of
each suite's contract the tests held for any input.
Rewrite each as `if cmd; then echo "FAIL: ..."; false; fi`, the form
already used where the trap had been noticed one file at a time, and
give every one a message naming what must not have happened. The
message states the property, not the pattern: where the forbidden
string is a substring of a required value -- an https endpoint the
test requires in its http form -- naming the string would describe the
opposite of what is asserted.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The chainsaw scripts source run-kubernetes.sh under `set -eu`, where the
wrapper's bare `_out=$(kubectl wait ... 2>&1)` on a failing wait aborts
the whole sourcing script at the assignment -- before `_rc=$?`, before
the retry loop, before anything prints, with the error text captured
into the substitution and lost. Every `kubectl wait` failure therefore
killed the suite instantly and silently, and the transient-retry
machinery the wrapper exists for (etcd leader flap et al.) has never
been reachable. This is how the kubernetes-latest suite died in CI with
an empty stderr: the wait it was executing failed, and the script
simply stopped.
Guard the assignment (`_rc=0; _out=$(...) || _rc=$?`) so the wrapper's
own logic decides: allowlisted transient errors retry with a visible
notice, everything else prints and propagates, exactly the documented
contract.
New unit suite executes the wrapper in a `sh -eu` subshell against a
fake kubectl, the same way chainsaw runs it: a transient etcd error
must be retried with the script surviving (against the previous lib
this exits 1 after one call with empty output), and NotFound must not
be retried, must fail, and must be printed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>