Found by the full-stack VM harness, which 3524 unit tests and a 7-image
container matrix had all missed. On a real Ubuntu 24.04 install the log read
"Opening HTTP/HTTPS (80, 443) via nftables" and not one line of
bootstrap_firewall appeared: it returned at its "a firewall is already active"
guard, so the box got its two ports opened and no default-deny policy at all.
The headline feature of that work was a no-op on exactly the population it was
written for, and it looked green the whole way.
The guard asked `[ "$(firewall_detect)" != "none" ]`. firewall_detect answers a
different question — "which backend do I open a port THROUGH" — and returns
`nftables` whenever the nft binary exists and can list a ruleset, including an
empty one. That is true of every modern Linux, so the guard reported every box
as already protected. Verified directly against a stock-image fixture: same
inputs, `firewall_detect` -> nftables, inbound-default-deny -> unprotected.
firewall_inbound_default_deny() asks what the bootstrap actually cares about:
does this host already deny inbound by default? Presence of rules is
deliberately NOT the test -- Docker installs plenty of nft and iptables chains
and protects nothing. A default-deny policy on the input hook is: firewalld
running, ufw active *with* "deny (incoming)", an nft base chain hooked to input
with policy drop or reject, or an iptables INPUT policy that is not ACCEPT.
Unknown answers 1 (unprotected), because the caller's response is to offer
protection and every caller re-checks before acting.
firewall_detect is left alone -- configure_firewall legitimately wants a
backend name for a binary that exists -- and both functions now carry a comment
saying which question they answer, since that is what was conflated.
Six cases pin the behaviour, five of them shapes the old guard got wrong: an
empty nft ruleset must run the bootstrap, Docker's chains must not read as
protection, an active-but-allow-inbound ufw must be fixed rather than
respected, a real nft policy drop must be left alone, and the two predicates
must disagree on a stock image -- that last one asserted at the predicate level
so the distinction survives a refactor of either function. make_stub_stdout is
new because an exit-code stub cannot express "the tool ran fine and reported
nothing", which is the whole stock-image case.
Green on all seven CI images (265/265/265/275/275/275/263, 0 failed).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
verify_pg_dump only runs its structural `pg_restore -l` TOC check when the
binary is on PATH. The harness stubs every other external command it touches
(systemctl, nginx, npm, curl, docker, flask, pg_dump) but never pg_restore, so
two assertions were decided by the host:
* no postgresql-client -> branch skipped, check degrades to "non-empty",
and the synthetic `PGDMP fake contents` fixture passes
* postgresql-client present -> the real binary parses the version bytes
after the PGDMP magic, rejects the fixture, verify_pg_dump returns 1
That is the CI failure. T10f failed directly; T10e failed downstream because
backup_current correctly halts on an unverifiable backup. update.sh is right --
the tests leaked.
Add a make_stub_pg_restore factory mimicking the real -l semantics (rejects
missing/empty, rejects anything without the PGDMP magic, else prints a TOC) and
install it in T10e/T10f. Give T10f a non-empty non-archive fixture too: that is
the case separating the two branches inside verify_pg_dump -- the size check
passes it, only the TOC listing catches it -- so the path the test was named
for is now actually exercised.
Verified both with and without a pg_restore on PATH: 59 passed, 0 failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
EL9/Fedora containers ship without diffutils and openSUSE Leap without
tar, which failed 4 of the 7 cross-distro unit jobs:
- files_identical() in stubs.sh: cmp when available, sha256sum
(coreutils, everywhere) otherwise
- the two tarball-fixture tests skip cleanly when tar is absent
(harness's existing skip convention; proven on the other distros)
- the t16/t16r assertion lists are now && chains: bash suppresses
set -e inside an if-condition subshell, so sequential statements
only gated on the LAST one — the cmp-less EL9 run proved it by
passing t16r vacuously while cmp exited 127
Verified in the actual images: rockylinux:9 65p/0f/0s,
opensuse/leap:15.5 61p/0f/4s (was 63p/2f and 61p/2f).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The July 2 outage function had a unit test — with a populated fixture
only. This encodes the missing net:
- fresh-minimal-box fixture loop: every observation/discovery/snapshot/
report function in update.sh (17) and install.sh (16) must survive a
zero-app box under set -Eeuo pipefail; policy comment requires new
functions to be appended
- scripts/test/stubs.sh: shared failing-stub factories (systemctl exit
5, docker fail/empty, curl exit 22, GitHub rate-limit JSON)
- assertion sweep: 43 bare capture-then-compare sites now include exit
status in the assertion
- best-effort contract test: the run block's real APP_BASELINE ->
report chain exits 0 on a fresh box AND on a genuine regression
- fix(update): the loop immediately caught report_failure's trailing
'[ -n $UPDATE_LOG ] && printf' returning 1 when UPDATE_LOG is empty
(same species as the outage) — converted to if, added to the loop
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
