Files
cozystack/hack/e2e-chainsaw/_lib
Aleksei Sviridkin 7ec5dff93a test(e2e): add a runner-layer fixed-work canary around the node-join wait
The counters the node-join failure path collects are read in whatever
unit their source publishes: time in the CPU rows, events in the KVM
exits, instantaneous values and running maxima in the files beside
those. Not one of them carries a unit of work of its own, so a machine
that spent every tick and got a fraction of the work done reads healthy
in all of them.

Add a collector that carries its own unit of work. It runs two fixed
amounts of work at the runner layer and times each: a loop over a
handful of integer-valued scalars, which pressure on the shared cache or
the memory controller moves little, and a store stream through blocks
larger than the last-level cache one core can allocate into, which is
dominated by it. Fixed work gives each reading a scale of its own, so a
red run can be read with no green one beside it, and the capture states
the expected range and where the estimate comes from.

Sampled where the three counter pairs already are, before the wait on
every run and after it on both paths, but outside their brackets: it
occupies a core rather than reading a file, and a burn inside one of
those intervals would land in the interval each pair divides by:
runner-kernel CPU time charged to the join window, and for the two
counting the guests, a core taken away from what they count.

Two arms, each under a twenty-second ceiling with the five-second kill
grace behind it, so fifty seconds at worst against the 420s diagnostics
budget. The budget guard derives that from the ceiling the source
declares, and the phase-warning guard names the canary among the
collectors that keep running when timeout is missing. An exit shaped
like the ceiling is attributed to it only when the whole bound was on
the clock: an outside kill produces the same status, and read by status
alone it would publish exactly the finding this collector exists to
detect.

A reading below the range the legend states, and an arm the ceiling
stopped before it could produce one, reach the job log from all three
reporting sites, each naming where it was taken, the way a missing
reading does from the two outside the diagnostics block. On the passing
path the report that holds the figure is the artifact nobody downloads.
Each floor is printed in the legend it comes from and sits where a
bounded arm crosses it before the ceiling stops the arm, because a floor
the ceiling reaches first never fires: the arm is cut off before its
rate can fall that far, and every alert then arrives as a bound rather
than as the slowdown the floor was meant to name.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-08-20 11:22:23 +03:00
..