Files
cozystack/hack
Aleksei Sviridkin 5400f165e3 fix(e2e): keep a failed mirror lookup from pinning the public factory
resolve_talos_image_factory_url asked whether the in-cluster Talos image
mirror exists with a plain `kubectl get deploy`, where a connection
refused, an Unauthorized and a real absence all arrive as the same
non-zero exit. It then cached that reading for the whole suite, so one
unlucky moment sent every tenant worker to the public Image Factory over
live egress -- the dependency the mirror exists to remove -- while the
run still looked ordinary.

Ask with --ignore-not-found -o name instead: a real absence answers exit
0 with no output, which leaves a non-zero exit meaning only that the
question failed, and re-ask a couple of times before giving up. Only the
empty answer turns the suite toward the public factory. An unanswered
lookup warns, falls back for its own caller alone, and is not written to
the decision file, so the next tenant test asks again rather than
inheriting a guess.

Each attempt is bounded twice, and the two bounds are deliberately not
equal. kubectl defaults --request-timeout to 0, so a connection that
establishes and then stalls has no deadline of its own; the flag alone is
not a wall-clock bound either, because kubectl retries discovery several
times before giving up. Measured against a stub apiserver, the three ways
this call fails behave differently: a refused port answers in under a
second and names the refusal, one that stalls mid-TLS gets kubectl's own
handshake deadline and still produces text, and a blackholed address
produces nothing until some deadline expires. With both bounds at the
same value the expiry that arrives first is always the wall-clock kill,
so --request-timeout can never fire and that last case stays silent. The
wrapper therefore sits ten seconds above it: the request deadline ends
the attempt and kubectl says why, and the wrapper remains the backstop
for a client that overruns its own deadline.

Where timeout is not installed the read runs unwrapped rather than
through it: its exit 127 would otherwise count as an attempt that did not
answer, three times over, and a missing local binary would pin the whole
suite to the public factory while every message blamed the apiserver.

kubectl's stderr is left alone, because it is the only thing separating
refused from unauthorized from stalled. It is still not guaranteed, since
a kill can land before the client has said anything, so the lookup also
names the last attempt's exit status before giving up, in the vocabulary
the diagnostic reads use for 124 and 137.

The diagnostic path asks the same question and keeps its own answer to
it. That one gives up after a single attempt, under a phase budget where
re-asking spends what the snapshot behind it needs; this one runs on the
happy path, where a re-ask costs a suite nothing and buys it the mirror.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-08-09 13:31:55 +03:00
..
2026-02-17 22:54:12 +01:00