Every backup walkthrough carried its own wait_for_field, and most their
own wait_hr_ready and wait_deleted: twelve, eight and two copies, which
had already drifted into five, three and two bodies. A fix made in one
copy did not reach the others, and a diagnostics change carried by hand
between two of them once went into the wrong function and was caught
only by reading the diff.
The helpers now live in examples/backups/_lib/wait-helpers.sh, sourced
by each walkthrough's 00-helpers.sh. Where copies differed, the shared
body keeps the diagnostics any of them had: wait_for_field leaves
kubectl's stderr attached, and wait_hr_ready prints the conditions and
history dump on Stalled or timeout. Call sites and budgets are unchanged.
A new unit test fails when any walkthrough defines one of these helpers
again, which replaces the byte-identity pin between copies.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Reconcile the Chainsaw e2e migration with ~500 commits of main drift:
- keep main's run-kubernetes.sh helpers (kubectl_wait_retry, tenant
drain, LINSTOR pool-free wait, SC-fallback probe) under the branch's
_lib/ rename with the Chainsaw finally/snapshot deltas re-applied
- resolve modify/delete on 10 drifted hack/e2e-apps/*.bats as deleted;
their post-branch assertion drift is re-ported into the Chainsaw
suites in follow-up commits
- drop the 4 new main-side bats (kubernetes-oidc-*, securitygroup,
serviceexposure) from _lib/ relocation; ported as Chainsaw suites in
follow-up commits
- docs/agents/e2e-testing.md: merge main's plan-job TIA wording with
the Chainsaw suite phrasing
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The backup round-trip test could wedge for 20 min and then report a bare
EtcdSnapshot phase=Started with no root cause: dump_diagnostics dumped the
(healthy) EtcdCluster and the operator log, but the snapshot agent runs in
its own Job Pod, so the etcd-read / S3-upload error lives only in that Pod's
log and in the BackupJob/EtcdSnapshot status.
- dump_diagnostics now dumps the whole backup chain: BackupJob/Backup and
EtcdSnapshot describe, each snapshot Job's describe + Pod logs (selected by
the operator's etcd-operator.cozystack.io/cluster label), and recent
namespace events (the primary signal when a hung Pod logs nothing).
- wait_for_field gains an optional terminal-failure value; steps 04/05 pass
Failed so the BackupJob/RestoreJob waits return immediately on a terminal
phase instead of polling to the timeout — which also keeps the snapshot
Job Pod within its 600s TTL so its log is still readable at dump time.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Rewrite docs/agents/e2e-testing.md for the Chainsaw suite: declarative
asserts replace the until-get/wait backstops, Chainsaw cleanup replaces
test-level EXIT traps, catch blocks replace the `|| { ...; false; }` idiom,
and the TIA/diagnostics sections describe suites + `make test-chainsaw
CHAINSAW_SUITES`. Move the BATS->Chainsaw bullet out of 'in-flight' since it
is now the standard.
Fix references the migration left stale: hack/e2e-apps/ -> hack/e2e-chainsaw/
in the gemini styleguide and the nats/etcd backup READMEs. The
backup-postgres.bats / backup-foundationdb.bats files cited by the postgres
examples, the postgres README, and the foundationdb controller comment never
existed -- drop the false 'CI e2e drives this' claims and describe those
backup flows as the manual reference flows they are.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin