The harbor, openbao and qdrant app charts deploy their workload through
a nested HelmRelease named `<release>-system`, and pass
`fullnameOverride: <release>` down to it. The inner chart therefore
stamps app.kubernetes.io/instance from its own release name, with the
suffix, while object names come from the override, without it. The
catch blocks selected on the outer release name, so podLogs matched no
pods and every failed run collected an empty result.
A pod name says nothing about its instance label; the two come from
different sources on purpose. In qdrant the unsuffixed value is
therefore not retired but kept as a collector of its own: the parent
chart's PVC-cleanup Job labels its pod template with the parent release
name, and that suite deletes the parent inside a `try:`, so the Job is
in scope while the test still runs. The comment there is explicit about
how narrow that window is -- the hook always exits 0, so Helm calls it
succeeded and `hook-succeeded` takes the pod away; what is left is an
uninstall that hangs with Helm still waiting on it.
Harbor renders the same hook and does not get the same collector. It
has no `delete` op, so its parent release is uninstalled only in
Chainsaw's cleanup phase, which no catch can observe: catch is a defer
inside the step, while the cleaner runs from t.Cleanup, after the test
function returns. That collector would be the same permanently-empty
capture this commit removes. A comment records the asymmetry so the
next reader does not restore it for symmetry. openbao renders no such
hook at all.
Each workload value is now also pinned on a resource in an assert the
suite already runs, so the chart renaming a label fails a step instead
of quietly emptying a capture. The comments state what that does and
does not cover: it does not catch a wrong value typed into a selector,
and it catches nothing when the run fails before the assert.
Harbor's database was reached by no collector at all. It is selected on
`cnpg.io/cluster`, CNPG's own primary label, rather than on the
instance label CNPG merely derives from the Cluster name -- the primary
label has nothing to drift from, and it is what the postgres suite and
the WorkloadMonitors already use.
Each lookup keeps stderr apart from the value it captures. kubectl exits
0 while logging a klog line about an API group it could not reach, and
an aggregated APIService in that state is what a failed run usually
looks like; folded into the value, that line makes the "nothing found"
branch unreachable and prints as though it were a result.
Telling those outcomes apart is one helper in hack/e2e-chainsaw/_lib/
rather than a copy per call site, separated from the kubectl call so it
can be pinned without a cluster, the way etcd-probe.sh already is. Both
the census and the qdrant cleanup listing report through it, differing
only in wording, so the discrimination is implemented and tested once.
The tests cover each outcome for both wordings, including the zero exit
with a klog line on stderr -- the case a folded stream turned into a
row.
A collector that matches nothing cannot say so: podLogs is silent, and
`kubectl get -l` exits zero on an empty match. Each test-wide catch
prints the instance label of every pod in the namespace, and the qdrant
cleanup catch reports its three outcomes apart from a single call --
printing what that call returned rather than running it again, which
could put a healthy listing under a failure heading.
That census runs on a degraded cluster, where kubectl otherwise retries
discovery against an unreachable apiserver until something kills the
script, so every lookup carries `--request-timeout`. Measured against an
unreachable apiserver, a bounded call costs about five times the flag,
since kubectl retries discovery five times before it gives up. Each
script then carries an explicit op timeout sized above its own calls --
2m for the single-call censuses, 5m for the qdrant cleanup listing,
which makes three. An inner budget is what should fire: a SIGKILL at the
op boundary yields nothing, where a kubectl that gives up leaves an
error to print.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The openbao suite's finally deleted PVC data-openbao-test-0 while the
StatefulSet pod still mounted it; the kubernetes.io/pvc-protection finalizer
blocked removal and the delete hit its 5m timeout, failing the suite on every
run (confirmed in CI runs 28084122948 and 28025182013). Delete the OpenBAO CR
first so the HelmRelease/StatefulSet/pod tear down and release the PVC, then
remove the now-unmounted PVC — mirroring the BATS teardown order.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replaces the per-app BATS E2E suite (hack/e2e-apps/*.bats) with declarative
Kyverno Chainsaw tests under hack/e2e-chainsaw/, extending this branch's pilot
(postgres + bucket) to all 18 app suites.
- Declarative ports assert on status/conditions and concrete fields; the
repeated "timeout ... until kubectl get" + "kubectl wait" boilerplate
collapses into a single polling assert with structured failure diffs and
automatic events/describe/podLogs capture on failure.
- Inherently imperative suites keep their logic in script steps: openbao
init/unseal, kuberture split-horizon probes, vminstance, the gateway
admission/impersonation cases, and kubernetes-latest/previous, which wrap
the relocated hack/e2e-chainsaw/_lib/run-kubernetes.sh verbatim.
- Gateway tests derive the tenant apex from the namespace host label at
runtime, so they are host-independent.
CI: the e2e job now runs "chainsaw test hack/e2e-chainsaw/" via a new
test-chainsaw target and uploads the JUnit chainsaw-report.xml; the chainsaw
binary is added to the e2e-sandbox image. install-cozystack and test-openapi
stay BATS. The per-app 3-retry loop is dropped, since assertion polling
replaces the fixed-timeout flakiness it papered over.
Verified against a dev cluster (DB/app, storage and VM suites pass). Service
port assertions use the ports[*].port projection form because Chainsaw
v0.2.15 mis-evaluates JMESPath number-literal port filters.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin