## What this PR does
Every `*.bats` file in the repo runs through `hack/cozytest.sh`, not
real bats — the e2e files via `packages/core/testing/Makefile` and the
`hack/*.bats` unit tests via `make bats-unit-tests`. cozytest runs each
`@test` under `set -e`, and POSIX errexit is suppressed for a pipeline
beginning with `!`. So a bare `! cmd` must-not-happen assertion never
fails the test: when the forbidden condition actually occurs (the
command succeeds), the negated pipeline returns non-zero but errexit
ignores it and the test passes anyway. Ten such assertions across six
files were silently vacuous.
This converts each to `if cmd; then echo "<reason>"; false; fi`. The
`if` condition is exempt from errexit (so the probe itself never aborts
the test), and the `false` in the taken branch is a plain command under
`set -e`, so it aborts and fails the test with a legible message when
the forbidden condition holds. This is the must-not-happen inverse of
the `cmd || { echo …; false; }` pattern already used in the suite (e.g.
harbor.bats, per docs/agents/e2e-testing.md). Original redirections and
quoting are preserved, so happy-path behavior is unchanged — only the
failure path now actually fails.
Converted assertions:
- `bucket`: a read-only user must not be able to upload (access-control
regression guard)
- `gateway`: a child tenant must not have its own gateway HelmRelease
(inheritance)
- `kuberture`: external-dns public/internal hostname isolation
- `e2e-install-cozystack`: tenant-name validation must reject rather
than warn; the version ConfigMap must be gone after delete
- `build-matrix` (unit test): talos/installer must be excluded from the
parallel matrix
- `capture-dataplane` (unit test): kube-dns and pending rows must be
filtered out
A separate in-flight etcd test change owns two more such assertions in
`etcd.bats`; they are left out of this PR to avoid a conflicting edit.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Strengthened several end-to-end/Bats assertions to fail more
explicitly when expected cluster conditions aren’t met.
* Improved matrix validation to reliably confirm excluded components are
absent.
* Updated dataplane load balancer filtering, bucket readonly upload, and
gateway inheritance checks to avoid false/ambiguous pass cases.
* Tightened DNS hostname isolation probes and cozystack installer
validations (tenant name dashes, deletion protection) with clearer
“expected vs found” failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
These bats files are all run by hack/cozytest.sh, which executes each
@test under `set -e`. POSIX errexit is suppressed for a pipeline that
begins with `!`, so a bare `! cmd` must-not-happen assertion never fails
the test: when the forbidden condition actually occurs (cmd succeeds) the
negated pipeline returns non-zero but errexit ignores it and the test
passes anyway. These assertions were silently vacuous.
Replace each with `if cmd; then echo "<reason>"; false; fi`, the idiom
already used and documented elsewhere in these files — the if-condition
is exempt from errexit and the `false` in the taken branch fails the test
loudly. The forbidden condition now fails the test as intended:
- bucket: read-only user must not be able to upload (access control)
- gateway: child tenant must not have its own gateway HelmRelease
- kuberture: external-dns public/internal hostname isolation
- e2e-install: tenant-name validation must reject (not warn); the version
ConfigMap must be gone after delete
- build-matrix: talos/installer excluded from the parallel matrix
- capture-dataplane: kube-dns and pending rows are filtered out
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
cozytest.sh — the runner for every bats file here — is not real bats: it
never invokes setup()/teardown(). So these hooks were dead. Cleanup ran
only via each test's success-path inline delete, and a failing test
leaked its resources into the shared tenant-test namespace (and, for
kuberture, cluster-scoped RBAC that persists across sandbox re-runs).
Wire the cleanup to cozy_cleanup(), the hook cozytest does run at suite
exit and on the first failing test:
- external-dns: dead teardown() -> cozy_cleanup() deleting all ExternalDNS
CRs. ExternalDNS is a stateless Deployment with the inmemory provider,
so a plain delete fully reclaims it — no drain needed.
- kuberture: dead teardown() -> cozy_cleanup() routed through the existing
_cleanup(), so the probe Pods, SA, and cluster-scoped ClusterRole and
ClusterRoleBinding are reclaimed on the failure path too.
build-matrix_test: drop the dead setup() entirely — it only cd'd to the
repo root, which cozytest already runs from, so the relative
hack/build-matrix.sh calls resolve without it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The matrix selector's full_rebuild_pattern omitted the root Go module dirs
(api/, cmd/, internal/, pkg/). Eight matrix image Dockerfiles COPY those dirs,
so a PR changing only root Go source emitted an empty matrix, skipped the build
job, and let e2e install stale digests — the Go change was never exercised. Add
the four dirs to the pattern and pin it with bats cases for api/- and pkg/-only
diffs forcing the full matrix.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replaces the single serial Build job (one ephemeral VM running `make
build`, ~26 images one after another, wedging on the shared buildkit
under concurrent load — #2937) with a fan-out:
plan changed files + hack/build-matrix.sh -> scoped matrix
checks unit + controller Go tests (decoupled from e2e)
build one package per job, own docker-container buildx,
uploads a per-package digest patch fragment
build-talos talos image + nocloud disk (its own leg; e2e needs it)
finalize apply all fragments, then build the installer LAST (its
flux-push bundles the whole patched tree) and emit the
unified pr.patch e2e consumes unchanged
build-matrix.sh scopes the matrix to the packages a PR touches (full set
on shared-dependency changes) and fans seaweedfs out to
objectstorage-controller. common-envs.mk gains an SBOM=1 flag wiring
buildx --sbom, kept OFF until the OCIR referrers probe confirms support.
.github/actionlint.yaml registers the custom ephemeral runner labels.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
hack/build-matrix.sh emits the CI build matrix (the list of package dirs
to build) from the root Makefile `build:` target — the single source of
truth — so the matrix can't drift from `make build`. It supports
diff-driven scoping: only units whose package dir changed are emitted,
unless a shared-dependency path (cozy-lib, the build macros, go.mod, the
build workflows) is touched, in which case the full matrix is emitted.
talos and installer are excluded from the parallel matrix: talos shares
heavy _out/assets between its image and the nocloud disk (built once in a
dedicated leg), and installer's `flux push artifact --path=packages`
bundles the whole digest-patched tree, so it must run in finalize after
all other fragments merge.
hack/build-matrix_test.bats covers full/scoped/empty selection, the
talos+installer exclusion, shared-dep escalation, and JSON/Makefile parity.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin