admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-11-08 21:14:56 +09:00
Code Issues Packages Projects Releases Wiki Activity
Files
6cfeaddf783645510746ff5ca5c70cdbf1c261bf
cozystack/hack/select-e2e_test.bats
T
Aleksei Sviridkin 575b41e1cd test(hack): let each bats file declare the EXIT traps it still holds 
The ban on EXIT-trap cleanup in hack/*.bats was enforced by two lists
inside hack/cozyreport.bats: one naming the files known to be clean, one
freezing an exact trap count for each file that was not. Both had to be
edited from whatever change moved them, and that is where the guard kept
failing. A change adding a trap to its own file had to edit a string in
a suite it otherwise never touches, so two changes sharing no line still
invalidated each other: each stayed green against its own base, git
merged both cleanly, and the guard went red only once the second landed.
A file arriving with traps hit the same wall from the other side -- the
inventory did notice it, since the string it compared was built by
scanning the directory, but absorbing it meant an edit in a file nothing
in the author's diff pointed at.

Replace both lists with a declaration each file makes about itself: one
"# EXIT-TRAP DEBT: N" comment, exact rather than a ceiling. A file
carrying none must install no EXIT trap, which covers the converted
files, the files that never had one, and every file added later.
Growing or shedding a trap now fails in the file the change already
edits, so two changes that disagree about a count collide textually
instead of silently.

The declaration is read from the leading comment block, not from
anywhere in the file. A .bats file is shell that writes shell, so the
same line turns up inside a heredoc, a fixture writer or an
expected-output string, where it is data belonging to one test. Reading
it there as a statement about the whole file would let an unrelated
fixture excuse a real trap, and would do it silently, since nothing in
that test's own diff looks like a declaration. A comment block is the
region with no interior: stopping instead at the first @test would still
read a line out of a helper's heredoc.

Not every counted handler is debt. A trap inside an explicit subshell
does not replace the bats binary's own, so a test failing inside
`( ... )` still prints its `not ok`. hack/e2e-test-openapi.bats kills a
backgrounded kubectl proxy that way, and moving the kill to the end of
the body would leak a process holding a fixed port. Its declaration
records that rather than scheduling a conversion, and because the
ratchet is exact, removing the trap fails too -- the count protects the
construct. What the count cannot do is tell the two apart: substituting
a test-level trap for the subshell one keeps the total at 1 and stays
green, which the header states rather than leaves to be found.

Counting bounds the keyword and the signal the same way, at any
character that cannot be part of an identifier, and matches the signal
in either case. Whitespace on the right missed `trap ... EXIT; cd
"$tmp"`; whitespace on the left missed `tmp=$(mktemp -d);trap ... EXIT`
and `(trap ... EXIT; true)`; upper case missed `trap ... exit`, which
bash and dash both install. All are real handlers that scored zero, and
the left boundary matters most, since the inventory being replaced had
none and caught the semicolon form. A bare word boundary is not enough
either way: `bootstrap ` ends in `trap `, and it must keep scoring
nothing. A quoted signal counts for the same reason as the rest.

Two handlers sharing one line are reported rather than counted, since
the count is a count of lines and the second would otherwise arrive
without moving the total; splitting the line properly needs a shell
parser, a semicolon inside a handler's own action not being a
separator, and guessing wrong undercounts -- the one direction a ratchet
cannot afford.

The scan recurses, so hack/e2e-apps/*.bats is covered rather than
sitting one directory below the guard that claims the tree. It reads
.bats and nothing else, so a handler arriving through a sourced .sh
stays outside it -- hack/e2e-chainsaw/_lib/run-kubernetes.sh installs
two, each benign for its own reason rather than by design: one sits in
a function declared with `(` and so runs in a subshell, the other in a
brace function no @test calls. That boundary is stated in the header
rather than papered over.

The guard moves to hack/bats-no-exit-trap.bats: its subject is every
unit suite under hack/, not the report collector it grew up in. Its
fixture helpers assemble the trap keyword and the signal from separate
arguments, because the guard scans its own source and a fixture written
as a literal would be counted as a real trap in it.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-08-07 23:53:40 +03:00

174 lines
6.8 KiB
Bash
Raw Blame History

#!/usr/bin/env bats
# EXIT-TRAP DEBT: 15 -- see hack/bats-no-exit-trap.bats; lower it as the traps go, delete it at zero.
# -----------------------------------------------------------------------------
# Unit tests for hack/select-e2e.sh
#
# cozytest.sh's awk parser recognizes only @test blocks and a bare `}` on its
# own line; there is no bats `run` or `$status`. Each test runs as a shell
# function under `set -eu -x`, so assertions are direct shell tests that exit
# non-zero on failure. setup()/teardown() are not honored — each test creates
# and cleans its own scratch dir.
#
# Run with: hack/cozytest.sh hack/select-e2e_test.bats
# -----------------------------------------------------------------------------
@test "single app diff selects only that suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "packages/apps/postgres/values.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$output" = "postgres" ]
}
@test "operator diff selects all dependent app suites" {
# postgres-operator is depended on by postgres-application, harbor-application
# (Harbor uses postgres as its backing DB), and monitoring-application (Grafana
# DB). monitoring has no chainsaw suite so it's filtered out by the selector.
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "packages/system/postgres-operator/values.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
echo "$output" | grep -wq postgres
echo "$output" | grep -wq harbor
if echo "$output" | grep -wq kafka; then
echo "operator diff must not trigger full suite; got: $output" >&2
exit 1
fi
}
@test "engine-dependency change does not fan out via the ordering edge" {
# cert-manager is a dependency of cozystack-engine, and every app declares
# dependsOn cozystack-engine purely as an INSTALL-ORDERING edge (the app's
# *-rd HelmRelease waits for the ApplicationDefinition CRD). That edge must
# not propagate test selection: a cert-manager change selects only its
# genuine direct dependents (postgres, harbor, ...), never unrelated apps
# like kafka that reach cert-manager solely through the engine.
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "packages/system/cert-manager/values.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
echo "$output" | grep -wq postgres
echo "$output" | grep -wq harbor
if echo "$output" | grep -wq kafka; then
echo "cert-manager change must not fan out via engine; got: $output" >&2
exit 1
fi
}
@test "networking change triggers full suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "packages/system/cilium/values.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
# Full suite means more than 5 suites
[ "$(echo "$output" | wc -w)" -gt 5 ]
}
@test "library change triggers full suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "packages/library/cozy-lib/templates/_helpers.tpl" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$(echo "$output" | wc -w)" -gt 5 ]
}
@test "docs-only diff selects nothing" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "docs/README.md" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ -z "$output" ]
}
@test "kubernetes-application maps to the four kubernetes suites" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "packages/apps/kubernetes/values.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
echo "$output" | grep -q "kubernetes-latest"
echo "$output" | grep -q "kubernetes-previous"
# The OIDC render-side suites exercise the same kubernetes app chart, so a
# chart-only change must select them too.
echo "$output" | grep -q "kubernetes-oidc-system"
echo "$output" | grep -q "kubernetes-oidc-customconfig"
}
@test "dashboards-only diff selects nothing (path is plural)" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "dashboards/gpu/gpu-fleet.json" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ -z "$output" ]
}
@test "shared E2E helper script triggers full suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "hack/e2e-chainsaw/_lib/run-kubernetes.sh" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$(echo "$output" | wc -w)" -gt 5 ]
}
@test "chainsaw config change triggers full suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "hack/e2e-chainsaw/.chainsaw.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$(echo "$output" | wc -w)" -gt 5 ]
}
@test "install bats triggers full suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "hack/e2e-install-cozystack.bats" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$(echo "$output" | wc -w)" -gt 5 ]
}
@test "per-suite edit selects only that suite, never escalates" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "hack/e2e-chainsaw/redis/chainsaw-test.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$output" = "redis" ]
}
@test "pull-requests workflow change triggers full suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo ".github/workflows/pull-requests.yaml" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$(echo "$output" | wc -w)" -gt 5 ]
}
@test "backup example harness edit selects its app suite" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "examples/backups/postgres/run-all.sh" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
[ "$output" = "postgres" ]
}
@test "backup example without a matching suite selects nothing" {
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp -r packages/core/platform/sources "$tmp/sources"
echo "examples/backups/no-such-app/run.sh" > "$tmp/diff"
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources") || true
[ -z "$output" ]
}
Reference in new issue View Git Blame Copy Permalink