The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.
Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Dropping every package that no suite owns also drops services the rest
of the platform relies on: multus for secondary networks, the cozystack
and LINSTOR schedulers, flux-plunger, cozy-proxy and ouroboros, whose
hairpin NAT the Kubernetes suites assert. Nothing waits for them, so no
PackageSource edge reaches them, and adding one would also change
reverse test selection.
The remaining packages stay uninstalled on purpose, also when test
selection answers with every suite: nothing in the run looks at them,
so installing them only adds a readiness wait.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The gateway namespace-churn regression requires a registered and ready
pod admission webhook. Its availability cannot be tested when selective
installation removes the package that registers it.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Tenant HelmReleases need a shard reconciler, and the root SeaweedFS
release creates COSI resources even when no backup suite is selected.
Without those controllers, a fresh reduced install cannot become ready.
The Kubernetes suites create worker pools through KubernetesNodes, and
MongoDB renders an operator-owned CR. Selecting only the cluster API or
application chart drops resources those suites require.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Backup roundtrip suites create Bucket and backup resources that the
application graph does not require. A graph-only closure therefore
omits the APIs those suites need on a fresh install.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Dangling references and cycles make the inventory unsuitable for a
fresh install even when the selected suite does not reach them.
Accepting that inventory would hide a broken platform graph.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
PackageSource names contain dots, so regular-expression matching can
accept a different source. Selecting a suite also does not prove its
install closure retains every API that suite uses.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
A missing suite owner or a discarded discovery failure can leave a
reduced install without the APIs its tests exercise. Graph-only checks
cannot establish that the chart preserves the requested package set.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Chainsaw builds step files only from directory entries that are not
directories, so a directory named like 01-x.yaml makes nothing a
suite. Both selectors listed it as one, and select-install demanded a
mapping for it. A symlink still counts, as it does for Chainsaw. A
directory named chainsaw-test.yaml keeps its suite listed: Chainsaw
reads that name whatever its type and fails loudly on a directory, and
an unlisted suite would hide that failure.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Chainsaw walks every subdirectory of the path it is handed, and the
runner hands it each top-level directory under hack/e2e-chainsaw/, so
a test in <dir>/<sub>/ runs as part of <dir>. Both selectors read only
one level down, so a directory whose tests all sit deeper was never
listed: no selection named it, the full-suite list left it out too,
and only a run with no selection at all executed it.
Both scripts now take the top-level directory of a test file found in
that directory or anywhere below it.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
With no chainsaw-test.* in a directory, Chainsaw v0.2.15 builds a Test
from files named like 01-install.yaml. Such a directory runs whenever
the whole tree is handed to chainsaw, but neither selector listed it,
so no escalation carried it and no edit to it could select it.
Both scripts now derive the suite list from the file names Chainsaw
itself accepts.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Chainsaw loads chainsaw-test.yml when a directory has no
chainsaw-test.yaml, but select-e2e and select-install enumerated the
.yaml name only. A suite spelled that way ran whenever the whole
directory was handed to chainsaw, yet no escalation listed it and no
edit to it could select it, so it silently dropped out of every
narrowed run.
select-install also sent find's errors to /dev/null inside a pipeline
that reported sort's status, so a failed listing validated no suite at
all and passed. It is now captured and fails validation.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
extra/etcd renders kind: EtcdCluster from
etcd-operator.cozystack.io/v1alpha2, but cozystack.etcd-application
declared no dependsOn edge to cozystack.etcd-operator -- the only
operator-backed application source in the tree that did not. The
edge has been absent since the sources were first added.
Two consequences, one per side of the graph.
On install this is a latent ordering bug. Without the edge the
etcd-rd HelmRelease registers the ApplicationDefinition as soon as
cozystack-engine is up, so a tenant can create an Etcd before the
operator exists and its HelmRelease then fails with `no matches for
kind "EtcdCluster"`. hack/select-install.sh's forward closure for the
etcd suite omitted the operator for the same reason, which would
install the suite against a cluster with no CRD to apply to.
On test selection cozystack.etcd-operator reached no runnable suite,
so every change to the operator escalated to all 21 suites -- 5 of
the last 150 merged pull requests. The etcd suite exists; only the
edge that makes it reachable from the operator was missing.
Scope of the ordering change: both Packages are emitted
unconditionally by the same bundle (templates/bundles/system.yaml,
under the single bundles.system.enabled guard, in every variant), so
no supported configuration has one without the other. An admin who
lists cozystack.etcd-operator in bundles.disabledPackages while
keeping the app now leaves it DependenciesNotReady -- the same
exposure postgres, mariadb, kafka and redis already carry, with
Package.spec.ignoreDependencies as the escape hatch.
select-install.sh --validate confirms the edge adds no cycle and
nothing dangling.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
A `trap ... EXIT` inside an `@test` replaces the one the bats binary
installs for its own bookkeeping, so a test that then fails prints no
TAP line at all. Not `not ok`, nothing: the run ends with a warning
that N of M tests executed, and anyone judging it by grepping for
`not ok` reads a green suite. Cleanup moves to the last line of each
body instead. Both runners set -e, so a failed test leaves its scratch
directory behind, which is what you want to look at anyway.
hack/promote-retag_test.bats was failing this way already, reporting
five passes and no failures while six of its eleven tests printed
nothing. Those six invoke the script for real, and it requires
sha256sum; they pin PATH to the stub directory plus /usr/bin and /bin,
which is where coreutils installs that binary and where macOS does not,
so the failure was of the platform rather than of the code and CI never
saw it. The stub directory already supplied yq and skopeo for the same
reason and now supplies sha256sum too, resolved from absolute
candidates because the stub itself is first on PATH.
Two file headers described the trap arrangement they no longer use and
are corrected rather than left to outlive it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Scrub the last "-" mention from the --validate usage doc (the escape no
longer exists, so the header asserted the opposite of the behavior), and
add the negative bats case the tightened branch was missing: a suite
whose hardcoded mapping names a PackageSource absent from the graph must
fail validation with the "maps to ..., not a PackageSource" error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Commit 3dbc7d366 removed ExposureClass/ServiceExposure (api/network, the
controller, and the CRDs shipped in cozystack-controller/definitions) in
favor of the native LoadBalancer path, but the chainsaw port that landed
the next day (dd2f89d20) brought the serviceexposure suite along. The
suite creates network.cozystack.io/v1alpha1 fixtures against an API that
no longer exists, so every full-suite run (any PR touching core paths,
and every release cut) fails its two tests with "no matches for
network.cozystack.io/v1alpha1".
Remove the dead suite and its wiring: the suite_to_source case and its
core-platform bats test in select-install (the "-" mechanism stays for
the next core-platform-owned suite), and the README suite list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Address the review blockers on the diff-driven install selector:
- B1: seed cozystack.cozystack-engine for any non-empty closure. Suites
backed by a system package (e.g. securitygroup, which applies a
sdn.cozystack.io resource served by the aggregated cozystack-api) carry
no dependsOn edge to the engine, so the forward walk could never reach
it and the emitted set could not run the suite. Hard-error if the engine
source is absent from the graph.
- B2: an unmapped suite is now a hard error in closure mode (every bad
suite reported in one run, no partial output) instead of failing open.
--validate additionally asserts every hack/e2e-chainsaw suite resolves
to a real PackageSource or "-", and fails when the suites dir is missing.
Also correct the inaccurate "securitygroup also stands up a Kubernetes
app" comment (its fixtures reference Postgres/Kubernetes as selectors
only), note the variant-union over-selection in build_forward_deps, drop
the unnecessary source copies from the tests, and add coverage for stdin
input, the engine seed, fail-closed mapping, the --validate suite guard,
and the missing-suites-dir / missing-engine failure modes.
Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Add hack/select-install.sh: given the E2E suites select-e2e.sh picked,
emit the forward dependsOn closure over the PackageSource graph — the
minimal set of packages to install for those suites. Companion to the
reverse (test-selection) walk in select-e2e.sh; keeps the cozystack-engine
edge since install needs it.
Suite->source mapping follows the <suite>-application convention with a
bare-<suite> fallback plus explicit cases for irregular names
(kubernetes/vminstance/securitygroup) and core-platform features that have
no separately-enabled package (serviceexposure). Includes a --validate mode
(reachability + cycle check) and a 13-case unit suite auto-discovered by
make unit-tests.
Track A of #2847; tracked in #3278.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Myasnikov Daniil