mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-03 10:37:21 +09:00
feat/externalip-api
31
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Myasnikov Daniil
|
186547d835 |
test(e2e): run the merge-gating lanes on Talos containers
The srv1-srv3 nodes of both lanes that gate a merge -- the same-repo one in pull-requests.yaml and the fork one in e2e-fork.yaml -- run as Talos containers instead of QEMU guests, so a tenant worker sits at L2 rather than L3. Measured in CI on commits where both substrates ran the same tree: whole-job wall clock down 24-49 minutes, essentially all of it in Chainsaw, while the two tenant-Kubernetes suites go from 1143-2040s to 466-809s and stop depending on the runner's vgif flag. The node-join soft-red gate goes with the substrate, which is what it was for. It existed because nested virtualisation on the shared runners degraded far enough that a worker registering in minutes elsewhere could miss any deadline this test could afford; #3513 established that the discriminator was the host's kvm_amd vgif flag, and removing a nesting level removes what it acted on. So hack/e2e-node-join-soft-red.sh, the SOFT-RED-node-join.txt marker and the soft_red job outputs are gone from all four workflows, and a missed deadline is an ordinary red again. The ::warning annotation survives, because the reason a run went red should be legible without opening the log, and so does the 124-versus-everything -else test, because "the workers were slow" and "the wait never ran" send a reader to different places. Done by swapping the substrate inside the existing e2e jobs rather than by promoting a separate container job. Those jobs carry more than a substrate -- the GitHub App token, the report-overrun guard, the images list, the SSH breakpoint -- and promoting a job that never had them would have dropped four features without saying so. Three properties of container mode drive the design, and each fails quietly rather than loudly. machine.kernel.modules is a silent no-op -- kernel_module_spec.go returns early on ModeContainer with no error and no event -- so the host loads openvswitch and zfs before the nodes start and the workflow step asserts both rather than trusting them; a missing module surfaces an hour later looking like a CNI or a storage regression. Docker gives a privileged container its own tmpfs /dev seeded once at creation, so /dev/kvm works but ZFS zvols created later never appear and linstor-csi fails ControllerPublishVolume; the compose file binds the host devtmpfs, which is also why talosctl cluster create docker can never carry LINSTOR. And there is no ARP VIP, so 192.168.123.11 stands in for the QEMU lane's .10. The QEMU machinery stays in the tree and keeps running: nightly.yaml and e2e-tag.yaml still call `make prepare-env`, so DRBD, the replicated StorageClass, its Immediate binding mode, the tenant StorageClass propagation that only applies to remotely-accessible classes, and the Cozystack Talos node image with its extensions all keep nightly and release-candidate coverage. None of them is exercised per pull request any more. That is the trade. Nothing on the PR path builds or downloads a nocloud disk now, so build-talos drops the talos-nocloud step and the talos-image artifact. build-talos itself stays: finalize consumes its digest fragment. One property this does not fix, recorded beside the runner class because it will be read off a red run eventually: a container node reports the HOST's CPU and memory as capacity, and hack/e2e-container-up.sh counters that with kubelet systemReserved. That corrects the scheduler and not what a pod reads from /proc, so a workload sizing itself from /proc/cpuinfo or /proc/meminfo is configured for the whole runner. Also carried here because the same files carry them: the ghcr.io pull-through mirror removal (see #4007 -- whichever lands first makes the other a no-op), the authoritative HelmRelease readiness gate, the backup-credential preflight, the prepull guard against an image-less container fragment, and folding the two standalone OIDC suites into the latest-version tenant cluster. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
ad961f6de1 |
test(e2e): cover webhook admission after namespace churn
Exercise ingress and pod admission immediately after deleting a tenant namespace, and route ingress-nginx changes to the gateway suite. Assisted-By: GPT-5 <noreply@openai.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
adf231513f |
fix(ci): correct two reachability defects in the selection graph
Both are cases of the walk reaching the wrong set of suites for a package, and both were found by enabling the site-router suite on #3426, which is the first suite in the tree to land downstream of kubevirt-cdi. `vm-disk-application` now maps to the `vminstance` suite. It mapped to a `vm-disk` suite that does not exist, which intersect_suites() drops, so the source reached nothing runnable and neither did kubevirt-cdi above it: every CDI change ran all 21 suites. The vminstance suite creates a VMDisk and asserts the DataVolume behind it (hack/e2e-chainsaw/vminstance/vmdisk.yaml, vmdisk-vmi.yaml), so it is the suite that covers both. A CDI change now selects `vminstance` alone. `cozystack.cozystack-basics` joins `cozystack.cozystack-engine` as a propagation hub excluded from the reverse-dependency walk. Every edge into it exists so a namespace or a platform-wide policy is in place before the dependent installs -- kubevirt-cdi says exactly that in its own source -- which is install ordering, not behaviour. The damage runs opposite to the engine's: the engine fans one change out to every app, while basics narrows down instead. It reaches no suite today so it escalates, but it sits upstream of kubevirt-cdi, so the first suite to land under CDI silently converts the platform's namespace-and-policy package from the full run to that one suite. On #3426 that is 22 suites down to `site-router` alone, with nothing in the output to say coverage was lost. Dropping the reverse edges keeps the package reachable, stops it propagating, and leaves a change to it running everything through the per-path escalation. Both rules have a test, each verified to redden with its rule removed: the mapping test falls back to the full-suite escalation, and the hub test -- which seeds the hazard by giving redis-application an edge onto basics, since no suite sits downstream of it in this tree yet -- selects `redis vminstance` instead of everything. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
73bc4c0ae3 |
fix(ci): build the backstop's reason list without a blind pipeline
`unmatched=$(echo … | tr … | sort -u | grep -v '^$' | paste -sd ' ' -)` takes its exit status from paste, so a failure in tr, sort or grep is invisible under set -e. Demonstrated with a sort stub that exits 1: the pipeline reports rc=0 and yields an empty string, so the line becomes `no runnable suite is named by ''`. That is the same last-command blindness this script already handles twice -- for the suite-list find and for the two yq indexes -- and the comments there explain why each was a bug. Selection stays fail-safe because the escalation is decided before the message is built, so the only casualty is the reason line; but the reason line is a contract as of this change set, and a contract that degrades in silence is what the change set exists to remove. Replaced with the shell's own split plus a `case` membership test: no external command, nothing that can half-succeed, dedup preserved, and the same idiom resolve_suites already uses a few lines up. A test feeds two unresolved directories with one repeated and pins the whole line, so a regression to a form that can emit a partial or empty name turns the file red rather than printing a worse message. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
c28d7804af |
fix(ci): enumerate the inert .gitattributes instead of matching by name
The rule was broader than its justification. It read `(^|/)\.gitattributes$`, so it claimed every present and future .gitattributes anywhere, while the argument for it was about content: every one in this tree holds nothing but `linguist-generated` markers, which reach no build, chart or test. The filename does not carry that argument. .gitattributes can also set `filter`, `eol`, `working-tree-encoding` and `export-subst`, each of which changes what lands in the working tree and therefore what gets built, so a by-name rule would eventually make a live file inert with nothing to notice it. Nothing is lost by narrowing, because only three files ever reached the rule: api/.gitattributes and internal/crdinstall/manifests/.gitattributes are escalated by full_suite_pattern first, so entries for them would be dead. Measured against the base selector, those three are exactly the paths whose classification this branch changes, and they still select nothing. Same trade as the workflow enumeration a few lines up, and the same upkeep, which the comment now states: after adding or moving one, re-check with `git ls-files '*.gitattributes' '.gitattributes'` and confirm the contents are still only linguist markers. A .gitattributes not on the list is classified by whatever rule its path falls under -- the graph inside a package, the unclassified fall-through at the root -- and both fail safe. A test pins that root case, so a future widening back to a by-name match turns the file red. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
b2702621d3 |
fix(ci): decide the per-app bats rule per path, not after the merge
Two defects in the hack/e2e-apps/ arm added by the previous commit. The arm set trigger_any=1 for whatever basename it derived and left the verdict to the final intersection. That makes the escalation depend on the REST of the diff: hack/e2e-apps/monitoring-oidc-system.bats alone emptied the selection and the backstop escalated, but the same file beside packages/apps/redis/templates/redis.yaml selected `redis` with nothing on stderr, where the pre-change selector ran all 21 suites. That is the merge-before-escalate shape #3330 removed from the graph walk, reintroduced one rule over. The basename is now membership-tested against the suite list on the spot and an unmatched one sets trigger_full=1 and names the file. POSIX case matches `/` with `*`, so the arm also sees nested paths, and the unanchored capture turned hack/e2e-apps/fixtures/postgres.bats into the "suite" fixtures/postgres -- right outcome by accident, reason line printing a path where a suite name belongs. The capture is anchored to one segment, and what the arm does with anything that is not a top-level <name>.bats is now stated rather than left to fall out of a regex: it is shared material for those suites the way hack/e2e-chainsaw/_lib/ is for the Chainsaw ones, so it escalates for the same reason. Both claims that said otherwise are corrected -- the arm's own comment and docs/agents/e2e-testing.md both asserted the deferred escalation was equivalent to the old fall-through, which held only for a single-file diff. The backstop comment drops the e2e-apps clause it gained last commit, since that route no longer reaches it. The header's "four of the seven" becomes count-free: this commit adds two escalating branches, so a hardcoded total was already stale. Tests: the mixed-diff case is the regression pin (the isolated case passes with the escalation deferred and cannot see the bug), plus a nested-path case and a non-.bats case. Both fixes are mutation-verified -- restoring either defect turns the file red. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
4d4f5edf92 |
fix(ci): escalate only the e2e bats files, not the whole unit lane
full_suite_pattern took every hack/*.bats, so all 63 of them ran the whole Chainsaw suite -- including the 60 that are the unit lane and that the e2e sandbox never executes. The root Makefile is the authority on the split and draws it at exactly the prefix this narrows to: BATS_UNIT_FILES := $(filter-out hack/e2e-%.bats,$(wildcard hack/*.bats)) The 60 files it keeps run under `make unit-tests`; the three it filters out -- e2e-prepare-cluster, e2e-install-cozystack, e2e-test-openapi -- are the ones packages/core/testing's recipes execute inside the sandbox, and those still escalate. So no Chainsaw suite can regress from an edit to one of the 60. Inert here is not a green gate with nothing behind it. `make unit-tests` is gated on pull-requests.yaml's `code` output, computed in the workflow as "any changed path outside docs/" and never from select-e2e.sh, so a bats-only pull request still runs the whole unit lane. The `e2e` job reads the same output, so it still installs the platform and runs the OpenAPI tests before skipping Chainsaw. Two tests: one pins the rule on both sides of the prefix and asserts an inert bats file beside an app path does not mask the selection, and one walks every hack/*.bats in the tree and requires the selector's verdict to agree with the Makefile's split, so a file named to fit neither lane surfaces here rather than on the next PR that touches it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
33428671fa |
fix(ci): classify hack/e2e-apps/ and packages/tests/ in select-e2e
Three path classes matched no rule in hack/select-e2e.sh and escalated to all 21 Chainsaw suites through the unclassified fall-through -- 7 of the last 150 merged pull requests between them. hack/e2e-apps/<name>.bats now maps to the <name> suite, off the basename exactly as the per-suite rule takes the name off a hack/e2e-chainsaw/<app>/ directory. Not marked inert: what is left in that directory after the Chainsaw migration is wired to nothing, and inert would bake that orphan status into the rule and go quietly wrong the day a lane runs those files again. A basename no suite carries drops out of the intersection at the bottom of the script and escalates through the backstop, which is the outcome those paths already had, so mapping optimistically loses nothing and gains the correct answer for every name that is a suite. packages/tests/ and every .gitattributes join inert_config_pattern, which is what the script's own header says to do with a genuinely inert path rather than widening the fall-through. packages/tests/ is a helm-unittest fixture chart: changing a test OF cozy-lib does not change cozy-lib, no PackageSource lists those paths as a component, and nothing installs them -- a change to the library itself still escalates through packages/library/. .gitattributes is matched by name wherever it lives because every one in this tree is a linguist-generated marker, which reaches no build, chart or test; one under a tree full_suite_pattern already covers still escalates, since that pattern is checked first. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
3c3f9450ae |
fix(ci): name the cause on every select-e2e escalation
Seven branches in hack/select-e2e.sh escalate to the full Chainsaw suite and four of them did it in silence: the shared hack/e2e-chainsaw/_lib/ and .chainsaw.yaml case, the full_suite_pattern match, a packages/ path no PackageSource claims, and the empty-selection backstop. "Run everything" is the same 21 suite names whichever rule produced it, so a reason line is the only thing that tells the seven apart. full_suite_pattern is the commonest cause of a full run by a wide margin -- 88 of the last 150 merged pull requests, 68 of them with no other cause -- so the usual answer to "why did this pull request run the whole suite" was the one the log never carried, recoverable only by re-deriving the selection by hand. Each new line names the path that caused it rather than only the rule, and the backstop names the directly-selected names it could not resolve. They go to stderr, never stdout: stdout is the suite list and both e2e lanes parse it, so a reason line there would be read as a suite name. A unit test asserts each of the four messages. A reason that regresses to silence changes no selection, so it is invisible to every other test in the file. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
d0990299ed |
feat(clickhouse): let the backup sidecar trust a private S3 CA, unblocking the backup e2e round-trip (#3385)
## What this PR does Gives the ClickHouse `clickhouse-backup` sidecar a way to trust an S3 endpoint whose certificate is signed by a private CA, and uses that to make the ClickHouse backup/restore e2e round-trip actually run in CI. This actualizes #2600, whose feature code and example scripts have since landed on `main` independently, and whose remaining contribution — the e2e test — was still in the retired BATS format. #2600 can be closed once this merges. ### 1. `backup.endpointCA` on the ClickHouse chart The sidecar had **no TLS/CA surface at all**: bucket coordinates and credentials arrive as `S3_*` env, and nothing let it verify an endpoint signed by a private CA. Cozystack's own in-cluster SeaweedFS is exactly that case — it serves `:8333` behind the self-signed "SeaweedFS CA" — so a ClickHouse release could only ever back up to a publicly-trusted endpoint. MariaDB (`storage.s3.tls.caSecretKeyRef`) and Postgres (barman `endpointCA`) have had a CA field all along; this closes the gap for ClickHouse. `backup.endpointCA` takes `{name, key}`. When set, the chart mounts that Secret into the sidecar and points `SSL_CERT_DIR` at it. Empty — the default — renders no env, no mount and no volume, so existing releases are byte-for-byte unchanged. **Why `SSL_CERT_DIR` and not `AWS_CA_BUNDLE`.** Both are honoured by `altinity/clickhouse-backup:2.7.4`; I measured them against a self-signed endpoint and public AWS S3 in turn. Go reads `SSL_CERT_DIR` *in addition to* the system bundle, so both endpoints verify. `AWS_CA_BUNDLE` *replaces* the SDK's trust pool: the private endpoint verifies, and public S3 then fails `x509: certificate signed by unknown authority` — it would silently break a release that also talks to a public endpoint. `S3_DISABLE_CERT_VERIFICATION` was rejected outright, since it drops verification rather than trusting one more CA. ### 2. The e2e suite, and what it was really testing Two Tests added to the existing `hack/e2e-chainsaw/clickhouse/` suite (in name order, mirroring the `etcd`/`mariadb`/`postgres` shape — see the layout rationale below for why they live here rather than in a new directory): - **`clickhouse-1-backup-contracts`** (CI, `tenant-test`, ~98s): a backup-enabled ClickHouse asserted against the *running* cluster — the `clickhouse-backup` sidecar on the `ch-backup-api` `:7171` port, and the `<release>-backup-api-auth` Secret whose keys the sidecar's `API_USERNAME`/`API_PASSWORD` actually resolve `valueFrom`. Each is a silent break if a chart bump renames it. Needs no object store. - **`clickhouse-2-backup-roundtrip`** (CI, `tenant-root`): `Bucket` → `Altinity` strategy + `BackupClass` → source ClickHouse with a sentinel row → `BackupJob` → in-place `RestoreJob` → to-copy `RestoreJob` into a second instance, driving `examples/backups/clickhouse/run-all.sh` so test and docs cannot drift. The round-trip was originally gated behind `CLICKHOUSE_E2E_S3_ROUNDTRIP` and **skipped in CI** — its `PASS (1.01s)` meant the gate fired, not that a backup happened. `backup.endpointCA` removes the reason for the gate, so the suite now adopts the same three preconditions mariadb/postgres rely on: run in `tenant-root` (an isolated tenant's Cilium egress allowlist blocks its Pods from reaching `tenant-root`'s SeaweedFS), target the in-cluster endpoint instead of the `Bucket`'s external ingress URL (an unroutable placeholder in CI), and trust its certificate via the copied CA. It also pre-cleans with `cleanup.sh`, because `tenant-root` is shared and a leftover `Succeeded` `BackupJob` would satisfy the harness's wait without a backup ever running. Both backup Tests live in the existing `hack/e2e-chainsaw/clickhouse/` suite rather than a new suite directory, so no selector mapping is added: `hack/select-e2e.sh` already derives `clickhouse` both from `packages/apps/clickhouse/**` (the `*-application` rule) and from `examples/backups/clickhouse/**` (the harness path), and both paths therefore select the suite that actually runs the harness. That alignment is the reason for the layout — a separate `clickhouse-backup/` directory would have made an edit to the harness select a suite that cannot exercise it. `hack/select-e2e.sh` and `hack/select-install.sh` are otherwise untouched (the only change is a comment recording that invariant), and the existing `backup example harness edit selects its app suite` case in `hack/select-e2e_test.bats` covers the mapping. ### Verification - `helm unittest` — 17 pass, including 5 new `endpointCA` cases: the unset default renders nothing, the env/mount/volume wiring, a custom key still projected as `ca.crt`, composition with `useSystemBucket`, and `backup.enabled=false` renders nothing. - The CA-trust behaviour was measured against the real image, not assumed: without a CA knob the sidecar reports `x509: certificate signed by unknown authority`; with `SSL_CERT_DIR` the handshake succeeds. - `hack/select-e2e_test.bats` (16) and `hack/select-install_test.bats` (17) pass; `select-install.sh --validate` reports graph OK. - `make generate` is idempotent, so the pre-commit job's `git diff --exit-code` stays clean. - The contracts Test passed in CI on earlier runs of this branch. The round-trip only became CI-live with this work, so it still needs a green E2E run before merge — the branch has since been rebased onto current `main` to get one. ### Screenshots Not applicable — no UI changes. ### Downstream repositories Walked the trigger map against the diff, file by file. The load-bearing change is the new `backup.endpointCA` field in `packages/apps/clickhouse/values.yaml` / `values.schema.json`; everything else is `hack/e2e-chainsaw/`, `examples/backups/`, one docs paragraph, and regenerated artifacts, none of which appear in the map. - [ ] No downstream repository is affected by this change - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/18 — "add, remove or rename a field in an app's `values.schema.json`" reaches the provider's hand-maintained schema, model and expand/flatten pair, so the new nested object is not surfaced until it is added there. Opened as an issue rather than a PR because the natural moment is alongside that repo's next `api/apps/v1alpha1` pin bump, which is a maintainer call. Not ticked, with reasons rather than silence: **cozystack/website** — the trigger is *adding, renaming or removing a package*, and this adds a field to an existing one. `clickhouse` is already in the website `Makefile`'s app list and its reference page is generated from this package's `README.md`, so the regenerated `backup.endpointCA` rows are carried by the existing release-tag automation with no manual PR. **cozystack/external-apps-example** — vendors `hack/package.mk` and `update-crd.sh`, neither of which this PR touches. **cozystack/ansible-cozystack**, **ccp**, **talm**, **cozyhr**, **cozy-proxy**, **cozystack-telemetry-server**, **examples** — no installer values, `hack/` layout, namespace, variant, node-prerequisite, annotation, label or metric in their trigger rows is touched here. ### Release note ```release-note feat(clickhouse): add `backup.endpointCA` so the clickhouse-backup sidecar can verify an S3 endpoint whose certificate is signed by a private CA, such as Cozystack's in-cluster SeaweedFS. Leave it unset for publicly-trusted endpoints; existing releases are unaffected. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional private S3 endpoint CA support for ClickHouse backup/restore sidecars via `backup.endpointCA`, including configurable Secret name and CA bundle key (default `ca.crt`). * Sidecars mount the CA and extend trust while leaving public endpoints unchanged. * **Documentation** * Updated ClickHouse chart docs, operational backup guidance, and backup/restore examples to cover CA setup, overrides, and cleanup. * **Tests** * Added helm-unittest and end-to-end coverage validating CA rendering/mounting behavior and ClickHouse backup/restore workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
587a7b9fbd |
fix(ci): decide E2E coverage per changed path
The full-suite escalation for a changed package that reaches no suite was
read off the merged selection, so it depended on the whole diff rather
than on the path that needed it. Any other changed path contributing a
suite name made the selection non-empty and the escalation vanished with
nothing in the output to record it: edit a platform component, adjust the
Chainsaw test beside it, and the component ran that single suite instead
of everything. Over one values.yaml per package under
packages/{apps,system,extra}, 82 of the 159 escalate on their own and
stop escalating once an unrelated suite edit joins the diff.
Coverage is now decided per changed path. Each path resolves through its
owning sources to the suites that exist, and a path reaching none of them
forces the full run whatever the rest of the diff selected. The unit is
the path rather than the source: system/postgres-operator belongs to two
PackageSources where one reaches no suite, so deciding per source would
escalate on that half and run everything for every change to it.
The escalation names its cause on stderr, as every other escalation in
this script already does. Most packages reach no suite of their own, so
this is the branch a full run usually takes; without a line saying which
sources bought it, "why did this run everything" has no answer in the log.
The walk moves into resolve_suites() to be called once per path. POSIX sh
has no `local`, so the function writes into the caller's scope and is
safe only inside the $( ) its single call site wraps it in; a test pins
that the call count stays one.
Two entries leave src_to_suites. `postgres-application` restated what
stripping the suffix produces and `external-dns` restated what carrying
the source name through produces, so both made the exception list read as
longer than the set of real exceptions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
0980e755eb |
fix(ci): map E2E suites from sources not named after an app
src_to_suites() covered PackageSource names ending in -application plus external-dns, and the filter after the graph walk dropped every other source it reached. cozystack.kuberture and cozystack.securitygroup- controller are neither, so both were walked to and then discarded: a CNI change ran neither of the two controller suites most likely to regress from it, and a change to either package reached no suite at all and escalated to the full run instead of selecting the one that covers it. Every reached source now maps through the table. That table is the inverse of select-install.sh's suite_to_source(), and a test walks every suite through both, so the next source name fitting neither convention fails there rather than quietly making its own suite unreachable. The cilium test measured the gap rather than hid it, asserting the 19 suites the selector could reach and naming the two it could not. The subtraction is removed rather than inverted: kept as an exclusion list it would stay green if the walk stopped reaching them for an unrelated reason. What replaces it is a second assertion against a tree holding one suite the graph reaches and one no source names, which an escalation cannot satisfy. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1abc653cfb |
fix(ci): treat a disabled chainsaw suite as inert in test selection
Editing hack/e2e-chainsaw/backup/chainsaw-test.yaml.disabled selects the whole Chainsaw suite. The per-suite rule reads the suite name off the directory and ignores the suffix, so the change selects "backup"; the final intersection against the suites that actually exist finds no such suite and empties; and the safety net for an empty selection escalates to the full run. A disabled suite is registered nowhere and executed by nothing, which makes its edits the one file class that provably cannot regress a test. The mapping had it backwards: they bought the most expensive outcome the selector has. Rare in practice, and paid on every commit by anyone grooming or re-enabling such a suite. Classify *.disabled as inert ahead of the per-suite rule, and record it alongside the other rules that select nothing so the header stays an accurate enumeration. The safety net keeps its job for paths no rule has looked at, which is what it was added for. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
37a395363f |
fix(ci): stop select-e2e from deciding on inputs it never checked
Three holes on the same seam: the script builds a list, never looks at whether the command that built it succeeded, and then decides from the empty result. The dependency indexes are built as `$(build_owners_index | sort -u)`, where the function is a single yq. A pipeline carries the status of its last command, so the script sees sort's zero and `set -e` never fires: a missing binary or a malformed source leaves the index empty, every changed path falls through to escalation, and CI runs the full Chainsaw suite on every pull request. The direction is safe and that is the problem — an empty index and "no owners matched" print the same thing, so the symptom is a full run, which reads as the conservatism the selector is supposed to apply. Selection can stay switched off indefinitely with nobody signalled. Check each half's status on its own and name yq once on stderr before taking the fallback the empty index would have reached anyway. The suite list one line up has the identical shape and a worse consequence: `$(find ... | sed | sort)` reports sort, never find, and errors went to /dev/null. That list is what every escalation prints. So a find that failed produced an empty list, and "escalate to the full suite" then emitted a blank line — which both e2e lanes read as "skip Chainsaw" before posting the required "E2E Tests" status green. The one outcome that means run everything delivered a run of nothing, on the pull request that earned a full run. Capture find on its own, check it, let its errors through. A failing find is not the only way to end up with an empty list, and the escalations are not its only reader, so the check sits once where the list is built. The examples/backups/<app>/ rule escalates nothing and still membership-tests against that list: with it empty the test fails, the app is not selected, and an edit to a backup harness that should run its suite reports nothing to run — output identical to a legitimately empty selection, on a path no escalation guard would ever see. An empty list is a broken enumeration, a moved directory or a wrong working directory, not a project without tests. Refuse to answer at all. Both lanes run the step under `bash -e`, so a non-zero exit fails the job rather than falling through to the empty selection this is here to prevent. Each is covered by a test that stubs the failing component and asserts both the exit status and that the script names what broke. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
8e273a7341 |
docs(ci): record what escalating on the non-PR workflows costs
Three of the four workflows on the escalation list cannot be exercised by the run they trigger. `e2e-fork` runs on `workflow_run`, which always takes the default-branch copy; `e2e-tag` runs on `workflow_call` from a tag push; `nightly` runs on a schedule. Only `pull-requests.yaml` is executed from the pull request's own head, so only there does escalating test the edit rather than the tree around it. That argument existed in the reference doc and was removed by this branch rather than answered, and the list it argued against is what replaced it. Measured over six months of main, 26 commits touch one of the three with no other escalating path; each now pays a full Chainsaw run it used to skip, on a suite whose flake rate is the reason people avoid running it for nothing. Keeping them is defensible: the rule stays one idea rather than two, and dropping them leaves those files inert, which reads as an oversight rather than a decision. What was not defensible was paying it silently. The trade now sits next to the pattern that makes it, with the number, so the next person weighing it starts from measurement instead of rediscovering the mechanism. The reference doc points there rather than carrying its own copy. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
55f3c79e9f |
docs(ci): stop pointing at the inert list for paths it does not hold
Four places said an empty selection means the inert pattern matched, and sent the reader to `inert_config_pattern` to see which paths those are. Documentation, dashboards and `*.md` are dropped before that pattern is ever consulted, and a backups example whose app has no suite is dropped where backup examples are matched. A reader checking the pattern for any of them finds nothing and has to reconstruct the order by reading the script. Name the rules that produce an empty selection instead, and keep the claim to the one the lanes rely on: empty is a decision some rule reached, not a path nothing looked at. The same bullet claimed every path is classified. One is not. The inert pattern makes `.github/` inert as a whole directory and the workflows that run the suite are exempted from it by name, so a workflow added under `.github/workflows/` reads as inert and the fall-through never sees it. Today's enumeration is correct and I checked it against `rg -l test-chainsaw`, so this is not a live hole, but the sentence saying it cannot exist is what makes the reminder next to that list look decorative. Record it as the guard it is. Separately, the fork lane's rationale for appending a newline described a mechanism this branch removes: the reader now guards its own loop, so the terminator is no longer what keeps the last path alive. Say that it is now the text-file contract and a second line of defence across two files, which is a reason to keep it and a different reason than the one written there. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
548dfa9520 |
docs(ci): say what an empty selection actually means
Four places on this branch describe an empty selection as "classified inert", against a list of three outcomes that every path is said to land in. There is a fourth: an edit under examples/backups/<app>/ where the app has no Chainsaw suite selects nothing, decided inline where backup examples are matched rather than through the inert list. The rule is deliberate and explained where it lives; it just is not one of the three the summary enumerates. The property the two lanes actually depend on is weaker and true: an empty selection means some rule decided the change cannot affect a suite, never that nothing looked at the path. State that instead, in the header that defines the classes and in the privileged lane's comment that consumes the result. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
cc522e7def |
fix(ci): classify the last path when the list has no trailing newline
The classifier this branch introduces is only total if every line reaches the loop that applies it. POSIX read assigns the final line and then returns non-zero when the input does not end in a newline, so a plain `while read` drops it. The fall-through that escalates an unrecognised path lives inside the loop body, so the dropped line is never classified at all and the selection comes back empty. Both lanes read empty as "skip Chainsaw" and post the required status green. The shape that matters is a list whose only unclassified path is last: `docs/x.md` followed by an unrecognised path with no terminator selects nothing, where the same list with a terminator escalates to all 21 suites. The safety net this branch is built around is bypassed by exactly the input it exists to catch. No caller triggers it today, because `git diff --name-only` terminates its output and the fork lane appends a newline explicitly. That makes it caller discipline documented in one of two callers, which the next caller will not read. Guard it where the assumption lives. Both new cases are red without the guard and the twenty-five existing ones stay green, so they pin this behaviour rather than the loop's general shape. They clean up inline instead of through an EXIT trap: that trap replaces the one the bats binary installs for its own bookkeeping, and a test failing under it can print no TAP line at all, which is the opposite of what a regression pin is for. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
ae4e9e55cb |
fix(ci): classify every TIA path and escalate the unclassified
`select-e2e.sh` silently selected nothing for a changed path it did not recognise outside `packages/`. Both e2e lanes read an empty selection as "skip Chainsaw" and then post the required "E2E Tests" status green, so an unrecognised path produced a green merge gate with no suite run. The worked example is `hack/common-envs.mk`, which supplies the tag, push and output flags of every image in the tree and matched nothing: `full_suite_pattern` covered `hack/*.sh` and `hack/*.bats` but not `hack/*.mk`. `pkg/`, `tools/`, `go.mod`, `hack/lib/` and `hack/buildkitd.toml` were in the same position, as was `.github/workflows/e2e-fork.yaml` while `pull-requests.yaml` was already escalated. A blanket "empty means full suite" fallback would be the wrong fix: it escalates the inert paths the design deliberately skips. Instead every path now lands in one of three classes by an explicit rule — the full-suite list for shared build inputs and the e2e harness, an inert list for paths that cannot affect what e2e exercises, and the PackageSource graph for `packages/(apps|system|extra)/`. A path matching none of them escalates and logs why, so the next path someone adds fails safe instead of failing open. The two lists are checked in that order, so a specific escalation beats a broad inert directory: `.github/` is inert, the workflows that run the suite are not. `*.md` and `docs/`/`dashboards/` stay ahead of both, so a README under `packages/core/` does not inherit that tree's escalation. Ten tests cover both sides of the classification. Mutation-checked: reverting the fall-through fails the unclassified-path test, dropping `.github/` from the inert list fails the non-e2e-workflow test, and swapping the two list checks fails the existing pull-requests.yaml test. Note the per-entry escalations are belt-and-braces — removing `hack/*.mk` from the pattern still escalates via the fall-through, which is the point of having it. Closes #3392 Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
82aa967674 |
test(e2e): correct the round-trip budget and its stale comments
Second review pass on this branch.
1. The op timeout equalled the inner sum instead of exceeding it: the comment
enumerated 3900s of inner waits and then set 65m, and 3900s IS 65m — so there
was zero headroom for the applies and SQL calls between those waits, and the
claim of "room" was false. Sized the inner budgets against what CI actually
takes instead (the bucket suite completes in ~37s and a backup-enabled
ClickHouse plus Keeper reaches Ready in ~98s, measured on run 30353894332):
bucket claim/access 300->180 and the three job waits 600->480, summing to
3300s, under a 60m op timeout. Shrinking the inner budgets rather than
raising the ceiling also keeps this suite closer to its postgres and mariadb
siblings on a 180-minute job cap.
2. The strategy Pod's securityContext comment still justified root with "Pod
runs apk add at startup", which the previous commit made conditional — the
default path now runs the platform's pre-baked image and installs nothing.
Scoped the note to the alpine fallback and said explicitly that the field set
matches cozy-default-altinity, runAsNonRoot deliberately unset in both.
3. clickhouse-backup.yaml still referenced clickhouse-backup-{1-contracts,
2-roundtrip}: the file moved unchanged, so the Test rename never reached its
own header and a reader following either name found nothing.
4. wait_sts_ready's timeout diagnostic could not print anything. It selected
clickhouse.altinity.com/chi=${name#chi-}, which yields
clickhouse-<app>-clickhouse-0-0 while the label carries the CHI name
(clickhouse-<app>), and 2>/dev/null then swallowed kubectl's "No resources
found". Ask for the StatefulSet's own first replica by name instead and let
both streams through; verified it now surfaces the Pod events that explain
why the replica never came up.
Also dropped the clickhouse bats case added in the previous commit: it asserted
what the generic examples/backups/<app> -> <app> mapping already guarantees, so
it did not lock the invariant its comment claimed. That invariant is now
recorded where it belongs — in the mapping's own comment in select-e2e.sh.
The dedicated ch-backup-client image this example now inherits is tracked in
#3420; noted there that the command: override is load-bearing in two files.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrey KolkovandClaude Opus 4.8
|
c1fd7db51d |
test(e2e): fold the clickhouse backup Tests into the app's own suite
Review findings on the previous two commits. 1. TIA no longer selected the suite that runs the harness. select-e2e.sh maps examples/backups/<app>/** to suite <app>, so with the round-trip living in a separate clickhouse-backup/ dir an edit to the harness selected `clickhouse` — which did not execute those scripts — while the suite that does was left out. Fixed the way postgres/mariadb/etcd already do it: both Tests move into hack/e2e-chainsaw/clickhouse/ as clickhouse-1-backup-contracts and clickhouse-2-backup-roundtrip. That also deletes every special case the earlier commits added (src_to_suites, suite_to_source, and the bats case that guarded them — both scripts are now byte-identical to main), and the suite list in hack/e2e-chainsaw/README.md stays correct because no dir is added. The bats case is replaced by one asserting the invariant that actually matters: an examples/backups/clickhouse edit selects the suite that runs it. 2. The docs claimed a fail-fast the harness did not have, and the budget was sized as if it did. wait_for_field polled to the full timeout with no terminal-value escape, and the HelmRelease/StatefulSet waits were bare `kubectl wait` with no existence backstop and no Stalled fail-fast. Ported wait_for_field's fail_value parameter and wait_hr_ready from the postgres helpers, added wait_sts_ready, and routed all eight waits through them. Verified against a fake kubectl: a Failed BackupJob now bails in 3s instead of 600s, a Stalled HelmRelease immediately. The inner budgets sum to 3900s, which the old 30m op timeout did not cover — a hang would have been SIGKILLed with no diagnostics, the failure mode .chainsaw.yaml calls strictly worse — so the timeout is now 65m with the arithmetic written out. 3. The strategy Pod ran `apk add --no-cache curl jq` on every backup and restore. Harmless while nothing in CI ran it; a per-PR dependency on the Alpine CDN once it does, and broken outright air-gapped — which is exactly why the shipped strategy bakes curl+jq into its image. Step 01 now reads that image off cozy-default-altinity and drops the apk add, falling back to alpine only when the platform strategy is absent. Also bounded the two polling curls with --max-time; a hung connection previously stalled the loop until the outer timeout. 4. 91-scenario-user-backup.md still documented the old tenant-test default. 5. .chainsaw.yaml claimed all app tests run in tenant-test, which the postgres and mariadb round-trips already contradicted before this branch. The round-trip header now also states its scope plainly: it exercises the example's strategy, not the shipped cozy-default-altinity. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
d8e003f4cb |
test(e2e): add clickhouse altinity backup chainsaw suite
Port the intended ClickHouse Altinity backup/restore e2e coverage from cozystack#2600 (never-merged hack/e2e-apps/backup-clickhouse.bats) to the current Chainsaw approach, reworked on origin/main where the Altinity backup feature and examples/backups/clickhouse already landed. Two Tests, etcd-suite shaped: - clickhouse-backup-1-contracts (CI): a backup-enabled ClickHouse, asserting the chart<->Altinity-strategy contract against the running cluster — the clickhouse-backup sidecar on the ch-backup-api :7171 port and the populated <release>-backup-api-auth Secret the strategy Pod authenticates with. Needs no reachable object store. - clickhouse-backup-2-roundtrip: the full Bucket -> BackupJob -> Backup -> in-place + to-copy RestoreJob round-trip, driving examples/backups/clickhouse/run-all.sh. Gated out of CI (CLICKHOUSE_E2E_S3_ROUNDTRIP=1) because kind's self-signed SeaweedFS S3 endpoint cannot be TLS-validated by the in-Pod sidecar. Wire clickhouse-application -> "clickhouse clickhouse-backup" in hack/select-e2e.sh so ClickHouse chart edits exercise the backup suite too. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey KolkovandClaude Fable 5
|
50fb3c87c2 |
test(postgres): run the backup round-trip e2e in CI under TIA
Ungate postgres-2-backup-roundtrip: instead of an opt-in env flag it now runs whenever TIA selects the postgres suite (postgres changes, and every release cut), like the other chainsaw tests. The blocker was network placement, not capability: the S3 client is the barman-cloud sidecar inside the Postgres Pod, and the isolated e2e tenant's Cilium egress allowlist blocks it from tenant-root's seaweedfs — so the Test runs in tenant-root (same-tenant egress) against the in-cluster S3 endpoint, trusting its self-signed cert via the copied seaweedfs-system-ca-cert. This is exactly the recipe validated end to end on a real cluster. Also map examples/backups/<app>/ edits to that app's suite in hack/select-e2e.sh (the round-trip executes those scripts as its harness; previously such edits were silently ignored), with bats coverage for both the mapped and the no-suite case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
dd2f89d208 |
test(sdn): port the securitygroup and serviceexposure suites to Chainsaw
Main added securitygroup.bats (#2922 membership model) and serviceexposure.bats (#3081 ExposureClass/ServiceExposure) after the Chainsaw migration branch was cut. - securitygroup: two Tests — the aggregated-API projection round-trip (backing CNP marker label, empty-value membership endpointSelector, lineage fromApp + membership fromSG peers, 1:1 rule translation, the view hiding storage internals, unmarked-CNP invisibility via an error op, delete propagation) and the controller membership-label stamp/clear lifecycle. Only the attachments annotation stays a jq script (it is a JSON string). Chainsaw auto-cleanup covers every fixture even on mid-test failure — the bats had no teardown hook and leaked its fixtures. - serviceexposure: the externalIPs backend resolve/Ready/assignedIPs path with the no-MetalLB-pool negative as an error op (ordered after resolvedBackend so it cannot false-pass), and the finalizer-removal test collapsed onto the delete op itself, which waits for full disappearance. - selection: kubernetes-application now also maps to the two OIDC render-side suites (they exercise the same app chart — a gap main's bats selection had too); the new dirs enter all_apps discovery automatically. Selector unit tests extended and green (13/13 under cozytest). - README: suites list updated. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
9297e51170 |
Merge origin/main into feat/e2e-chainsaw-example
Reconcile the Chainsaw e2e migration with ~500 commits of main drift: - keep main's run-kubernetes.sh helpers (kubectl_wait_retry, tenant drain, LINSTOR pool-free wait, SC-fallback probe) under the branch's _lib/ rename with the Chainsaw finally/snapshot deltas re-applied - resolve modify/delete on 10 drifted hack/e2e-apps/*.bats as deleted; their post-branch assertion drift is re-ported into the Chainsaw suites in follow-up commits - drop the 4 new main-side bats (kubernetes-oidc-*, securitygroup, serviceexposure) from _lib/ relocation; ported as Chainsaw suites in follow-up commits - docs/agents/e2e-testing.md: merge main's plan-job TIA wording with the Chainsaw suite phrasing Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
IvanHunters
|
e0597e3d99 |
fix(platform): make *-rd sources depend on cozystack-engine (#3041)
## Problem On an unlucky Flux reconcile order, a `*-application` PackageSource that ships a `system/*-rd` component can install before the `applicationdefinitions.cozystack.io` CRD exists. Every `*-rd` chart renders an `ApplicationDefinition` (`cozystack.io/v1alpha1`); that CRD is installed by the `application-definition-crd` component of `cozystack.cozystack-engine`. When an `*-rd` HelmRelease renders first, it hard-fails with `no matches for kind "ApplicationDefinition" in version "cozystack.io/v1alpha1"`, and that cascades into a broad install timeout — surfacing even on changes that have nothing to do with ordering, because it is a latent race. ## Fix Add `- cozystack.cozystack-engine` to the `dependsOn` of every `*-application` source that ships a `system/*-rd` component (29 sources). `gateway-application` already carried this edge — it was added earlier for the TenantGateway CRD, the same bug class, but only for that one source. A new invariant test fails if any source with a `system/*-rd` component is missing the edge. ## Why The ordering was previously implicit, relying on Flux happening to reconcile `cozystack-engine` before the `*-application` sources. Making the edge explicit removes the race deterministically. No dependency cycle is introduced: `cozystack-engine` depends only on `networking` and `cert-manager` (plus keycloak in its oidc variant), none of which reach back to any `*-application`. ## Test - New generic guard `hack/check-applicationdefinition-crd-ordering.bats` (auto-discovered by `make bats-unit-tests`, runs in CI via `make unit-tests`): parses every source and fails if any PackageSource variant with a `system/*-rd` component lacks the `cozystack.cozystack-engine` edge — so future sources are covered, not just today's set. - `helm unittest packages/core/platform`: 13 suites / 66 tests pass; the source edits do not disturb existing assertions. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved dependency ordering across several platform packages to ensure related components install in the correct sequence. * Prevented end-to-end test selection from being overly broad when only ordering-related dependencies change. * **Tests** * Added checks to catch missing dependency requirements and to verify test selection behavior stays accurate. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
e22d84f2c8 |
test(e2e): migrate app suite from BATS to Chainsaw and wire into CI
Replaces the per-app BATS E2E suite (hack/e2e-apps/*.bats) with declarative Kyverno Chainsaw tests under hack/e2e-chainsaw/, extending this branch's pilot (postgres + bucket) to all 18 app suites. - Declarative ports assert on status/conditions and concrete fields; the repeated "timeout ... until kubectl get" + "kubectl wait" boilerplate collapses into a single polling assert with structured failure diffs and automatic events/describe/podLogs capture on failure. - Inherently imperative suites keep their logic in script steps: openbao init/unseal, kuberture split-horizon probes, vminstance, the gateway admission/impersonation cases, and kubernetes-latest/previous, which wrap the relocated hack/e2e-chainsaw/_lib/run-kubernetes.sh verbatim. - Gateway tests derive the tenant apex from the namespace host label at runtime, so they are host-independent. CI: the e2e job now runs "chainsaw test hack/e2e-chainsaw/" via a new test-chainsaw target and uploads the JUnit chainsaw-report.xml; the chainsaw binary is added to the e2e-sandbox image. install-cozystack and test-openapi stay BATS. The per-app 3-retry loop is dropped, since assertion polling replaces the fixed-timeout flakiness it papered over. Verified against a dev cluster (DB/app, storage and VM suites pass). Service port assertions use the ports[*].port projection form because Chainsaw v0.2.15 mis-evaluates JMESPath number-literal port filters. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
02eeb9e0d5 |
ci(e2e): keep the cozystack-engine ordering edge out of test-impact selection
Every *-application source declares dependsOn cozystack.cozystack-engine so its *-rd HelmRelease waits for the ApplicationDefinition CRD to register before it reconciles. That is a universal install-ordering edge, not a behavioral one. The test-impact selector reused the same dependsOn graph to pick which e2e app suites to run for a diff. Because the engine transitively depends on postgres-operator, keycloak and cert-manager, a change to any of those fanned out through the engine to every app, defeating test-impact analysis. Exclude cozystack.cozystack-engine as a propagation hub in the reverse-dep walk so the engine stays reachable but no longer pushes selection to its app-dependents. A change to the engine itself still triggers the full suite via the existing no-app-descendants safety net. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
b81e92ac9c |
ci(tests): remove redundant Release E2E workflow
The Release E2E workflow (`release-e2e.yaml`) ran the full bats suite on every release tag as a non-blocking alarm — its own header notes "a failure here is an alarm, not a release blocker." It duplicates coverage the release-cut PR already provides: the "Prepare release" commit bakes image digests into `packages/core/` (and other packages), which TIA escalates to the full suite, so the release PR exercises the whole suite against the commit the tag will point at. It has also never been green since it was added (2026-05-02): every tag run failed at the build step. Even after #2973 fixed the registry login today, the next run still failed (Build Talos image). Removing a perpetually-red, non-gating duplicate removes noise without losing coverage. Removes the workflow, drops `release-e2e` from `select-e2e.sh`'s full-suite pattern, repoints its unit test to `pull-requests.yaml`, and updates the e2e-testing doc. Refs #2397 Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
6e5379eff9 |
fix(ci): tighten select-e2e.sh per review
- Use grep -Fxq instead of -w when intersecting with all_apps so apps with hyphenated names cannot prefix-match each other. - Collapse build_owners_index and build_reverse_deps into single yq invocations with -N. Output is identical to the per-file loop and about 30x faster on the current 91 PackageSource files. - Use paste -sd ' ' - on the two full-suite emit paths for consistency with the post-walk join at line 147. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
0788e94a7c |
feat(ci): test-impact analysis E2E (default-on) + release E2E workflow
Three changes that compose into a single E2E policy:
1. **Test-impact analysis selector** (`hack/select-e2e.sh`).
Reads `packages/core/platform/sources/*.yaml` (the PackageSource
dependency graph also rendered by `cozypkg dot`) and emits the bats
files affected by a PR diff. Library/core/networking/CI changes,
shared `hack/*.sh` helpers, the install bats, and the
`pull-requests` / `release-e2e` workflows trigger the full suite.
Doc/dashboard/`*.md` diffs select nothing. Per-app bats edits
select only that app. Anything inside `packages/` with no graph
entry, or a system source whose graph has no `*-application`
descendants, falls back to the full suite (conservative).
2. **Default-on TIA in `pull-requests.yaml`** with `full-e2e`
opt-out. Drops the previous `e2e-tia` opt-in shape — the
selector's behaviour has been validated, so making it the default
gives PR runtime savings out of the box. PRs that need the full
suite (e.g. cross-cutting refactors that the selector can't
reason about) carry the `full-e2e` label and get the unchanged
pre-TIA flow. Docs-only PRs (selector emits nothing) skip the
`Run E2E tests` step entirely.
3. **`release-e2e.yaml`** runs the full suite on every tag push
(`v*`). Independent of `tags.yaml`'s release publish flow — a
failure here is an alarm, not a release blocker. Closes the
coverage gap left by making PR runs use TIA: the full suite is
now exercised against shipped code at tag-cut time.
Bug fixes vs. the original `feat/select-e2e-script` draft:
- `dashboard/` -> `dashboards/` (path is plural in this repo).
- `hack/e2e-apps/<name>.sh` and `hack/<name>.bats` were silently
ignored; they now trigger the full suite (test infrastructure
affecting all apps).
- A path matching the graph but yielding zero `*-application`
descendants now falls back to the full suite instead of silently
emitting nothing (system-only changes are no longer silently
skipped).
- Per-app bats edits are matched before the full-suite pattern, so
editing one bats file no longer escalates to the full suite.
- Removed the dead `skip_pattern` variable (PCRE syntax that
`grep -qE` doesn't support; it was never matched).
Tests: `hack/select-e2e_test.bats` covers all 11 cases (the original
6 plus 5 for the bug fixes above).
Workflow inputs are passed via env vars rather than direct `${{ }}`
expansion into shell commands, per workflow injection guidance.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
|