Commit Graph
11 Commits
Author SHA1 Message Date
Andrey Kolkov 9cc3a9e459 test(vmi): add VMInstance and standalone VMDisk backup/restore e2e round-trips
The disabled hack/e2e-chainsaw/backup/ suite was a Chainsaw port of the retired
backup.bats, written against the old backup API, so the VMInstance and standalone
VMDisk Velero paths had no e2e coverage. Replace it with two round-trips in the
vminstance suite that drive the validated examples/backups/{vmi,vmdisk} scripts,
so the documented demo and the automated test stay in step.

Both run un-gated in CI:
- vminstance-2-backup-roundtrip boots a tiny cirros VM (u1.nano, 2Gi), backs it
  up via BackupClass + BackupJob, restores in place, and asserts the VM returns
  Running off a disk PVC the restore created: a new uid, Velero's restore-name
  label and the -orig- PVC keepOriginalPVC leaves behind. The label alone is not
  enough, because existingResourcePolicy: update also stamps it on a PVC Velero
  only patched, which moves no data. A guest-side boot proof is unavailable: the
  KubeVirt CR disables serial-console logging and cirros ships no guest-agent.
- vmdisk-2-backup-roundtrip imports a 5Gi disk, backs it up, and restores it in
  place with the same three assertions. The controller prepares the target
  (suspend HelmRelease, delete DataVolume, rename PVC). On a WaitForFirstConsumer
  class the Velero data mover waits for a Pod to select a node for the restored
  PVC, which a detached disk never gets, so the script starts a pause Pod on the
  new PVC, standing in for the VM that would attach it.

The disk storageClass is an override (default replicated; the CI steps pass the
lane's COZY_E2E_STORAGE_CLASS, 'local' on the DRBD-off gate) and the script waits
are bounded so their worst case stays under the 40m op timeout, sized to keep the
whole parallel:1 suite inside the 215m job cap. On failure the catch also dumps
the Velero Restore, DataDownload and DataUpload objects. Each harness stands up a
throwaway in-cluster BSL (the platform cozy-default BSL derives the external S3
endpoint, unroutable in kind); its finally resumes the suspended HelmReleases,
reclaims the keepOriginalPVC orig PVC and its LINSTOR volume, and drains the
Velero backups before deleting the BSL. The example scripts are non-interactive
so the harness can drive them.

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-10-13 18:05:45 +04:00
IvanHunters 2188f16461 test(vminstance): cover VMDisk clones from an image and from a disk (#4586)
## What this PR does

Closes #3482.

The `vminstance` suite now covers both VMDisk clone paths, which had no
e2e test at all. One disk is cloned from a golden image with
`source.image`, and a second disk is cloned from the first with
`source.disk`. For each clone the test checks that the DataVolume points
at the expected source PVC and becomes `Ready`, that its PVC is `Bound`,
and that the VMDisk reports `Ready` and `WorkloadsReady`, the same way
the standalone VMDisk test does. The `Bound` check is separate because
CDI derives `Ready` from the DataVolume phase alone, and can reach
`Succeeded` from the claim's annotations before the claim binds; a
`phase: Succeeded` assert would only repeat `Ready`.

The golden image is a fixture the suite creates itself.
`vm-default-images` is an opt-in package and the e2e install does not
enable it, so `alpine-3.21` from the default collection is not there to
clone. The suite imports the same Alpine 3.21 image into `cozy-public`
under the name `e2e-alpine-3.21`, so it can never collide with an image
the package owns. This is the only download in the new test, and the
`source.disk` case reuses the first clone as its base disk.

The long wait sits on the DataVolume, not on the HelmRelease. Since
#4380 a vm-disk release goes Ready once the DataVolume exists and the
clone's progress is reported through the VMDisk `WorkloadsReady`
condition, so the test does not assert the HelmRelease at all. Chainsaw
deletes the clones before the golden image at cleanup.

The clone itself is only proven by this PR's E2E run, on the `local`
class of the PR lane.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff touches only `hack/e2e-chainsaw/vminstance/`, and nothing in
the trigger map covers it.

### Release note

```release-note
test(vminstance): e2e now covers VMDisk clones from a golden image (source.image) and from another disk (source.disk).
```
2026-10-05 15:14:38 +03:00
IvanHunters 4b1a75a332 test(vminstance): dry-run evict the VM launcher pod (#4585)
## What this PR does

Part of #4422.

The e2e suite now asks KubeVirt whether it would let a VM launcher pod
be evicted, without evicting it. Until now nothing in e2e evicted a
launcher pod, so the drain behaviour set up in #4403 was known only from
reading KubeVirt. Once the `vminstance` VM is running, the suite sends a
dry-run Eviction (`dryRun=All`) for its virt-launcher pod and checks the
answer. Nothing gets deleted.

A dry run still goes through the KubeVirt eviction webhook, which is
registered with `sideEffects: NoneOnDryRun` in v1.9.0. With
`LiveMigrateIfPossible` its answer follows the VMI's `LiveMigratable`
condition. That condition depends on storage, so the test reads it
instead of assuming it:

- `LiveMigratable=True`: the eviction must be refused with the webhook's
`Eviction triggered evacuation of VMI "<ns>/<name>"` message (HTTP 429).
A drain retries on that while the VM migrates away.
- `LiveMigratable=False`: the eviction must be allowed. The old
`LiveMigrate` value refused this case with "is not live-migratable".
KubeVirt v1.9.0 creates no PodDisruptionBudget for a VMI, so no PDB can
refuse it either.
- Anything else, including a missing condition, fails the test.

The allowed answer alone proves little, because the webhook also lets an
eviction through under `None`. So the test also asserts
`evictionStrategy: LiveMigrateIfPossible` in the VMI spec, where
virt-api copies the cluster value at creation. After the dry run it
checks that the launcher pod is the same object and not terminating, and
that the VMI has no `evacuationNodeName`.

On the PR lane only the `False` branch runs. Post-install prep pins
`StorageProfile/local` to RWO because DRBD is off there, and an RWO disk
makes the VMI non-migratable. Nightly and tag runs use `replicated`,
where I expect the `True` branch, but I have not seen this suite run on
that lane yet. A real drain and a workload update after a KubeVirt bump
are still open in #4422.

I ran the script against a stub `kubectl`. Both expected answers pass,
and it fails on every wrong answer I tried, such as a refusal for a
non-migratable VMI, a PDB-style 429, a missing condition, two launcher
pods, or a replaced pod after the dry run. `chainsaw lint` (v0.2.15) and
shellcheck pass.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff touches only `hack/e2e-chainsaw/vminstance/` and the suite
README, and nothing in the trigger map covers either.

### Release note

```release-note
test(vminstance): the e2e suite now dry-run evicts the VM launcher pod and checks the KubeVirt eviction decision against the VMI's LiveMigratable condition.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Added end-to-end coverage for dry-run eviction of a running virtual
machine’s launcher pod. The test checks the VM’s live-migration policy
and `LiveMigratable` condition, expecting eviction to be refused when
the VM is live-migratable and accepted otherwise.
* Verified that the dry run leaves the launcher pod unchanged and does
not record an evacuation node on the VM.
* **Documentation**
* Documented the test’s behavior and the local `jq` requirement for
running it.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-05 15:14:32 +03:00
Aleksei Sviridkin 9873b39a2a test(vminstance): cover VMDisk clones from an image and from a disk
Only source.http had e2e coverage, so a regression in either clone
path would ship unnoticed. The suite now clones a VMDisk from a golden
image in cozy-public and then clones that disk again, checking that
each DataVolume points at the expected PVC and becomes Ready, that its
PVC is Bound, and that each VMDisk reports WorkloadsReady. CDI derives
Ready from the DataVolume phase alone and can reach Succeeded from the
claim's annotations before the claim binds, so Ready does not stand in
for the binding.

vm-default-images is opt-in and the e2e cluster does not enable it, so
the suite creates its own small golden image under an e2e name rather
than relying on the package. The second clone reuses the first disk,
so the Alpine image is the only download.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 22:40:24 +03:00
Aleksei Sviridkin 966fccfb37 test(vminstance): dry-run evict the VM launcher pod
No suite evicts a virt-launcher pod, so the platform's eviction
behaviour is known only from reading KubeVirt. A server-side dry-run
eviction goes through the same KubeVirt admission webhook as a drain
and returns its decision without deleting anything.

The test reads the VMI's LiveMigratable condition rather than assuming
it, because the answer depends on the storage class: under
LiveMigrateIfPossible a migratable VMI is refused with the evacuation
message, and a non-migratable one is let through. It also pins the
evictionStrategy that virt-api stamped into the VMI spec, since the
webhook passes an eviction under strategy None as well, and checks that
the dry run left the launcher pod and the VMI untouched.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 15:01:18 +03:00
Aleksei Sviridkin d1aea3f40e test(e2e): assert vm-disk readiness through WorkloadsReady
The VMDisk scenarios waited on the HelmRelease, which does not cover
the import behind it. They now wait for the DataVolume to be Ready and
then for Ready and WorkloadsReady on the VMDisk itself. The DataVolume
wait keeps the combined budget the two old waits had.

That alone would pass with the monitor ignoring DataVolumes, since an
empty monitor reports operational. A new scenario imports from an
unresolvable URL and waits for Ready True with WorkloadsReady False,
which only a monitor that reads the DataVolume phase can produce. A
DataVolume that has not started is not ready either, so the scenario
first waits for the importer to fail, which CDI reports as Running
False with reason Error, and only then asserts the VMDisk.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 14:02:32 +03:00
Aleksei Sviridkin a7ac37be65 test(e2e): let HR-Ready waits outlast the install timeout of the release they wait on (#4471)
## What this PR does

Stacked on #4468, which holds the raised backup waits to a start margin;
review that one first. This PR's own diff is the six commits on top of
it.

Most HR-Ready waits in the E2E suites gave up before the release they
were waiting on could fail its install. cozystack-api gives every
release it generates a 600s install timeout, and the Chainsaw asserts in
front of those releases sat at 5 or 6 minutes. When an install genuinely
failed, the suite reported a bare assert timeout against a release that
was still installing, and the reason the install failed, which the
release records when its own timeout fires, was never reached. Red runs
failed just as often as they should; they only said less.

I raised every HR-Ready wait above the install timeout of its own
release, by the 60s start margin #4468 introduces:

- Chainsaw asserts on releases cozystack-api generates go to 11m, across
bucket, clickhouse, etcd, external-dns, foundationdb, harbor, mongodb,
openbao, opensearch, postgres, qdrant, rabbitmq, redis, valkey and
vminstance.
- Asserts on nested `<name>-system` releases take their floor from the
template that renders them: 11m for external-dns, openbao and qdrant
(10m `spec.timeout`), 16m for `harbor-test-system` (15m). `harbor-test`
sat exactly on its 600s floor, which expires first because the assert's
clock starts before the install's; it goes to 11m too.
- The kuberture Package release takes its 10m from cozystack-operator
and goes to 11m.
- In the backup walkthroughs, the bucket waits go from 300s to 660s in
clickhouse, kafka, mariadb, mongodb, postgres, rabbitmq and redis, and
the two postgres waits go from 360s to 660s. The kafka bucket wait
passed no budget and silently took the helper's 300s default, which is
why earlier scans for literal budgets missed it. Each Chainsaw op that
runs a walkthrough grows by exactly what its waits gained (clickhouse
78m, kafka 46m, mariadb 48m, mongodb 42m, postgres 66m, rabbitmq 48m,
redis 36m), so a failure at any later wait keeps the headroom it had and
is still reported by the script instead of being cut off by the op.

Two waits stay as they are, on purpose:

- kafka: `kafka-rd` sets a 7h install timeout, which no budget inside
the 215-minute E2E job can clear. The kafka assert and the Kafka waits
in the kafka and kafka-metadata walkthroughs keep their numbers, and the
assert now says why in-line.
- `vm-instance-test`: `vm-instance-rd` disables the install wait, so its
release goes Ready as soon as helm has applied and there is no floor to
clear. Earlier lists of this problem included it by mistake.

The cost falls on red runs only: a suite whose release never goes Ready
now spends up to six more minutes before it reports, and in exchange the
report carries the release's own reason. Green runs are unchanged.
Suites run one after another in a single job capped at 215 minutes, so a
systemic failure that reddens many suites now reaches that cap sooner,
and a job cancelled at the cap loses its log and `cozyreport.tgz`. I
took that trade because a failed install that says why is worth more
than a red run that ends a few minutes earlier.

`docs/agents/e2e-testing.md` stopped being true in several places once
the waits were raised: it described most of the tree as under the floor,
mariadb as the only suite raised, the walkthrough bucket waits as under
it, and harbor's asserts as sitting on and under theirs. I rewrote those
sentences, and the one that pointed at "every wait above the band" of
5-6m, kept the start-margin wording from #4468, and added that a
walkthrough wait passing no budget takes the helper's default. The
comments in `hack/mariadb-first-boot-waits.bats` that quoted the old op
and bucket figures are restated; its existing checks and constants are
unchanged, because every op grew by exactly its bucket delta and each
remainder still sits at exactly its constant. I also added postgres to
the table that file holds the walkthroughs to, so its raised waits and
its op cannot drift back apart unnoticed, and a test that holds each
table suite's bucket wait to the bucket release's floor, read from the
api-server default with `bucket-rd` checked for an override; the op
check alone could not catch a bucket wait dropping back, since a lower
wait only widens the slack. redis stays out: its walkthrough waits on no
redis HelmRelease, only on the bucket and on the sentinel master, so the
table's floor check has nothing to read there.

How I checked it, since E2E cannot run locally:

- I re-ran the enumeration from the sizing section of
`docs/agents/e2e-testing.md` over every
`hack/e2e-chainsaw/*/chainsaw-test.yaml` and every `wait_hr_ready` call
under `examples/backups/`, literal or default, and held each against its
own release's floor. Every `Ready=True` assert and every `wait_hr_ready`
call a suite drives now clears its floor, except the kafka ones named
above. The opt-in etcd round-trip uses bare `kubectl wait hr` calls at
300s instead of that helper; they are listed in #4470. The `Ready=False`
assert on the rejected redis upgrade is not a readiness wait and is left
out.
- `make bats-unit-mariadb-first-boot-waits` passes, 31 of 31. For the
new postgres row, putting the op back to 50m fails only the postgres
slack test, and a postgres wait at 659s fails only the postgres floor
test. Setting any one table suite's bucket wait to 659s fails only the
bucket test, naming only that suite, and an install-timeout override on
`bucket-rd` fails it too. I ran the mutations under plain `bats` so a
fail-fast runner could not hide a second failure.
- `hack/select-e2e.sh` over this diff selects bucket, clickhouse, etcd,
external-dns, foundationdb, harbor, kafka, kuberture, mariadb, mongodb,
openbao, opensearch, postgres, qdrant, rabbitmq, redis, valkey and
vminstance.

Out of scope, and tracked in #4470: the tenant addon waits in
`hack/e2e-chainsaw/_lib/run-kubernetes.sh` (an edit there escalates the
whole E2E run), the redis and kafka-metadata `wait_hr_ready` helpers
that print no reason on timeout, and the opt-in etcd round-trip's bare
`kubectl wait` calls.

Known and left for later, none of them changed here:

- Only the mariadb Chainsaw asserts and the walkthroughs in the guard's
table are held by a test. The other raised asserts, and the redis and
kafka bucket waits, rest on literals with in-line reasons; #4470 covers
extending the guard.
- The Kafka waits in `examples/backups/kafka/04-create-kafka.sh`,
`07-restore-to-copy.sh` and `kafka-metadata/run-all.sh` stay at 300s
against the 7h floor without an in-line reason at the call site. The
doc's rule asks that of Chainsaw asserts only, and the kafka Chainsaw
assert carries it.

The postgres waits are the ones #3748 is about. Their budget now clears
the install-timeout floor and a test holds it there; what a CNPG first
boot costs on this platform is still unmeasured, so that issue stays
open.

Closes #3771

Closes #3753

### Screenshots

Not a UI change.

### Downstream repositories

I walked the trigger map in `docs/agents/contributing.md` against the
diff. It touches E2E suites, backup walkthrough scripts, a unit-lane
bats file and an agent doc, none of which any downstream repository
restates.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(tests): HelmRelease readiness waits in the E2E suites and backup walkthroughs now outlast the install timeout of the release they wait on, so a failed install reports its own reason instead of a bare timeout.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Extended readiness and backup round-trip test timeouts across multiple
services to accommodate slower startup and provisioning.
* Added checks to verify that bucket readiness waits meet configured
installation-timeout requirements.

* **Documentation**
* Updated testing guidance to clarify how readiness waits relate to
release-specific timeouts and overall test budgets.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-02 12:15:58 +03:00
Aleksei Sviridkin 9e6cb7d0cc fix(tests): let HR-Ready asserts outlive the install they wait on
cozystack-api gives every release it generates a 600s install timeout,
and these asserts gave up at 5-6m. A failed install was therefore
reported as a bare assert timeout while the release was still
installing and had not yet recorded why it failed.

Raise them to 11m, above that floor, so the release's own reason is on
record before Chainsaw gives up. The cost is up to six more minutes on
a red run of each suite; a green run is unchanged. vm-instance-test
stays: vm-instance-rd disables the install wait, so its release goes
Ready as soon as helm has applied.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-01 17:40:21 +03:00
Myasnikov Daniil 7fe713580d test(e2e): drop readiness asserts that HelmRelease Ready already covers
The qdrant and external-dns apps render their workload in a nested
-system HelmRelease, and both suites already assert that HelmRelease
Ready, which helm-controller reports only once the workload is up.
Asserting readyReplicas on the same Deployment or StatefulSet again
adds nothing, so those asserts keep only the instance-label pin.

The vminstance PVC asserts follow a DataVolume Ready assert, and a
DataVolume is not Ready before its PVC is Bound.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-01 18:03:25 +05:00
Myasnikov Daniil 186547d835 test(e2e): run the merge-gating lanes on Talos containers
The srv1-srv3 nodes of both lanes that gate a merge -- the same-repo one
in pull-requests.yaml and the fork one in e2e-fork.yaml -- run as Talos
containers instead of QEMU guests, so a tenant worker sits at L2 rather
than L3. Measured in CI on commits where both substrates ran the same
tree: whole-job wall clock down 24-49 minutes, essentially all of it in
Chainsaw, while the two tenant-Kubernetes suites go from 1143-2040s to
466-809s and stop depending on the runner's vgif flag.

The node-join soft-red gate goes with the substrate, which is what it
was for. It existed because nested virtualisation on the shared runners
degraded far enough that a worker registering in minutes elsewhere could
miss any deadline this test could afford; #3513 established that the
discriminator was the host's kvm_amd vgif flag, and removing a nesting
level removes what it acted on. So hack/e2e-node-join-soft-red.sh, the
SOFT-RED-node-join.txt marker and the soft_red job outputs are gone from
all four workflows, and a missed deadline is an ordinary red again. The
::warning annotation survives, because the reason a run went red should
be legible without opening the log, and so does the 124-versus-everything
-else test, because "the workers were slow" and "the wait never ran"
send a reader to different places.

Done by swapping the substrate inside the existing e2e jobs rather than
by promoting a separate container job. Those jobs carry more than a
substrate -- the GitHub App token, the report-overrun guard, the images
list, the SSH breakpoint -- and promoting a job that never had them
would have dropped four features without saying so.

Three properties of container mode drive the design, and each fails
quietly rather than loudly. machine.kernel.modules is a silent no-op --
kernel_module_spec.go returns early on ModeContainer with no error and
no event -- so the host loads openvswitch and zfs before the nodes start
and the workflow step asserts both rather than trusting them; a missing
module surfaces an hour later looking like a CNI or a storage
regression. Docker gives a privileged container its own tmpfs /dev
seeded once at creation, so /dev/kvm works but ZFS zvols created later
never appear and linstor-csi fails ControllerPublishVolume; the compose
file binds the host devtmpfs, which is also why talosctl cluster create
docker can never carry LINSTOR. And there is no ARP VIP, so
192.168.123.11 stands in for the QEMU lane's .10.

The QEMU machinery stays in the tree and keeps running: nightly.yaml and
e2e-tag.yaml still call `make prepare-env`, so DRBD, the replicated
StorageClass, its Immediate binding mode, the tenant StorageClass
propagation that only applies to remotely-accessible classes, and the
Cozystack Talos node image with its extensions all keep nightly and
release-candidate coverage. None of them is exercised per pull request
any more. That is the trade.

Nothing on the PR path builds or downloads a nocloud disk now, so
build-talos drops the talos-nocloud step and the talos-image artifact.
build-talos itself stays: finalize consumes its digest fragment.

One property this does not fix, recorded beside the runner class because
it will be read off a red run eventually: a container node reports the
HOST's CPU and memory as capacity, and hack/e2e-container-up.sh counters
that with kubelet systemReserved. That corrects the scheduler and not
what a pod reads from /proc, so a workload sizing itself from
/proc/cpuinfo or /proc/meminfo is configured for the whole runner.

Also carried here because the same files carry them: the ghcr.io
pull-through mirror removal (see #4007 -- whichever lands first makes the
other a no-op), the authoritative HelmRelease readiness gate, the
backup-credential preflight, the prepull guard against an image-less
container fragment, and folding the two standalone OIDC suites into the
latest-version tenant cluster.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:19:17 +05:00
Myasnikov DaniilandClaude Opus 4.8 e22d84f2c8 test(e2e): migrate app suite from BATS to Chainsaw and wire into CI
Replaces the per-app BATS E2E suite (hack/e2e-apps/*.bats) with declarative
Kyverno Chainsaw tests under hack/e2e-chainsaw/, extending this branch's pilot
(postgres + bucket) to all 18 app suites.

- Declarative ports assert on status/conditions and concrete fields; the
  repeated "timeout ... until kubectl get" + "kubectl wait" boilerplate
  collapses into a single polling assert with structured failure diffs and
  automatic events/describe/podLogs capture on failure.
- Inherently imperative suites keep their logic in script steps: openbao
  init/unseal, kuberture split-horizon probes, vminstance, the gateway
  admission/impersonation cases, and kubernetes-latest/previous, which wrap
  the relocated hack/e2e-chainsaw/_lib/run-kubernetes.sh verbatim.
- Gateway tests derive the tenant apex from the namespace host label at
  runtime, so they are host-independent.

CI: the e2e job now runs "chainsaw test hack/e2e-chainsaw/" via a new
test-chainsaw target and uploads the JUnit chainsaw-report.xml; the chainsaw
binary is added to the e2e-sandbox image. install-cozystack and test-openapi
stay BATS. The per-app 3-retry loop is dropped, since assertion polling
replaces the fixed-timeout flakiness it papered over.

Verified against a dev cluster (DB/app, storage and VM suites pass). Service
port assertions use the ports[*].port projection form because Chainsaw
v0.2.15 mis-evaluates JMESPath number-literal port filters.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-25 13:42:34 +05:00