mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-08 21:14:56 +09:00
fix/keycloak-backup-system-image
589
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
6b1ad96093 |
fix(e2e): gate LINSTOR node-list probe on a ready controller endpoint (#3250)
## What this PR does The E2E `Install Cozystack` step runs `hack/e2e-post-install-prep.sh` in the background, which waits for 3 LINSTOR satellites to come Online by dialing the `linstor-controller` Service (`linstor+ssl://linstor-controller:3371`) from the controller pod. That probe was gated only by a one-shot `kubectl wait deployment/linstor-controller --for=condition=available`, which goes stale. The controller carries `reloader.stakater.com/auto=true` and mounts the cert-manager-issued API-TLS Secret (`packages/system/linstor/templates/cluster.yaml`), so when cert-manager (re)writes that Secret during bring-up, reloader rolls the single-replica controller. The roll drops the Service to zero ready endpoints *after* `Available` already passed, and a dial into that window fails with `[Errno 113] No route to host` (EHOSTUNREACH — the state of a ClusterIP whose Service has no ready backend). The blind `until` loop then churned the CLI on that error, tolerating a real un-Ready-controller race instead of gating on it. This gates every probe on the `linstor-controller` Service actually having a ready endpoint (a non-empty `Endpoints .subsets[].addresses`), so the CLI only dials a routable controller and a mid-loop reload re-waits for the endpoint instead of erroring. The `Online == 3` satellite-convergence check stays the real readiness assertion, and the dedicated 300s budget is preserved from the previous `timeout 300`; on timeout the loop now dumps the controller endpoints and pods for diagnostics before failing, per the E2E diagnostics convention. Evidence for the mechanism: the same single `Error: Unable to connect to linstor+ssl://linstor-controller:3371: [Errno 113] No route to host` line appears exactly once per install across multiple runs and is immediately recovered by the retry — consistent with one reloader-driven controller roll during initial bring-up, not a persistent outage. Scoped to the LINSTOR readiness path in `hack/e2e-post-install-prep.sh` only; no change to install ordering, dependencies, or timeouts. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved the LINSTOR startup readiness check to wait more reliably for the service to become reachable before verifying node status. * Made the node-online verification more resilient by handling transient connectivity errors and enforcing a clearer timeout. * Added extra diagnostic output when readiness does not complete in time, helping with troubleshooting failed installs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
6c9edf5038 |
fix(platform): route OIDC backend calls through the in-cluster Keycloak by default (#3289)
## What this PR does Defaults `authentication.oidc.keycloakInternalUrl` to `http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy` instead of an empty string. With the empty default, oauth2-proxy performed OIDC discovery against the external Keycloak hostname. That request resolves through public DNS and lands on the root ingress controller, which does not serve yet on a fresh install: the root tenant's IngressClass only exists once `tenant-root` is ready, and `tenant-root` is still reconciling while the dashboard starts. Discovery has nothing to talk to, the gatekeeper fails, and the dashboard never comes up — with no obvious diagnosis for a first-time installer. The bundled Keycloak is the only supported provider (the external hostname is templated as `keycloak.<host>` and the realm is fixed), so backend-to-backend calls have no reason to leave the cluster. The new default is the same Service address `keycloak-configure` already uses for its own backend calls. The dashboard and linstor-gui gatekeepers then run with `--skip-oidc-discovery` and send token, jwks, userinfo and logout requests straight to the Service, while browser-facing redirects stay on the external host. Both gatekeeper templates already branch on the value being non-empty, so neither needed a change. The value stays overridable for a Keycloak that does not live at the standard in-cluster address, and setting it back to `""` restores the previous external-discovery behaviour. Token validation is unaffected: Keycloak pins `KC_HOSTNAME` to the external URL, so tokens redeemed over the internal Service still carry the external `iss` and match the unchanged `--oidc-issuer-url`. ### Upgrade behaviour Existing OIDC clusters that never set `keycloakInternalUrl` pick up the new default on upgrade: their gatekeepers roll once and move to the internal path. Clusters that set the value explicitly — including to `""` — keep exactly what they configured. ### Security posture The backend leg (token redemption, which carries the gatekeeper's Keycloak client secret, plus jwks/userinfo/logout) now travels as plaintext HTTP between pods rather than HTTPS to the ingress. This is stated in the `values.yaml` comment rather than left implicit. The `keycloak-http` Service has no TLS listener, so in-cluster HTTPS is not an available alternative, and `keycloak-configure` already sends Keycloak *admin* credentials over that identical address — anyone positioned to read this traffic can already reach a strictly more powerful secret. The externally reachable front door (browser to oauth2-proxy) is unchanged. ### Testing helm-unittest coverage across all three affected charts: the default landing on the `_cluster` channel, both gatekeepers rendering the in-cluster backend URLs while keeping browser URLs external, an explicit override winning, the empty-string escape hatch still rendering the external-discovery shape, and OIDC-disabled clusters (now the common case of "OIDC off, internal URL set") ignoring the value entirely. The E2E OIDC case previously applied this exact URL as a manual workaround. It now sets only `oidc.enabled` and asserts the consequence: it waits for the `auth-proxy` container to appear in the gatekeeper Deployment, then gates on `kubectl rollout status`. Readiness alone would prove nothing there — the HelmRelease is still `Ready` from the token-proxy install, and with `replicas: 1` and `maxUnavailable: 25%` (which rounds down to 0) the Deployment keeps the old ReplicaSet `Available` while a new pod crashloops. Only the rollout catches a regression back to external discovery. Follow-up: the OIDC documentation in the website repo still frames `keycloakInternalUrl` as an optional tweak "for self-signed certificates or restricted external access". It is now the default path and needs a separate docs PR. Closes #3288 ### Screenshots No UI changes. ### Release note ```release-note fix(platform): `authentication.oidc.keycloakInternalUrl` now defaults to the in-cluster Keycloak Service (`http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy`). OIDC backend calls from the dashboard and linstor-gui gatekeepers no longer leave the cluster, so enabling OIDC on a fresh install no longer leaves the dashboard down while the root ingress is not yet serving. Existing clusters that never set the value pick up the new default and roll their gatekeepers once. Set the value explicitly to point at a non-standard Keycloak placement, or to an empty string to restore external OIDC discovery. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - OIDC-enabled applications now use the internal Keycloak service for backend authentication by default, improving reliability and avoiding unnecessary external discovery. - Browser redirects and login flows continue using the external Keycloak address. - Operators can provide a custom internal Keycloak URL or set it to an empty value to restore external OIDC discovery. - **Bug Fixes** - Dashboard and Linstor GUI authentication now correctly apply internal Keycloak settings only when OIDC is enabled. - Dashboard upgrades now wait for authentication components to become ready before completing. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
44507d2cd0 |
docs(agents): add downstream repository checklist to the PR template
Cozystack is upstream for repositories that nothing keeps in sync with it, and no CI job compares the two sides, so a change here breaks them in silence. Only two narrow paths are automated, and both fire on a release rather than on a PR. It has already happened. The website docs generator works from an app list hardcoded in its own Makefile, and packages/apps/opensearch was never added to it, so an application we ship has no reference page at all. The Terraform provider still offers a Kubernetes version this repo removed. The plugin repo still sends operators to hack/ scripts that were deleted, one of them rewritten as a Go command. Add a Downstream Repositories trigger map to the contributing guide, naming per repository which change here forces a change over there and which file to touch, and a checklist in the PR template that links to it. The template addresses AI agents directly: they author a large share of PRs and will otherwise tick a box without opening the follow-up it claims. The coupling is wider than the obvious ones, and mostly invisible. talm restates the node contract, overlapping ours but already diverged, so it must be diffed rather than overwritten. cozyhr reads an annotation the operator writes and silently merges the wrong values if it is renamed. cozy-proxy is vendored in from its own repo, yet consumes labels produced here and quietly stops handling external VM services if they move. The telemetry server queries metric names emitted here, and a rename just empties its dashboards. None of these fail loudly. Guard the map with a unit test, because a map about silent drift must not drift silently itself. It pins every in-repo path the map cites to a file that exists, pins the map and the checklist to the same repository list, and pins the workflow carve-out below to the map's location, to the job that reads the flag, and to the output that carries it: cutting any one of those three leaves the other two looking wired up. The header states plainly what the suite cannot check, so a green tick is not mistaken for a correct map: whether a coupling is real, and whether it still holds over there, stays a human's job. Editing the map alone is a docs-only PR, which skips the unit tests and would exempt the map from its own guard. Rather than widen the docs-only rule, which also gates the Talos image build, the plan step exports a separate flag that only the unit-test job reads: a docs PR now runs the tests and still builds nothing. It detects the map by its old path too, since a rename is reported as the new path alone. Document that gh pr create --body and --body-file replace the body wholesale, which would otherwise drop the checklist from every CLI-authored PR. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6f5889816b |
feat(ci): guard migrations.targetVersion against silent skip (#2771)
## Summary - Add `hack/check-migrations-target.sh` that fails when the latest numbered file under `packages/core/platform/images/migrations/migrations/` is not matched by a bumped `migrations.targetVersion` in `packages/core/platform/values.yaml`. - Wire it into `unit-tests` via a new `migrations-target-check` Makefile target so PR CI catches the regression mechanically. `run-migrations.sh:34` iterates `seq $CURRENT_VERSION $((TARGET_VERSION - 1))`, so a migration whose number equals the configured `targetVersion` is silently skipped. The cost is state corruption on upgrade — a class of bug worth eliminating with a 10-line check rather than relying on reviewer vigilance. ## Test plan - [x] `make migrations-target-check` passes on `main` HEAD (`targetVersion=43`, latest migration `42`, `43 >= 43`). - [x] Synthetic failure: `touch packages/core/platform/images/migrations/migrations/99` makes the script exit 1 with a clear error pointing at `packages/core/platform/values.yaml` and the required bump. - [x] Tested on macOS (BSD `ls` + mikefarah `yq`). ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Added a CI validation step to unit tests to ensure `migrations.targetVersion` is kept in sync with newly added database migrations, preventing upgrade scripts from skipping new migration steps. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a4bbc74b61 |
feat(agents): add md-no-hardwrap hook enforcing the prose rule
An ~80-column wrap is correct in Go comments and commit bodies, where coding agents spend most of their time, and the reflex leaks from there into markdown files and GitHub bodies. A documented rule alone does not survive that reflex, so check it mechanically at the moment of writing. The hook refuses a Write/Edit/MultiEdit that would add a hardwrapped paragraph, list item or blockquote to a markdown file, and a gh command that would publish one to GitHub via --body, --body-file, a heredoc, or an api body field. Judging what an edit ADDS is the whole difficulty. Comparing the text of each wrapped block refuses a typo fix inside a paragraph that was already wrapped; counting the blocks lets a file be rewritten as entirely fresh hardwrapped prose so long as the total does not rise. So the hook asks about the break rather than the block: a soft break is the edit's doing only when both lines it joins are lines the edit wrote. Whatever it cannot read with certainty, it allows -- a body the shell assembles at runtime, an unsplittable command line, a path that is not a regular file, a --body-file the same command has yet to write. A guard that blocks legitimate work gets switched off, and then it guards nothing. Vendored charts, published changelogs and Go templates are out of scope entirely, so hack/upload-releasenotes.sh keeps publishing its historical release notes. Line breaks are only rejected inside prose. Code blocks, tables, HTML, front matter, link reference definitions, footnotes, badge stacks and explicit hard breaks -- two trailing spaces or a trailing backslash, which render as <br> -- keep their breaks, because reflowing those would change the rendered output. A heredoc is attributed to the command that opened it, so a YAML manifest piped to kubectl is not read as a PR body. The hook and its registration are tracked by name so contributors get them on clone, while personal agent state stays ignored; personal overrides belong in .claude/settings.local.json. hack/md-no-hardwrap.bats covers both surfaces with 60 cases and make unit-tests picks it up automatically. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f24b56327b |
fix(platform): route OIDC backend calls through the in-cluster Keycloak by default
authentication.oidc.keycloakInternalUrl defaulted to an empty string, which left oauth2-proxy performing OIDC discovery against the external Keycloak hostname. That request resolves through public DNS and lands on the root ingress controller, which does not serve yet on a fresh install: the root tenant's IngressClass only exists once tenant-root is ready, and tenant-root is still reconciling while the dashboard starts. Discovery has nothing to talk to, the gatekeeper fails, and the dashboard never comes up. The bundled Keycloak is the only supported provider — the external hostname is templated as keycloak.<host> and the realm is fixed — so backend-to-backend calls have no reason to leave the cluster. Default the value to http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy, the same Service address keycloak-configure already uses. The dashboard and linstor-gui gatekeepers then run with --skip-oidc-discovery and send token, jwks, userinfo and logout requests to the Service, while browser redirects stay on the external host. Those calls travel as plaintext HTTP between pods, as keycloak-configure's admin calls to the same address already do. The value remains overridable for a Keycloak that does not live at the standard in-cluster address, and setting it back to an empty string restores external discovery. Existing OIDC clusters that never set it get a gatekeeper rollout onto the internal path on upgrade. Enabling OIDC swaps the dashboard's token-proxy container for oauth2-proxy, so the E2E install now drives that swap to completion: it waits for the auth-proxy container to appear, then gates on the gatekeeper rollout. Readiness alone proves nothing there — the HelmRelease is still Ready from the token-proxy install, and the Deployment keeps the old ReplicaSet Available while a new pod crashloops, so only the rollout catches a regression back to external discovery. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
IvanHunters
|
3380bb4ab6 |
refactor(hack): replace ls|grep with pure-bash glob (shellcheck SC2010)
Filenames in packages/core/platform/images/migrations/migrations/ are numeric-only by convention so the original `ls -1 | grep -E` filter was safe in practice, but the cleaner pure-bash form is equivalent and silences SC2010 without an annotation. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
c3d3e2596e |
feat(ci): guard migrations.targetVersion against silent skip
Adds hack/check-migrations-target.sh and wires it into unit-tests via the migrations-target-check Makefile target. The check fails when the highest numbered file under packages/core/platform/images/migrations/migrations/ is not matched by a bumped migrations.targetVersion in packages/core/platform/values.yaml. run-migrations.sh:34 loops `seq $CURRENT_VERSION $((TARGET_VERSION - 1))`, so a migration whose number equals targetVersion never runs and the intended state change is silently skipped on cluster upgrade. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
8af4ed543e |
test(e2e): address review nits — shellcheck directive + ports-workaround TODO
Non-blocking review notes from #2826: - Add `# shellcheck shell=bash` to the four sourced `_lib/*.sh` helpers (etcd-cleanup, remediation-guard, run-kubernetes, talos-image-cache) so shellcheck stops flagging SC2148 (no shell directive) — they are only ever `. sourced`, so there is no runtime interpreter to infer from a shebang. - Add a `# TODO(chainsaw>0.2.15)` marker at the postgres Service asserts explaining the `(ports[0].port)`/`(ports[*].port)` projection is a v0.2.15 number-literal-filter workaround (and that `ports[0]` checks only the first port), so it gets reverted on a chainsaw bump; cross-references the same workaround in kafka/mariadb/harbor and the e2e-testing.md gotchas section. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
ff0d0d77a7 |
test(e2e): make the talos image cache reachable by tenant workers (#3254)
The in-sandbox Talos image mirror (#3244) is unreachable by the tenant workers it serves, so the kubernetes-* e2e tests fail deterministically at the 12m node-join wait instead of relieving the public-factory flake (#3231). Root cause (from #3254 by @lexfrei, verified against the failing run's crust-gather snapshot): tenant namespaces run a default-deny Cilium egress (allow-internal/external-communication + the <tenant>-egress CCNP) that permits the world entity and the tenant tree but NOT an arbitrary kube-system Service, so a worker's CDI importer resolves talos-image-cache.kube-system.svc yet its TCP connect to the ClusterIP is silently dropped and no disk imports. #3244's gate never caught this because it probed the server over localhost inside the serve pod, which passes regardless of cross-pod reachability. Port of #3254 to the Chainsaw layout (hack/e2e-chainsaw/_lib/): - ship a tightly-scoped CiliumClusterwideNetworkPolicy (importer pods in tenant-test -> mirror pod in kube-system), applied at point-of-use once Cilium CRDs exist; excluded from the pre-Cilium install apply via yq. - replace the readiness probe with a throwaway pod in tenant-test labelled cdi.kubevirt.io=importer that curls the ClusterIP with a range request and must get 206 - the exact egress path a real importer faces; tenants use the mirror only when that end-to-end check passes, else fall back to the public factory, so it can only help, never make CI worse. - unit test pinning the manifest split, probe/policy label agreement, the strict 206 gate, and the --overrides JSON builder. Also adds (a2) DataVolume import-stage diagnostics to the node-join failure block in run-kubernetes.sh (importer logs + a ClusterIP re-probe via talos_image_cache_diagnose) so the import-stage failure mode is legible. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
1f84d2d49e |
Merge origin/main into feat/e2e-chainsaw-example (post-#3244)
Fold the in-sandbox Talos image cache work into the Chainsaw layout: the new shared helper relocates to hack/e2e-chainsaw/_lib/ talos-image-cache.sh and run-kubernetes.sh sources it from there (the ouroboros DNS-check no-retry rework merges clean). The OIDC bats are resolved as deleted again; their Chainsaw fixtures deliberately skip the talos.imageFactoryURL injection — the helper only emits it when the cache is up, and the zero-worker render-side tests instantiate no DataVolume imports for it to redirect. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
336f873b07 |
fix(e2e): serve the Talos image cache with a range-capable HTTP server
The talos-image-cache serve container ran `busybox httpd`, but the pinned alpine/k8s image ships no `httpd` applet and no standalone httpd, so the container exited 127 and crash-looped and the Deployment never became Available. The resolver then burned its full 12-minute rollout wait before falling back to the public factory — a fixed latency tax on the first kubernetes-* test while the flake the mirror targets stayed live. Serve the seeded image with a small range-capable server on the image's bundled python3 instead (stock `python3 -m http.server` ignores Range headers and answers 200, not 206). It needs no extra package pull, so the mirror comes up deterministically and keeps CDI's fast range import. The seed curl now also aborts and retries a stalled transfer (--speed-time/--speed-limit), and the resolver verifies a 206 range response before pointing tenants at the mirror. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f60fe2270b |
Merge remote-tracking branch 'origin/main' into feat/kubernetes-talos-image-factory-override
Signed-off-by: Aleksei Sviridkin <f@lex.la> # Conflicts: # packages/system/kubernetes-rd/cozyrds/kubernetes.yaml |
||
|
Aleksei Sviridkin
|
41eb6f912b |
fix(e2e): gate LINSTOR node-list probe on a ready controller endpoint
The post-install-prep node-list loop dialed the linstor-controller Service (linstor+ssl://linstor-controller:3371) as soon as the controller Deployment reported Available once. That gate goes stale: the controller carries reloader.stakater.com/auto=true and mounts the cert-manager-issued API-TLS Secret, so when cert-manager (re)writes that Secret during bring-up, reloader rolls the single-replica controller. The roll drops the Service to zero ready endpoints after Available already passed, and a dial into that window fails with "[Errno 113] No route to host" (EHOSTUNREACH). The blind until-loop then churned the CLI on that error instead of waiting for a routable controller. Gate every probe on the linstor-controller Service having a ready endpoint, so the CLI only dials a reachable controller and a mid-loop reload re-waits for the endpoint instead of erroring. The "Online == 3" satellite-convergence assertion and the dedicated 300s budget are preserved; a timeout now dumps controller endpoints and pods for diagnostics before failing. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
scooby87
|
6c69c0055e |
fix(monitoring)!: clean up orphaned storage PVCs and TLS secrets on tenant module delete (#3091) (#3094)
## What this PR does
When the `monitoring` tenant module is disabled, the
VictoriaMetrics/VictoriaLogs storage PVCs (created by vm-operator) and
the `alerta-tls` / `grafana-ingress-tls` cert-manager secrets were left
orphaned in the tenant namespace.
This adds a `post-delete` cleanup hook
(`packages/extra/monitoring/templates/hooks/cleanup.yaml`) that:
- deletes the VM/VL storage PVCs scoped to this release via `-l
apps.cozystack.io/application.name={{ .Release.Name }}-system`. That
label is stamped on the `storage.volumeClaimTemplate.metadata.labels` of
the VMCluster/VLCluster CRs (`packages/system/monitoring`); vm-operator
copies claim-template labels onto the PVCs (`IntoSTSVolume`), so the
selector matches and stays scoped to this monitoring release.
`managed-by=vm-operator` is intentionally **not** used — vm-operator
does not reliably stamp it onto the storage PVCs;
- deletes the `alerta-tls` and `grafana-ingress-tls` secrets by name
(cert-manager TLS secrets carry only the cluster-wide
`controller.cert-manager.io/fao=true` label, so a label sweep would be
unsafe — they are removed by name).
CNPG PostgreSQL PVCs (`alerta-db-*`, `grafana-db-*`) are intentionally
**not** targeted — CNPG cleans those up itself, and they do not carry
`apps.cozystack.io/application.name`, so the selector leaves them
untouched.
**Upgrade path:** the claim-template label only lands on PVCs created
after the change, so **migration 50**
(`packages/core/platform/images/migrations/migrations/50`) backfills
`apps.cozystack.io/application.name` onto the pre-existing VM/VL storage
PVCs of every cozystack-managed monitoring cluster, so the cleanup
reclaims storage on already-deployed clusters too.
A `test` Makefile target and helm-unittest suites are added for both the
cleanup hook and the claim-template label. The cleanup Job is hardened:
non-root, read-only root fs, dropped capabilities, `seccompProfile:
RuntimeDefault`, resource requests/limits, `activeDeadlineSeconds: 120`,
`backoffLimit: 0`, `--wait=false` on both deletes; RBAC scoped to PVC
get/list/delete and the two named secrets. Failures are logged but never
block teardown.
> [!WARNING]
> **Disabling the monitoring module now permanently deletes its
VictoriaMetrics/VictoriaLogs storage PVCs and the metrics/logs they
hold.** This is intentional and fixes the storage leak. Back up before
disabling if the data matters.
Fixes #3091.
### Release note
```release-note
fix(monitoring): garbage-collect the orphaned VictoriaMetrics/VictoriaLogs storage PVCs and the alerta/grafana TLS secrets left behind when a tenant's monitoring module is disabled; a migration backfills the release label onto pre-existing storage PVCs so already-deployed clusters are covered too. WARNING: disabling monitoring now permanently removes its storage PVCs and the data they hold.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Helm **post-delete** monitoring cleanup job to best-effort
remove managed PVCs and two specific TLS secrets.
* Included dedicated cleanup RBAC and hardened pod security settings for
the cleanup job.
* **Tests**
* Added Helm chart tests validating hook weight/policies, job settings,
exact deletion commands, and negative cases.
* Added a `make test` target to run Helm unit tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
a28ef9ba13 |
test(e2e): make the hairpin DNS check tolerate a tenant node recycle (#3241)
## What this PR does Makes the ouroboros hairpin DNS assertion in the `kubernetes-latest` e2e deterministic against a tenant worker-node recycle. This is a widespread pre-existing flake: it fails PRs that touch nothing virt-related and passes on re-run. ### Root cause (mechanism, from a failed run's own event stream) The assertion resolves `hairpin-*.example.invalid` from inside the tenant cluster using a single `--restart=Never` netshoot Pod, then waits for that Pod to reach `Succeeded`. The tenant crust-gather snapshot of a failed run shows the pod never ran the DNS loop at all: - `09:00:02` scheduler: `Successfully assigned default/dnscheck to ...-md0-vqcs7-hhfxx` - `09:00:06` kubelet on `hhfxx`: `Pulling image "nicolaka/netshoot:v0.13"` (no subsequent `Pulled`/`Started`) - node `hhfxx` then disappears from the tenant node list; on the management cluster its Machine is gone and a replacement Machine is created at `09:02:01` (`Provisioning`), while the other worker (created `08:43:39`) survives - `09:02:47` `taint-eviction-controller`: `Cancelling deletion of Pod default/dnscheck` The tenant's CAPI `MachineHealthCheck` remediates a node whose `Ready` goes `Unknown` for just 30s by deleting its Machine, KubeVirt VM and Node and provisioning a replacement. On the memory-constrained single-node CI sandbox a worker VM losing its kubelet heartbeat briefly is routine. A bare Pod bound to the deleted node is removed by the node controller and, with `restartPolicy: Never`, is never recreated — so the probe silently vanishes while its (large) image is still pulling and it has never left `Pending`. The single-shot wait then times out and reports `dnscheck pod did not reach Succeeded phase (last seen: <empty>)`, which is indistinguishable from a real DNS-resolution failure. ### The fix Wrap the probe create-and-wait in a bounded outer deadline (7m) that recreates the Pod when it is confirmed `NotFound` — the node-recycle signature — so the check is retried on a surviving node instead of failing the whole test. An empty phase is treated as ambiguous (a transient API blip is not misread as a lost Pod); only a definitive `NotFound` triggers a recreate. A Pod that actually runs to a non-`Succeeded` terminal state, or the deadline elapsing with no successful run, still fails the test. The in-Pod DNS resolution loop that the assertion exists to exercise is unchanged, so the reconciliation coverage is preserved. This fixes the mechanism at the right layer: the check tolerates a transient node recycle of a throwaway diagnostic pod rather than reporting it as a DNS regression. It is deliberately not a timeout bump — the pod was actively deleted, not merely slow, so more time alone would not help. ### Out of scope The `kubernetes-previous` failure seen in the same run is a different mechanism: both tenant worker VMs stayed `Provisioning` for 12m (DataVolume/CDI storage-import stall on the management cluster), so no tenant node ever joined and the run fails at the existing node-join diagnostic block, well before this assertion. Both smell like the same underlying single-node-sandbox resource/storage pressure, but that is an infrastructure-capacity concern, not a harness race this change can fix. Left for a separate follow-up. ### Screenshots N/A — CI/test-harness change, no UI impact. ### Release note ```release-note test(e2e): recreate the tenant hairpin DNS probe pod on a bounded deadline so a worker-node recycle mid-check is retried instead of reported as a spurious DNS failure ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved reliability of the Kubernetes end-to-end DNS check. * The DNS validation now retries more safely during infrastructure-related pod restarts or node churn. * Failures now surface faster with clearer diagnostic output when the check does not pass. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Алексей Артамонов
|
ff037c04d3 |
test(monitoring): reproduce the real vmselect cache PVC name in the backfill test
/branch-review flagged that the migration comment and its bats test used the simplified vmselect cache PVC name cache-vmselect-<cr>-<ordinal>, while the real name is vmselect-cachedir-vmselect-<cr>-<ordinal>. The anchored regex matches both (it suffix-matches <component>-<cr>-<ordinal>), so behaviour was correct, but the test did not exercise the production name. Feed the realistic name in the fake kubectl so the regex is validated against what vm-operator actually provisions, and reword the migration comment to describe the suffix match generically instead of asserting a specific claim-template prefix. Signed-off-by: Алексей Артамонов <aleksei.artamonov@aenix.io> Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Алексей Артамонов
|
09c422562a |
test(monitoring): unit-test the VM/VL PVC backfill migration behaviour
helm-unittest only inspects rendered manifests, not the operator's runtime PVC labels, so it cannot prove migration 50 actually relabels the right PVCs — the exact no-op/mis-scope class of bug this PR iterated on. Add a bats test (fake kubectl on PATH, mirroring hack/kubernetes-md0-migration.bats) that pins the runtime behaviour: it relabels the monitoring cluster's vmstorage/vmselect/ vlstorage PVCs with application.name=monitoring-system, never touches the CNPG db PVCs or a user's own VMCluster, and stays a safe version-stamping no-op when the VM/VL CRDs are absent. Signed-off-by: Алексей Артамонов <aleksei.artamonov@aenix.io> Assisted-By: Claude <noreply@anthropic.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
dea35f7432 |
Merge origin/main into feat/e2e-chainsaw-example (post-#3233)
Main's #3233 landed mid-reconciliation and touched the two OIDC bats again: nodeGroups (and storageClass) dropped from the render-side test CRs — the schema default {} now renders one zero-replica MachineDeployment via CAPI defaulting, so the tests provision zero worker DataVolumes (#3231). Resolve the modify/delete as deleted and mirror the fixture change (and its rationale) into the ported kubernetes-oidc-system / kubernetes-oidc-customconfig Chainsaw suites. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
IvanHunters
|
0c7b598304 |
fix(apps/kubernetes): honour nodeGroup minReplicas in MachineDeployment.spec.replicas (#3233)
Refs #3231 (parts 1 + 2 addressed here; part 3 — intra-file `cozy_wait_linstor_pool_free` gate + orphan `default-chk-clickhouse-test-keeper-cluster1-0-2-0` PVC cleanup — deferred to a separate follow-up PR against `hack/e2e-apps/run-kubernetes.sh`). **Do not auto-close the issue on merge** — the hardening work is a distinct concern and belongs on its own tracker entry. ## Root cause chain (recap from the issue) `packages/apps/kubernetes/templates/cluster.yaml` hardcoded `MachineDeployment.spec.replicas: 2` regardless of the nodeGroup's `minReplicas`. The two render-side `kubernetes-oidc-{system,customconfig}` bats set `minReplicas: 0` and delete their `Kubernetes` CR without waiting for readiness — but the chart still spun up 2 × 20 GiB `replicated` (DRBD) worker `DataVolume`s each. Their create/delete churn overlapped `kubernetes-previous`, which then saw slow LINSTOR provisioning (my cozyreport from run 28843794670 shows a ~6-minute gap between the `Kubernetes` CR being created and the CDI `ExternalProvisioning` event actually firing for the worker root disk); the tenant node registered late; Cilium's CNI init missed the 12-minute node-join deadline; and the test failed with `cni plugin not initialized`. ## Changes Two edits, applied together — the chart edit is a no-op without the test edit, and vice-versa (dropping `nodeGroups` falls back to the chart helper's default `md0`, which itself has `minReplicas: 0`, so both pieces are needed for zero workers to actually render). ### 1. `packages/apps/kubernetes/templates/cluster.yaml` ```diff spec: clusterName: {{ $.Release.Name }} - replicas: 2 ``` The field is **omitted entirely** so the cluster-autoscaler owns it alone. CAPI's MachineDeployment defaulting webhook seeds `spec.replicas` from the `cluster-api-autoscaler-node-group-min-size` annotation on Create (documented at https://cluster-api.sigs.k8s.io/tasks/automated-machine-management/autoscaling — *"if it's a new MachineDeployment or MachineSet, use min size"*), and Server-Side-Apply field-manager ownership then keeps chart upgrades from overwriting whatever value the autoscaler patched in the meantime. Emitting the field from the chart at all — even with the semantically-correct `{{ $group.minReplicas }}` — would keep the helm-controller ↔ cluster-autoscaler field-ownership fight alive: helm-controller applies with SSA + force-conflicts (https://fluxcd.io/flux/components/helm/helmreleases/), so every routine `helm upgrade` re-asserts the chart's chosen value and drains whatever the autoscaler had patched in the meantime. ### 2. `hack/e2e-apps/kubernetes-oidc-{system,customconfig}.bats` Drop the explicit `nodeGroups:` block and the paired `storageClass:` from the render-side CR bodies. The tests only assert on the KamajiControlPlane and the cozy-realm Keycloak objects — worker plumbing was never part of their coverage. After the chart fix, the chart helper's default `md0` (with `minReplicas: 0`) renders a MachineDeployment with `spec.replicas: 0` — the resource is still emitted for API-surface completeness but never creates a KubevirtMachine, never provisions a DataVolume, and never touches DRBD. ## Verification ### helm-unittest ``` Charts: 1 passed, 1 total Test Suites: 16 passed, 16 total Tests: 171 passed, 171 total ``` No suite asserted on `MachineDeployment.spec.replicas` directly, so nothing needed updating. ### helm template — verify the omit With `tests/values/common.yaml` (which has `minReplicas: 0`), the rendered MD carries the `cluster-api-autoscaler-node-group-min-size: "0"` annotation and **no `spec.replicas` field**. CAPI's defaulting webhook will fill in `spec.replicas: 0` on Create — no worker DataVolumes provisioned until scale-out. With a user CR that sets `minReplicas: 2` (the shape the `kubernetes-latest` / `kubernetes-previous` bats CRs use), the rendered MD carries the `cluster-api-autoscaler-node-group-min-size: "2"` annotation and again **no `spec.replicas` field**. CAPI's defaulting webhook will fill in `spec.replicas: 2` on Create — byte-identical topology to the pre-PR chart's fresh-install state (2 workers), but without the recurring drain-on-upgrade cycle. ### Render-side render walk `helm template` of the OIDC bats CR bodies (after this PR): 1 MachineDeployment with **no chart-managed `spec.replicas`** and `min-size: "0"` on the autoscaler annotation. CAPI seeds `replicas: 0` on Create, 0 KubevirtMachines emitted, 0 DataVolume provisioning triggered. Test assertions on KamajiControlPlane / KeycloakClient / KeycloakClientScope / AuthenticationConfiguration Secret are unchanged. ## Upgrade impact Existing customers stop losing workers on `helm upgrade`. Prior to this PR the chart re-asserted `MachineDeployment.spec.replicas: 2` on every Helm action, so a customer running with the autoscaler at, say, `replicas: 8` would see 6 workers drained on every routine Cozystack platform bump and then wait for the autoscaler to scale back up in response to unschedulable Pods. With `spec.replicas` no longer emitted by the chart, SSA field-manager ownership preserves the autoscaler's value across upgrades unconditionally. Fresh installs with `nodeGroups: {}` (the schema default) transition from "2 workers by default" to "0 workers until scale-out". The chart helper's default md0 carries `minReplicas: 0`, which is what CAPI's defaulting webhook seeds `spec.replicas` from. The `values.yaml` / README / helper docstring / regenerated Go type doc + openAPISchema call this out explicitly; the release-note repeats it below. See finding 5 in the paired self-review for the specific `addons.ingressNginx.enabled: true` interaction operators need to be aware of on fresh installs. ## Test plan - [x] `helm unittest packages/apps/kubernetes/` — 171/171 passing. - [x] `helm template` with `tests/values/common.yaml` (minReplicas: 0) — MD.replicas: 0. - [x] `helm template` with a `minReplicas: 2` override — MD.replicas: 2 (byte-identical to pre-PR). - [ ] E2E: `kubernetes-latest` unchanged (spawns its two workers as before), `kubernetes-oidc-{system,customconfig}` render zero workers, `kubernetes-previous` no longer flakes on LINSTOR pool pressure originating from the earlier OIDC suite. ## Notes - Part 3 of #3231 (intra-file `cozy_wait_linstor_pool_free` gate between kube sub-tests and the orphan clickhouse keeper PVC cleanup) is **not** in this PR — worth a separate targeted change to `hack/e2e-apps/run-kubernetes.sh`, and it's a hardening-only improvement rather than a root-cause fix. Happy to send it as a follow-up if wanted. ## Release note ```release-note fix(apps/kubernetes): stop the chart from managing `MachineDeployment.spec.replicas`; the cluster-autoscaler owns the field alone. CAPI's defaulting webhook now seeds `spec.replicas` from the `cluster-api-autoscaler-node-group-min-size` annotation on Create, and SSA field-manager ownership keeps subsequent `helm upgrade` actions from overwriting whatever value the autoscaler patched. Two behavioural changes worth calling out: - Existing customers: `helm upgrade` no longer drains workers on every platform bump — the autoscaler-held `spec.replicas` is preserved unconditionally. Prior to this change the chart hardcoded `replicas: 2`, which drained back to 2 on every reconcile action. - Fresh installs with `nodeGroups: {}` (the schema default) no longer come up with two workers by default. The chart helper's default md0 carries `minReplicas: 0`, and CAPI's defaulting webhook seeds `spec.replicas` from that. Operators enabling `addons.ingressNginx.enabled: true` on such a CR must either supply an explicit nodeGroup with `roles: [ingress-nginx]` and `minReplicas >= 1`, or accept that the cluster-autoscaler will bring the default md0 up in response to the ingress-nginx controller Pods becoming Pending on install. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Rendered `MachineDeployment` manifests no longer set `spec.replicas`, avoiding chart upgrades overwriting autoscaler-managed sizing. * **Documentation** * Clarified `nodeGroups: {}` behavior: a default `md0` is emitted with `minReplicas: 0`/`ingress-nginx` role, and workers are provisioned only after an autoscaler-triggering unschedulable Pod or manual scaling; updated related storage and ingress guidance. * **Tests** * Strengthened default node group rendering assertions and adjusted OIDC e2e manifests to reduce CI provisioning churn. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Fable 5
|
1f9555a3c6 |
test(e2e): resolve outstanding bot review findings
- redis: assert both StatefulSet-templated PVCs Bound (replicas: 2 provisions two, only ordinal 0 was checked) - gateway: || true each delete inside the three cleanup() traps — they run under set -eu, so one transient delete failure skipped the remaining cleanup commands - bucket, backup(.disabled): guard the trap's kill against an unset pf_pid (script can exit before the port-forward starts) Not addressed on purpose: the sort(ports[*].port) suggestions on clickhouse/kafka — sort() is among the JMESPath forms chainsaw v0.2.15 (still the latest release) mis-evaluates, which is why the suites use the ordered (ports[*].port) projection; services keep spec order. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
dd2f89d208 |
test(sdn): port the securitygroup and serviceexposure suites to Chainsaw
Main added securitygroup.bats (#2922 membership model) and serviceexposure.bats (#3081 ExposureClass/ServiceExposure) after the Chainsaw migration branch was cut. - securitygroup: two Tests — the aggregated-API projection round-trip (backing CNP marker label, empty-value membership endpointSelector, lineage fromApp + membership fromSG peers, 1:1 rule translation, the view hiding storage internals, unmarked-CNP invisibility via an error op, delete propagation) and the controller membership-label stamp/clear lifecycle. Only the attachments annotation stays a jq script (it is a JSON string). Chainsaw auto-cleanup covers every fixture even on mid-test failure — the bats had no teardown hook and leaked its fixtures. - serviceexposure: the externalIPs backend resolve/Ready/assignedIPs path with the no-MetalLB-pool negative as an error op (ordered after resolvedBackend so it cannot false-pass), and the finalizer-removal test collapsed onto the delete op itself, which waits for full disappearance. - selection: kubernetes-application now also maps to the two OIDC render-side suites (they exercise the same app chart — a gap main's bats selection had too); the new dirs enter all_apps discovery automatically. Selector unit tests extended and green (13/13 under cozytest). - README: suites list updated. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
cac07db8fc |
test(kubernetes): port the Phase-1 OIDC render-side suites to Chainsaw
Main added kubernetes-oidc-system.bats and kubernetes-oidc-customconfig.bats after the Chainsaw migration branch was cut; without a port the 260 lines of coverage would vanish with the hack/e2e-apps/ deletion. Both are light cluster e2e (apply CR -> assert rendered objects; no Kamaji bringup, minReplicas: 0), exactly Chainsaw's declarative shape: - System mode: KamajiControlPlane carries --authentication-config + the chart-owned volume/mount; the AuthenticationConfiguration Secret carries the cozy-realm issuer (regex, script step) + per-cluster audience; per-cluster KeycloakClient/KeycloakClientScope land in the cozy realm - CustomConfig: the flag is still wired; the Secret carries the tenant-supplied issuer and NOT the cozy realm (fail-loud negation); no Keycloak objects and no pre-rendered OIDC kubeconfig Secret, expressed as error ops ordered after the positive asserts so an unreconciled HR cannot false-pass them The port also sheds two latent bats bugs: cozytest defines no skip() (the Keycloak guard would have died with exit 127 had it ever fired) and the api-resources group probe exits 0 even when the group is absent — the chart capability-gates Keycloak rendering and CI always ships the operator, so the asserts are unconditional here. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
6c76a462a0 |
test(e2e): fold remaining post-branch bats drift into the Chainsaw suites
- qdrant: add the post-delete PVC-reclaim guard (#3059) — delete the CR and poll the ordinal-anchored data PVC to absence via the error op (retries transient API errors internally), with a PVC/jobs/pods dump on failure; Chainsaw auto-cleanup alone would never notice a leaked StatefulSet-templated PVC - kuberture: make the split-horizon negative log assertions fail loud — under set -e a pipeline beginning with ! suppresses errexit, so both leak checks silently passed (the same vacuous-negation bug main fixed in |
||
|
Myasnikov DaniilandClaude Fable 5
|
d6ac62a6d9 |
test(bucket,harbor): fail fast on frozen BucketClaims; enforce readonly denial
Fold main's post-branch bucket/harbor bats drift into the Chainsaw suites: - BucketClaim bucketReady budgets tightened to 2m (bucket had the 5m default, harbor a stale 10m with the reversed 'can lag on a loaded runner' rationale): the objectstorage-controller requeues the claim until the backend Bucket is ready ( |
||
|
Myasnikov DaniilandClaude Fable 5
|
21e21a9234 |
test(kafka): raise fixture presets off the 128Mi OOM threshold
Main raised kafka/zookeeper resourcesPreset nano -> c1.small
(
|
||
|
Myasnikov DaniilandClaude Fable 5
|
7b557db59f |
test(mariadb): port the single-replica webhook-regression guard to Chainsaw
Main's mariadb.bats gained "Create single-replica MariaDB" after the
Chainsaw port was cut: replication is guarded on replicas>1
(
|
||
|
Myasnikov DaniilandClaude Fable 5
|
c8e775d2d0 |
test(gateway): port the 12 ingress-hostname-policy cases to Chainsaw
Main added cozystack-ingress-hostname-policy (
|
||
|
Myasnikov DaniilandClaude Opus 4.8
|
47e7d224d8 |
test(e2e): serve Talos worker OS image from an in-sandbox cache
The kubernetes-* e2e tests fail intermittently and run 18-22 min each because every tenant worker VM imports its Talos OS disk directly from the public factory.talos.dev: that endpoint has no byte-range support (forcing CDI into a full copy-to-scratch) and its egress stream-resets / stalls mid-transfer from the CI runner, so a worker DataVolume import hangs past the chart's 12-minute node-join deadline and the test fails with zero tenant nodes joined (observed on run 28924455578, and on the unrelated branch in run 28929095889 - i.e. not specific to any one PR). Deploy a best-effort in-sandbox mirror (hack/e2e-talos-image-cache.yaml) during install: an initContainer fetches the image once with an aggressive curl retry loop, then busybox httpd serves it locally with range support. Tenant Kubernetes CRs point spec.talos.imageFactoryURL at its Service via a shared helper (hack/e2e-apps/talos-image-cache.sh) that gates on the mirror being Available and falls back to the public factory otherwise, so the mirror can only help, never make CI worse. Deployed early so the seed overlaps the Cozystack install; readiness is resolved once and cached across bats files. Uses the overridable chart value added in the previous commit. Refs #3231 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
ece64ee0a9 |
test(etcd): re-port the Chainsaw suite to the v1alpha2 operator contracts
The suite was ported from the pre-#2859 etcd.bats and asserted etcd.aenix.io/v1alpha1 objects that no longer exist on main — every Test hard-failed at the first assert on an unknown GVK. - drop the etcd-empty-backup and etcd-backup-schedule scenarios with their fixtures: the chart no longer renders EtcdBackupSchedule or etcd-s3-creds (backups moved to the generic backups API), and legacy-values tolerance is pinned by helm-unittest, not e2e - rewrite the remaining scenario into etcd-1-contracts, mirroring main's contracts test: gate on status.readyMembers=3 (Available=True flips at seed-member discovery and races replica-count contracts), assert the operator-owned member Pod shape (strict selector, container name, named metrics port), WorkloadMonitor operational/availableReplicas, and script the three contracts that need imperative probes: the /scale subresource, the plaintext-HTTP metrics probe against the Pod IP, and the on-demand defrag Job - add etcd-2-backup-roundtrip driving examples/backups/etcd/run-all.sh, gated on ETCD_E2E_S3_ROUNDTRIP=1 exactly like main (kind CI has no publicly-trusted S3 endpoint the snapshot agent could verify) - upgrade catch blocks to the v1alpha2 GVKs plus EtcdMember/ EtcdSnapshot/BackupJob describes and the snapshot-Job Pod logs - strengthen _lib/etcd-cleanup.sh with main's etcd_drain deltas: probe the Etcd CR itself (it can linger Terminating and block the next singleton apply) and use the err-sentinel pattern so a transient API error is never misread as "deleted"; add etcd_backup_cleanup wrapping examples/backups/etcd/cleanup.sh for the round-trip's teardown Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
efbf372d25 |
test(e2e): fix post-merge seams the conflict markers missed
- nightly.yaml still looped over hack/e2e-apps/*.bats with the removed test-apps-% target; switch it to the single chainsaw invocation (empty CHAINSAW_SUITES = full suite) and upload the JUnit report - pin the chainsaw-report upload-artifact step to the same SHA the rest of the workflows use (scorecard/zizmor gate) - repoint hack/run-kubernetes-drain_test.bats at the relocated hack/e2e-chainsaw/_lib/run-kubernetes.sh - replicate cozytest.sh's NotReady-pod data-plane capture (hack/e2e-capture-dataplane.sh) in the Chainsaw global error.catch so chainsaw-suite failures keep diagnostic parity with the BATS lanes - refresh stale bats path comments Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
9297e51170 |
Merge origin/main into feat/e2e-chainsaw-example
Reconcile the Chainsaw e2e migration with ~500 commits of main drift: - keep main's run-kubernetes.sh helpers (kubectl_wait_retry, tenant drain, LINSTOR pool-free wait, SC-fallback probe) under the branch's _lib/ rename with the Chainsaw finally/snapshot deltas re-applied - resolve modify/delete on 10 drifted hack/e2e-apps/*.bats as deleted; their post-branch assertion drift is re-ported into the Chainsaw suites in follow-up commits - drop the 4 new main-side bats (kubernetes-oidc-*, securitygroup, serviceexposure) from _lib/ relocation; ported as Chainsaw suites in follow-up commits - docs/agents/e2e-testing.md: merge main's plan-job TIA wording with the Chainsaw suite phrasing Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
699e429187 |
test(e2e): make the tenant hairpin DNS check tolerate a node recycle
The ouroboros hairpin assertion resolves a name from inside the tenant cluster via a single `--restart=Never` netshoot Pod and waits for it to reach Succeeded. On the memory-constrained single-node sandbox a tenant worker node is routinely recycled mid-test: its kubelet heartbeat lapses, the tenant's CAPI MachineHealthCheck marks Ready=Unknown after 30s and deletes the Machine, its KubeVirt VM and the Node, then provisions a replacement. A bare Pod bound to the deleted node is removed by the node controller and, having restartPolicy Never, is never recreated -- so the probe silently vanishes, typically while its large image is still pulling and it has never left Pending. The single-shot wait then times out and reports "dnscheck pod did not reach Succeeded phase (last seen: <empty>)", which is indistinguishable from a real DNS failure and flakes PRs that do not touch virt at all. Keep the DNS resolution asserted exactly once and fail-fast: a probe that runs to phase Failed is the reconciliation regression this test exists to catch and fails the test immediately. Recreate only the probe Pod, and only for the pure-infrastructure node recycle -- when the Pod vanishes and the node it ran on is confirmed gone or no longer Ready, retry on a surviving node within a bounded deadline. A probe that disappears while its node is still Ready is an unexpected deletion and fails loud rather than being retried, so no pod-churn regression is masked, and a transient API error while polling is not mistaken for a deleted Pod. The in-Pod dig loop that the assertion exercises is unchanged. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
IvanHunters
|
6ac52a108a |
docs(e2e): correct OIDC bats comments to match post-fix render behaviour
Both nodeGroups-omitted comments overstated the outcome as "zero
MachineDeployments render", but the chart helper (kubernetes.nodeGroups)
emits a default md0 group whenever the caller's nodeGroups map is
empty, so exactly one MachineDeployment renders. What is actually
zero is the number of KubevirtMachines / worker DataVolumes, because
the chart no longer manages spec.replicas and CAPI's defaulting
webhook seeds it to 0 from the autoscaler min-size annotation.
kubernetes-oidc-system.bats additionally described the bug in the
present tense ("chart hardcodes spec.replicas: 2 ... see #3231"),
which contradicts the sibling customconfig comment and would read
as an outstanding defect post-merge — rewrite it around the post-fix
rationale to match customconfig's shape.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
|
||
|
myasnikovdaniil
|
cece5c23b5 |
feat(release): immutable tags & rc→stable promotion (#2677) (#3017)
## What this PR does Implements the immutable-tag + rc-promotion flow from #2677. A **stable release becomes a renamed release-candidate**: the bytes shipped as `vX.Y.Z` are bit-for-bit the bytes built and e2e-tested as `vX.Y.Z-rc.N`. No tag is ever force-moved, and stable is never rebuilt — it is *promoted* by retagging the rc's existing images. Layered onto the build matrix from #2937/#2983 (this PR is **stacked on `refactor/build-matrix-2937`** and must merge after it). ## The five force-retag sites, removed | Site | Before | After | |---|---|---| | `tags.yaml` api/apps/v1alpha1 tag | `git tag -f` / `push -f` | write-once (create-if-absent, fail if it would move) | | `tags.yaml` release-X.Y.Z branch | `git branch -f` / `push -f` | compare-before-force (no-op if unchanged; staging branch only) | | `pull-requests-release.yaml` stable tag | `git tag -f` / `push -f` | write-once at the PR merge commit (force impossible by construction) | | `pull-requests-release.yaml` maintenance branch | `updateRef force:true` | fast-forward-only | | `auto-release.yaml` patch tags | delete-recreate (cron) | **workflow deleted** — stable only via explicit promote | ## Version decoupling (the enabler) The operator baked its version into the image at build time, so an rc image self-reported the rc string — blocking retag-promotion. It now reads `COZYSTACK_VERSION` from the environment (threaded via `cozystackOperator.platformVersion` → Deployment env, stamped by `make manifests`), falling back to the build-time value when unset. The same image bits can report any release name. The only runtime reader is the telemetry metric `cozy_cluster_info{cozystack_version=...}`. ## Promotion flow `promote-rc.yaml` (`workflow_dispatch`, `rc_tag=vX.Y.Z-rc.N`): 1. Validate the rc release exists and the stable tag does not. 2. `hack/promote-retag.sh` reads the rc's digest-pinned image refs from `packages/*/*/values.yaml` and `skopeo copy`s each — **by digest** — to `:vX.Y.Z` and `:latest`, verifying with `skopeo inspect`. 3. Rewrite the cosmetic `-rc.N` substring in vendored tags to the stable version (the `@sha256` wins regardless), restamp the version-stamped assets (operator manifests, cozypkg, openapi; the heavy Talos assets are copied verbatim from the rc draft), open a `release-X.Y.Z` PR. 4. Merging the PR reuses the existing release-PR e2e and `pull-requests-release.yaml` finalize to cut the **write-once** stable tag at the merge commit and publish the release. Squash is disallowed (the tag needs a real merge commit). `nightly.yaml` cuts write-once `*-nightly.<date>` tags (gated by `NIGHTLY_ENABLED`); `retention.yaml` keeps the newest 14 per line (dry-run by default). ## Validation **Local end-to-end — the core claim is proven, not asserted.** The whole design rests on "copy-by-digest to a new tag preserves the digest, so stable == the e2e-tested rc, bit-for-bit." This was exercised against a real registry (`ttl.sh`) with real `skopeo`: - Pushed a multi-arch image under a `:v1.4.0-rc.2` tag, built a temp tree exercising all three `values.yaml` digest shapes (single `image:` string, split `repository`/`tag`/`digest` map, and the `platformSourceRef` OCI artifact), then ran the actual `hack/promote-retag.sh v1.4.0`. - **Result: `:v1.4.0-rc.2`, `:v1.4.0`, and `:latest` all resolve to the identical digest** (`sha256:fd8d9aa6…`), across all 17 platform manifests. The script's own `skopeo inspect` post-check passed, and an independent re-inspection confirmed it. Local validation caught (and this PR fixes) two bugs that only surface against a real registry — never in static lint: 1. `skopeo copy --multi-arch all --all` — mutually exclusive flags, fatal error; every retag would have failed on CI. Now `--multi-arch all`. 2. The same digest was retagged twice when an image appeared in two value shapes; the ref set is now deduped on the canonical `repo@digest`. **Other local checks:** `go test ./pkg/version` + `go build`/`vet`; `helm template` renders `COZYSTACK_VERSION` in all 3 operator variants (omitted when unset); `make manifests` stamps the version into the install assets; `shellcheck` clean on `promote-retag.sh`; `actionlint` clean and `act -l` resolves the job graph on all workflows; the rc-tag parser and nightly version-math unit-tested (accept/reject + minor-vs-patch bumps). **Still CI-only (cannot be exercised offline):** the full workflow runtime — github-script API calls, OCIR auth, the rc-release / staging-branch preconditions, the nightly `base_ref` push mechanics, and self-hosted runners. The *logic* inside the steps is unit-tested; the *orchestration* is not. Treat it as unproven until the workflows run. ## Out of scope / required follow-up (needs repo admin — not doable from a PR) - Tag-protection rules: `v*` and `api/apps/v1alpha1/*` create-only by the CI app, no delete/update; "limit branches/tags updated in a single push" = 1. - Enable `NIGHTLY_ENABLED` (and `RETENTION_APPLY` when ready) repo variables. - Registry-side pruning of `*-nightly.*` image tags (no OCIR delete parity yet — tracked TODO in `retention.yaml`). ## Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added nightly publishing (mirror-by-digest, disk build, e2e validation) and nightly retention pruning. * Added RC-to-stable promotion via digest retagging (no rebuild). * Operator/installer and console now expose version/platform metadata, including runtime override via `COZYSTACK_VERSION`. * **Bug Fixes** * Enforced write-once tag behavior and fast-forward-only maintenance updates across release workflows. * Made nightly mirroring/selection and retention pruning more selective and safer. * **Tests** * Added Bats coverage for nightly mirroring and RC retagging, plus new Helm/unit checks. * **Documentation** * Updated the release model to center RC promotion and tag immutability. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
6fd98dc510 |
fix(apps/kubernetes): honour nodeGroup minReplicas + drop nodeGroups from render-side OIDC tests
Closes #3231. `packages/apps/kubernetes/templates/cluster.yaml` hardcoded `MachineDeployment.spec.replicas: 2` regardless of the nodeGroup's `minReplicas`. The two render-side `kubernetes-oidc-{system,customconfig}` bats — which set `minReplicas: 0` and delete their `Kubernetes` CR without waiting for readiness — still provisioned two 20 GiB `replicated` (DRBD) worker `DataVolume`s each. Their create/delete churn overlapped the very next test, `kubernetes-previous`, and put transient pressure on a LINSTOR satellite's ZFS pool: worker root disks landed late, tenant nodes registered late, Cilium's CNI init missed the 12-minute node-join deadline, and the test failed with `cni plugin not initialized`. Two changes, applied together — the chart fix is a no-op without the test fix, and the test fix is a no-op without the chart fix, because dropping `nodeGroups` falls back to the chart helper's default `md0` (with `minReplicas: 0`). **Chart**: `spec.replicas: {{ $group.minReplicas }}` so the field matches the autoscaler annotations already emitted just above. `minReplicas: 0` groups now genuinely scale from zero and do not silently provision workers; groups with `minReplicas >= 2` (the kubernetes-latest / kubernetes-previous bats CRs, the tests/values/ common.yaml default, the values.yaml default that gets templated into the helper's `md0`) render byte-identical to before. **Render-side OIDC tests**: drop the explicit `nodeGroups:` block and `storageClass:` — falling back to the chart helper's default `md0` (now with an effective `replicas: 0` after the chart change) means the MachineDeployment renders but never creates a KubevirtMachine and no DataVolume is provisioned. The tests only assert on the KamajiControlPlane and the cozy-realm Keycloak objects, so removing worker plumbing does not narrow their coverage. Upgrade impact: any existing customer CR that already carries `nodeGroups.md0.minReplicas: 0` (rare — the default in tests/values and the chart helper both use it, but real user CRs almost always have `minReplicas >= 1`) will see the MD's `spec.replicas` reconcile from `2` down to `0` on the next chart apply. If the cluster- autoscaler is enabled it will scale back up as soon as an unschedulable pod appears; if it is not, the group intentionally becomes empty (which is what `minReplicas: 0` requests literally). This matches the semantics the autoscaler annotations were already advertising — the hardcoded `2` was contradicting them. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Timofei Larkin
|
7b65cf2afd |
feat(platform): migrate to etcd-operator v1alpha2 (#2859)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for the newer etcd snapshot workflow (BackupClass/BackupJob/RestoreJob) with in-place restore. * Updated etcd chart/operator packaging and migration flow for the v1alpha2 operator release. * **Bug Fixes** * Switched readiness/condition handling to **Available**. * Improved waits to cover all etcd member pods during restore and scenario steps. * **Documentation** * Refreshed etcd backup/restore docs and example scenarios to match the v1alpha2 APIs and parameters. * **Tests** * Updated e2e coverage to validate v1alpha2 operator runtime contracts and replaced legacy backup-block tests with a snapshot round-trip test. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Fable 5
|
d252fc93a0 |
ci(release): make rc promotion transactional; defer registry writes to finalize
Promotion mutated the registry at dispatch time — retagging ~37 images to vX.Y.Z (+:latest), publishing the stable cozy-installer chart — before the promote PR's e2e ran and before any merge. An abandoned promotion then left stable-named bytes for a version that never shipped, and the (correct) image write-once check wedged any later re-promotion to the same version. Move every irreversible registry side effect into pull-requests-release.yaml finalize, which runs only after the promote PR merged (= after its e2e passed), making promotion atomic. promote-rc.yaml now only stages the release-X.Y.Z branch, drafts the stable release, and opens the PR; an abandoned promotion leaves nothing but an overwritable draft + branch. Folded into the same restructure: - Cut the api/apps/v1alpha1/vX.Y.Z Go-module tag in finalize, at the stable tag's commit. It was gated in tags.yaml on `release_exists==false && is_stable`, unsatisfiable under promotion (the draft always pre-exists), so promoted stables never got their submodule tag — a regression for Go consumers. Finalize is where the stable tag is born, so it cuts the matching submodule tag write-once (stable only). - Gate :latest by max-semver. promote-retag.sh now moves :latest only when MOVE_LATEST=1 (default off); finalize passes it the same make_latest decision the release gets, so a patch on an older line moves neither the release's `latest` nor the images' :latest. Added MOVE_LATEST bats coverage. - Stamp platformVersion into the published stable chart's default values (uncommitted, at package time) so `helm --install --version X.Y.Z` reports the stable version in telemetry instead of the rc version baked into the operator image. Not committed to the branch — that would leak platformVersion=vX.Y.Z onto main and mis-stamp the next rc. - promote-rc: serialize all promotions (static concurrency group, not per-rc); label the PR full-e2e so the full suite runs; scope the rc->stable sed to the depth-2 values.yaml glob plus the kubernetes .tag files (the build's own write set); tolerate a leftover draft so re-promotion is not wedged. - finalize: assert the PR author is cozystack-ci[bot], and run on an ephemeral oracle-vm runner (needs skopeo/helm). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
4f2530f9eb |
fix(release): scope the nightly-mirror host rewrite to the collected depth
collect_refs scans "$TREE"/*/*/values.yaml (depth-2), but the host rewrite used `find "$TREE" -name values.yaml` (any depth). A values.yaml nested deeper than group/pkg would then have its image host rewritten to GHCR without its image ever being mirrored there, leaving a dangling ref. Rewrite over the same depth-2 glob so the rewritten set is exactly the mirrored set (this also matches the existing --dry-run message). Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
d0740ed8c7 |
test(etcd): gate S3 backup round-trip behind ETCD_E2E_S3_ROUNDTRIP
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
011d6d1639 |
fix FUs
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
f2866edcf0 |
review fixes
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Timofei Larkin
|
5f76ac0d76 |
test(etcd): capture backup Job Pod logs, fail fast on terminal phase
The backup round-trip test could wedge for 20 min and then report a bare EtcdSnapshot phase=Started with no root cause: dump_diagnostics dumped the (healthy) EtcdCluster and the operator log, but the snapshot agent runs in its own Job Pod, so the etcd-read / S3-upload error lives only in that Pod's log and in the BackupJob/EtcdSnapshot status. - dump_diagnostics now dumps the whole backup chain: BackupJob/Backup and EtcdSnapshot describe, each snapshot Job's describe + Pod logs (selected by the operator's etcd-operator.cozystack.io/cluster label), and recent namespace events (the primary signal when a hung Pod logs nothing). - wait_for_field gains an optional terminal-failure value; steps 04/05 pass Failed so the BackupJob/RestoreJob waits return immediately on a terminal phase instead of polling to the timeout — which also keeps the snapshot Job Pod within its 600s TTL so its log is still readable at dump time. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Timofei Larkin
|
bd1c8d1070 |
test(etcd): gate contracts test on readyMembers=3 not Available
The v1alpha2 operator sets Available=True early (reason ClusterDiscovered) as soon as the seed member forms the cluster, then scales the remaining members up one at a time. Gating the runtime-contracts test on Available raced the member-Pod count assertion and saw only 1/3 Pods, failing the run. Gate on status.readyMembers=3 instead — the count of members whose EtcdMember Ready condition is True (and the /scale status path), which only reaches 3 once every member Pod is Running and serving. Bump the wait timeout to 300s to cover sequential one-at-a-time rollout. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Andrey Kolkov
|
80ad7048e0 |
test(etcd): select member Pods by strict operator labels in e2e reads
Address review feedback from coderabbitai on hack/e2e-apps/etcd.bats: read the container name, metrics port, and Pod IP with the strict member selector (name+instance+managed-by) plus a Running-phase filter instead of the broad app.kubernetes.io/name=etcd, so .items[0] cannot resolve to a non-member or not-yet-running Pod. Also assert a non-empty Pod IP before the metrics probe. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
087a8c4be9 |
chore(etcd-operator): bump to v0.5.2
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
cc5d688e15 |
feat(platform): migrate to etcd-operator v1alpha2
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Ivan Okhotnikov
|
57fc2cd566 |
fix(e2e): retry kubectl wait on transient management-etcd errors
kubernetes-previous.bats intermittently fails on `kubectl wait deploy --for=condition=available` with `Error from server: etcdserver: leader changed` even when the target Deployment is milliseconds away from becoming Ready. Observed in run 28772259934 on head `5f0bec622`: three of the four deployments already reported `condition met`, the fourth (the Kamaji tenant CP Deployment) was ~200ms from Ready per the crust-gather snapshot captured 20s later (`readyReplicas: 2`), and the wait exited non-zero on the FIRST etcd-side error it saw on the watch channel. Cause: the e2e sandbox management cluster is a 3-node kind on Talos VMs with a 3-instance etcd HA cluster. By the time kubernetes-previous starts (after kubernetes-latest, kubernetes-oidc-customconfig, kubernetes-oidc-system, plus the storageclass-fallback probe), accumulated CDI+DRBD IO pressure and lingering "still attached to node" PVCs from prior tests push etcd fsync latency high enough to trip heartbeat timeouts on the leader. The election that follows resolves in seconds, but `kubectl wait` sees the server-side error mid-watch and exits — clients would normally reconnect, but `wait` treats a watch-channel error as terminal. Wrap the four management-cluster `kubectl wait` calls in run_kubernetes_test with a retry helper that: - Retries only on a curated allowlist of server-side transients (`etcdserver: leader changed`, `etcdserver: request timed out`, `the server was unable to return a response in the time allotted`). - Never swallows legitimate errors (NotFound, --timeout expired, permission denied, etc.). - Caps at 3 attempts with a 5s backoff so a persistently-broken etcd still surfaces loudly. Untouched: - `kubectl wait namespace tenant-test --timeout=20s` at line 308 — short-lived, hits mgmt but the object always exists ~instantly. - `kubectl wait deployment ... --kubeconfig tenantkubeconfig-...` at line 509 — waits on the TENANT apiserver's own etcd, a different failure domain (single-instance kine or CNPG-backed via Kamaji, not the 3-instance mgmt etcd that flapped). - Other bats files with mgmt-cluster `kubectl wait` — out of scope for this PR; if the same flake reappears there a follow-up can lift the helper into a shared library. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
Ivan Okhotnikov
|
5f0bec6222 |
test(e2e): guard oidc-system grep against leading-dash pattern
The audience-list assertion used \`grep -qF "- tenant-test-..."\`. Because the pattern begins with a dash, grep treats the leading \`-\` as an option flag, aborting the test with \`grep: invalid option -- ' '\` before the actual match ever runs. Insert an explicit \`--\` end-of-options separator so the YAML list-item pattern is passed as a pattern rather than a flag. Custom-config counterpart is unaffected — its grep patterns do not start with a dash. Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
b637989fb0 |
test(e2e): port OIDC bats to cozytest lifecycle conventions
cozytest.sh (the e2e runner) is not real bats — it never invokes
setup()/teardown(). The OIDC files were structured around those hooks
to seed TEST_NAME and cleanup the tenant CR between tests, so with
cozytest strict-mode wrapping (\`set -eu\`), the first \${TEST_NAME}
expansion inside an @test aborted with 'parameter not set' before the
apply could even run.
Rework both files to the convention documented in external-dns.bats:
seed TEST_NAME at file scope (no per-test uniqueness needed — the
tenant-test namespace is single-suite), keep cleanup_kr() as a shared
helper, add a file-level cozy_cleanup() wrapper (the hook cozytest
actually calls on suite exit / first-failure), and move the pre-test
CR delete inline at the top of the first @test that creates the CR.
Subsequent @tests only read state and rely on the CR persisting until
cozy_cleanup.
Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
|