mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-26 13:07:04 +09:00
feat/postgres-credentials
53
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Andrey Kolkov
|
58b4343e5a |
ci(hack): route examples/backups/{vmi,vmdisk} edits to the vminstance suite
The two backup round-trips run the examples/backups/{vmi,vmdisk} scripts as their
harness, but select-e2e.sh mapped an examples/backups/<dir> edit to a suite only
when <dir> was itself a suite name. The dirs are vmi/vmdisk while the suite is
vminstance, so a harness-only edit selected no suite and the round-trip that runs
it never ran. Translate both dirs to vminstance, matching the vm-disk-application
entry in src_to_suites, and pin it with a bats case. The suite runs the vmi demo
with SKIP_RESTORE_TO_COPY=1, so 07-restore-to-copy.sh stays docs-only: selecting
the suite for it would pass without executing it. Both round-trips write to
the in-cluster seaweedfs-s3 endpoint, so a seaweedfs change selects the
vminstance suite as well. Also correct the controller-
owned-teardown note in docs/agents/e2e-testing.md (only Backup/RestoreJob
finalizers reach Velero; a BackupJob has none).
The two round-trips are also the only Tests that run a Velero backup and
restore, so the velero PackageSource maps to the vminstance suite; it
otherwise maps to a velero suite that does not exist, and a Velero change
ran neither round-trip.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrey Kolkov
|
43b02bda30 |
test(e2e): run the Redis backup round-trip against Valkey
The Redis backup demo now takes APP_KIND=Valkey, which switches the app kind and the chart prefix the helpers derive sentinel and Secret names from. The demo BackupClass routes Valkey to the same demo strategy. The valkey suite gains valkey-backup-roundtrip, which drives that demo end to end: seed a marker, back it up, restore it in place and into a copy. It writes to tenant-root's seaweedfs through seaweedfs-s3, so a seaweedfs change now selects the valkey suite too, and since the round-trip runs examples/backups/redis as its harness, an edit there selects both the redis and the valkey suites. The reduced E2E install keeps the backup stack for the valkey suite, as it does for the other backup round-trip suites. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
Myasnikov Daniil
|
cc942fed68 |
test(tests): run unit suites with Bats and retain POSIX coverage
The shell translator rewrites test bodies and loses Bats reporting. The unit lane needs complete TAP results and JUnit reports without changing how live-cluster suites run. Sourced production helpers still need a dash pass, and unit test bodies need nounset without exporting shell options into subprocesses. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
d35493a64d |
test(e2e): cover the computeplane module end to end
The module had render tests only. Nothing checked that turning it on in a tenant yields a working cluster: the Kamaji control plane, a worker pool shaped by the module's allowlist, an admin kubeconfig the tenant can neither list nor read, the pinned cert-manager and ingress-nginx addons running inside the cluster, and a teardown that removes the in-cluster addons before the control plane and leaves nothing behind. The suite runs the module in its own child tenant so the shared tenant-test is never toggled. It supplies its own pool because the default md0 asks for the replicated StorageClass, which the container lane does not have. On failure it prints the worker image source, since the pool allowlist drops talos and a ComputePlane worker cannot use the in-sandbox image cache. The computeplane source depends on the kubernetes application and runs its own ingress-nginx, so an ingress-nginx change now selects this suite too. It also owns apps/kubernetes-nodes, which used to reach no suite and run everything; kubernetes-nodes-application now maps to both kubernetes suites and computeplane, so a pool chart change keeps the suites that create KubernetesNodes natively. Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM |
||
|
IvanHunters
|
858ca4d0a3 |
test(e2e): cover monitoring OIDC modes with a Chainsaw suite (#4678)
## What this PR does Monitoring OIDC gets live-cluster e2e coverage again. It had none since the two pre-Chainsaw BATS suites were removed without a port. This adds a Chainsaw suite in `hack/e2e-chainsaw/monitoring/` that ports what they asserted, plus the in-place switch from an inline config to a `secretRef` that nothing covered. The suite takes one Monitoring release in `tenant-test` through three modes in a row: System, CustomConfig with an inline config, and CustomConfig with a `secretRef`. It creates that release itself and never reads the tenant-root monitoring instance, so it keeps working if #4023 or #4479 turns root monitoring off. What each phase checks: - System mode: cozystack-api admits `spec.oidc` and both HelmReleases go Ready. The Grafana CR points `auth.generic_oauth` at the cozy realm, has no `role_attribute_path`, and has `skip_org_role_sync` and `oauth_allow_insecure_email_lookup` forced. The KeycloakClient is confidential and references the generated Secret, which holds 32 alphanumeric characters. The audience scope is bound to this release's clientId. There is no chart-owned `KeycloakRealmGroup`. The users-Job carries the users and the mode, and `grafana-deployment` reads the client secret from the Secret. - Switch to inline: the Grafana CR takes the tenant's map, keeps the forced keys and drops every System-only key. The users-Job reruns with the new list. The KeycloakClient, the audience scope and the client Secret are pruned, and the Deployment stops referencing that Secret. - Switch to `secretRef`: the inline `auth.generic_oauth` block is gone from the Grafana CR. The operator Secret is mounted at `/etc/grafana/oidc` on both the CR and the Deployment. This checks the mount only and does not prove Grafana loads the file, see #4679. - After the install and after each switch, neither release may show a remediation cycle. That means no `failed` or `uninstalled` snapshot, and the latest revision must be exactly 1, then 2, then 3. The revision check is there because helm-controller truncates history after the previous deployed snapshot. An upgrade that failed, rolled back and then passed on retry leaves no failed entry, only extra revisions. I did not assert `GF_PATHS_CUSTOM_INI`. I could not find anything in Grafana that reads it (#4679), and pinning it would make possibly dead wiring part of the contract. The coverage split with the render tests is this. `packages/system/monitoring/tests/oidc_test.yaml` keeps the fresh-install render of each mode, including the CustomConfig negatives (no Keycloak objects, no client Secret). This suite covers what a render cannot see: admission, the Deployment the Grafana operator builds, and what a switch between modes prunes or leaves behind. The Keycloak objects are checked as written, so creating the client in Keycloak is not asserted. Removing it is, because the operator's finalizer holds the KeycloakClient until the client is gone from Keycloak. Before the first apply the suite also waits until the tenant's `cozystack-values` carries `oidc-enabled: "true"`. The outer release copies `_cluster` into the inner release once, so a Monitoring applied right after the tenant is created can render from stale values and fail the System-mode guard for the full 15m. That is what the first CI run on this PR hit. The Keycloak CRDs are required, not probed for. The e2e install always enables platform OIDC, so a missing CRD is a broken install. The removed suites skipped in that case, and their probe could not tell an absent API group from a present one. The selector picks the suite up by name from `cozystack.monitoring-application` and `cozystack.monitoring`. A change to `system/monitoring`, `extra/monitoring`, `monitoring-rd` or `grafana-operator` now runs only this suite instead of the full run. The tenant-root Grafana bring-up stays covered by the install BATS while it exists. One selector unit test relied on `cozystack.monitoring` reaching no suite, so it now renames that source in its copy of the graph. The second commit keeps Keycloak changes on every suite that drives a Keycloak client. Before this PR a change to `system/keycloak` or `system/keycloak-operator` escalated to the full run. With the monitoring suite it would select monitoring alone, because `monitoring-application` is the only source that depends on `keycloak-operator`, and `kubernetes-latest` with its OIDC lifecycle would drop out. `src_to_suites()` now maps both to `kubernetes-latest monitoring`. The mapping has to be explicit because `kubernetes-application` has no `dependsOn` on `keycloak-operator` (#4680). A `postgres-operator` change now also reaches `kubernetes-latest`, through Keycloak's database. The selector comment and the e2e doc no longer use `system/postgres-operator` as the example of a path with a source that reaches no suite. Cost: one monitoring stack bring-up, two in-place upgrades and teardown. I expect 10 to 20 minutes on every full run. The selector change makes the CI run on this PR a full run, so the suite's own timing is in the chainsaw report. A red run can spend up to 16m on each of four waits, since each one sits a minute above the 15m timeout of the release it waits on. The suite cannot run locally, so I checked it this way: - `chainsaw lint test` at v0.2.15, the sandbox version, passes. - I ran the suite's own steps against a local envtest apiserver. Chainsaw applies the real manifests, and a stand-in for helm and the operators renders the chart from the stored Monitoring spec. With the chart intact all nine steps pass. Each of 30 single breakages of the chart or of the simulated controllers turns the expected step red. - `hack/select-e2e_test.bats` and `hack/select-install_test.bats` pass. The new Keycloak selection case fails without the mapping and when `kubernetes-latest` is dropped from it. A browser login through Keycloak is out of scope, same as in the removed suites. Left for later: - KeycloakClient and scope `status` after the operator creates them in Keycloak is not checked. - `hack/e2e-chainsaw/README.md` lists suites by hand. I added `monitoring`, but `backup`, `kafka-metadata` and `kubernetes-proxmox` were already missing. - The `operator diff selects all dependent app suites` selector test could also check `monitoring`. - The comment on the Keycloak mapping says both suites wait for the finalizer to remove the client. The monitoring suite does wait. `kubernetes-latest` only reads once after its upgrade. - The ready checks compare `observedGeneration` with `metadata.generation` through `contains()`. Both are fields of the same object, not a number literal, so the v0.2.15 literal gotcha should not apply, but no live run has passed this suite yet. - The inline fixture carries no `enabled: "true"`, so that phase checks a disabled OAuth block. The asserts do not depend on it. Closes #4051 Closes #3785 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added end-to-end coverage for Monitoring installations using system OIDC, inline custom configuration, and Secret-based configuration. The tests verify readiness, rendered resources, configuration cleanup, and upgrade history. * Updated test selection so Keycloak-related changes also run the Monitoring suite. * **Documentation** * Added Monitoring to the documented Chainsaw suite list and clarified how changed paths determine test coverage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
02fbe76418 |
fix(ci): close three blind spots in the e2e test selectors (#4658)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The e2e selectors miss Chainsaw suites that have no `chainsaw-test.yaml` or `.yml` directly in their directory. Chainsaw v0.2.15 also runs a directory that holds only numbered step files such as `01-install.yaml`, and its discovery walks every subdirectory of the path the runner hands it. A suite in either layout ran only when nothing was selected. No narrowed selection named it, the full-suite list left it out, and `select-install.sh --validate` never asked for its mapping. Now `hack/select-e2e.sh` and `hack/select-install.sh` list a top-level directory as a suite when Chainsaw would load a test from it or from anywhere below it. No suite in the current tree uses either layout, so today's suite list stays the same. A file placed directly in `examples/backups/` (a shared `common.sh`, for example) selected nothing, because it fell through to the inert `examples/` rule. It now escalates to the full suite, same as `examples/backups/_lib/`. `README.md` there stays inert through the `*.md` rule. The `test-chainsaw` comment in `packages/core/testing/Makefile` said Chainsaw discovers every `chainsaw-test.yaml`, which was too narrow. I fixed it, and the matching sentences in `docs/agents/e2e-testing.md` too. Every behaviour change has a bats test that fails against the scripts on main. Known gaps I left for a follow-up, none of them a wrong answer today: - The `full_suite_list` helper in `hack/select-e2e_test.bats` repeats the selector's `find`/`sed`, so a bug in that shared pattern would pass every full-suite assertion. The step-file and nested tests compare against literal lists, which covers the new layouts. - The selectors are a little wider than Chainsaw: a broken `chainsaw-test.y*ml` symlink, or one with no Test in it, still lists its suite. That errs toward running more. - A few test comments describe what used to happen rather than why the rule exists. Closes #4630 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(ci): the e2e test selectors now see Chainsaw suites made of numbered step files or nested below a suite directory, and run the full suite for an edit to a file placed directly in examples/backups/. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * E2E suite discovery recognizes supported manifest and numbered-step files at any depth and maps them to the top-level suite. * Changes to files directly under `examples/backups/` trigger full-suite selection. * Suite selection reports an error if discovery fails or finds no suites. * **Documentation** * Clarified supported suite layouts, disabled suite-file handling, discovery and escalation rules, and per-path coverage guidance. * **Tests** * Added coverage for nested suites, numbered steps, unmapped suite layouts, and backup-path selection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
e5a1ca853f |
fix(ci): run every Keycloak client suite on a Keycloak change
With a monitoring suite in the tree, a change to system/keycloak or system/keycloak-operator stopped escalating to the full run and selected monitoring alone, because monitoring-application is the only source that declares the keycloak-operator edge. That dropped kubernetes-latest, whose OIDC lifecycle also creates a KeycloakClient and waits on the operator's finalizer to remove it. The mapping is explicit because kubernetes-application has no dependsOn on keycloak-operator. A postgres-operator change now reaches kubernetes-latest too, through Keycloak's database. The selector comment and the e2e doc no longer name system/postgres-operator as a path with a source that reaches no suite, which stopped being true when monitoring got its suite. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
393592a308 |
test(e2e): lower the seaweedfs volume size limit on a live instance
Every existing case installs SeaweedFS at one limit and stays there, so nothing walks the transition an operator makes when lowering it: each volume at or above the new value leaves the writable set and the bucket writes again only after a grow. The failure is a PUT that times out while every release stays Ready, so the suite ends on a write into a bucket filled between the two limits. The suite runs its own instance, created directly so the limit can move without racing the tenant module, and asserts on the live master that the value set on the resource arrives with the growth counts intact. seaweedfs changes now select the suite, which the selector's mapping and its tests are updated for. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5fde704a30 |
fix(ci): skip directories named like Chainsaw step files
Chainsaw builds step files only from directory entries that are not directories, so a directory named like 01-x.yaml makes nothing a suite. Both selectors listed it as one, and select-install demanded a mapping for it. A symlink still counts, as it does for Chainsaw. A directory named chainsaw-test.yaml keeps its suite listed: Chainsaw reads that name whatever its type and fails loudly on a directory, and an unlisted suite would hide that failure. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e82e5baf65 |
fix(ci): discover Chainsaw tests nested below a suite directory
Chainsaw walks every subdirectory of the path it is handed, and the runner hands it each top-level directory under hack/e2e-chainsaw/, so a test in <dir>/<sub>/ runs as part of <dir>. Both selectors read only one level down, so a directory whose tests all sit deeper was never listed: no selection named it, the full-suite list left it out too, and only a run with no selection at all executed it. Both scripts now take the top-level directory of a test file found in that directory or anywhere below it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
c1b99df98d |
fix(ci): escalate a file placed directly under examples/backups
A file beside the per-app backup directories rather than inside one belongs to no single walkthrough, but it matched neither backup rule and fell to examples/ in the inert list, so an edit to it selected nothing and both lanes skipped Chainsaw. It is now treated as shared by all of them and escalates the same way examples/backups/_lib/ does. README.md there stays inert through the *.md rule. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6913a04622 |
fix(ci): discover Chainsaw suites made only of numbered step files
With no chainsaw-test.* in a directory, Chainsaw v0.2.15 builds a Test from files named like 01-install.yaml. Such a directory runs whenever the whole tree is handed to chainsaw, but neither selector listed it, so no escalation carried it and no edit to it could select it. Both scripts now derive the suite list from the file names Chainsaw itself accepts. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
08011cef09 |
fix(ci): escalate an edit to the shared backup walkthrough helpers
examples/backups/<app>/ selects that app's suite, and a directory whose name is no suite selects nothing. A helpers directory shared by every walkthrough, examples/backups/_lib/, falls into the second case: an edit to it alone produces an empty selection, which both lanes read as "skip Chainsaw", so a change reaching every backup round-trip would run none. It now escalates to the full suite like hack/e2e-chainsaw/_lib/. The set of suites that run a walkthrough is spread across the suites' own scripts, and a derived subset that misses one would fail open. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
603ff2871e |
docs(hack): drop file and branch counts from the selector comments
The comments explaining the unit/e2e bats split quoted how many files each side held, and the escalation helper how many branches call it. Every figure had gone stale: nothing checks them, and they move each time a bats file or an escalation is added. The Makefile expression quoted beside the bats claim is what defines the set, so the explanation stands without the number. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a6ec30d56f |
fix(ci): discover Chainsaw suites written as chainsaw-test.yml
Chainsaw loads chainsaw-test.yml when a directory has no chainsaw-test.yaml, but select-e2e and select-install enumerated the .yaml name only. A suite spelled that way ran whenever the whole directory was handed to chainsaw, yet no escalation listed it and no edit to it could select it, so it silently dropped out of every narrowed run. select-install also sent find's errors to /dev/null inside a pipeline that reported sort's status, so a failed listing validated no suite at all and passed. It is now captured and fails validation. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d777f2a0b3 |
fix(ci): select the suites that use seaweedfs instead of all of them
cozystack.seaweedfs-application mapped to a seaweedfs suite that does not exist, so every seaweedfs change reached nothing runnable and paid for the whole Chainsaw run. The suites that can regress from it are bucket, the backup round-trips that write to the tenant-root instance through its in-cluster S3 endpoint, and harbor, whose registry storage is a BucketClaim served by the seaweedfs COSI driver with no graph edge between the two. bucket alone would miss the CA and service-name paths the round-trips depend on. The test derives the expected set from the suites' own yaml, so a new suite that names the endpoint or a BucketClaim there fails it until it is mapped. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e070bc7d1d |
fix(ci): escalate a per-suite Chainsaw path that names no suite
A path under hack/e2e-chainsaw/<dir>/ put <dir> into the selection without checking that it is a suite, leaving the final intersection to drop it. Alone the path escalated through the empty-selection backstop, but beside any path that contributed a suite the escalation vanished and the run narrowed to that suite with nothing on stderr. Shared material next to _lib/ or a suite nested deeper than discovery looks then went untested in exactly the pull requests that also touch an app. The directory name is now membership-tested where the path is classified, as the hack/e2e-apps/ arm already does, so the decision is a property of the path and names it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
c55ab268e2 |
test(monitoring): remove orphaned OIDC BATS suites
The two pre-Chainsaw monitoring OIDC suites have had no invocation site since the app suite migrated to Chainsaw. Remove them instead of carrying dead tests that force the selector to run every live suite. Keep the existing selector and capture guards for the historical path shape, and update their comments now that the directory is empty. Retain a named chart assertion for the no-KeycloakRealmGroup ownership invariant; the existing count and kind assertions already enforce the same behavior. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
9673af6aca |
fix(kafka): guard short backup drains and settle example teardown
- Short-drain guard: kafka-console-consumer exits 0 after a timed-out short read, so a truncated drain was tarred and reported Succeeded. Compare the drained line count against end-begin per partition and fail the backup on any shortfall, making the README's "fails closed" promise real for compacted/transactional topics. - Settling teardown: cleanup.sh returned before the Kafka apps and Bucket had uninstalled and could not report failure, letting the neighbouring chainsaw/kafka Test contend on PVC reclaim. Add a wait_deleted helper and a teardown_rc gate that waits for the heavy objects and exits non-zero when the teardown does not settle. - Fully qualify the ambiguous delete kinds (backup/bucket/backupclass/ restorejob/backupjob) so a bare shortcut cannot target a foreign group and silently leave the Cozystack artifact behind. - Default KAFKA_IMAGE from the running operator (newest STRIMZI_KAFKA_ IMAGES entry), falling back to the literal only when the operator cannot be read; drop the duplicate resolution note. - Make the strategy and throwaway CLI Pods restricted-clean (runAsNonRoot + JSON-patch securityContext on the CLI Pods). - gitignore .source-dump.txt; replace the deprecated "small" preset with c1.small; capture the to-copy cluster in the e2e catch; document the all-topics risk; de-enumerate the decaying suite-list comments. Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io> Assisted-by: LLM |
||
|
Aleksei Sviridkin
|
f9ea0da3de |
feat(backups): Kafka topic-metadata backup via a dedicated strategy driver (#4000)
## What this PR does Adds a dedicated `strategy.backups.cozystack.io/Kafka` Go driver that backs up the **topic metadata** of a Cozystack-managed `Kafka` application and wires it into the platform `cozy-default` flow — the counterpart to the app-specific drivers (CNPG, MariaDB, Altinity, MongoDB). It is a typed variant of the generic Job strategy: the controller renders a one-shot batch Job that talks to the Kafka Admin API, and adds on top an applicationRef Kind gate, a `Ready` precondition on the Strimzi `Kafka` cluster, and a per-run object key via `.BackupName` so distinct runs never overwrite each other. What it captures: every non-internal topic's partition count, replication factor and non-default configs (via `kafka-topics --describe`), stored as one small object per run in the system bucket; restore recreates them with `kafka-topics --create` + `kafka-configs --alter`. Deliberately **out of scope**: message payloads (a volume-snapshot strategy's job), consumer-group offsets (a log position, only meaningful with a byte-identical data restore), ACLs, quotas, and `KafkaUser`s. No PITR. Restored topics are created directly through the Admin API, so on the target they are unmanaged (no `KafkaTopic` CR) — the accepted tradeoff of an Admin-API-only driver. Both restore flows are implemented: **in-place** recreates dropped topics into the source, and **to-copy** applies the source's topics onto a freshly-bootstrapped empty `Kafka` (the "bootstrap a cluster, then apply the metadata state" flow). Contents: - Driver: `api/backups/strategy/v1alpha1/kafka_types.go`, `internal/backupcontroller/kafkastrategy_controller.go` (+ unit tests for its pure helpers), `internal/backupcontroller/kafkatypes/` (Strimzi `Kafka` readiness gate), dispatch cases in both controllers + `supportedBackupStrategyKinds()`, scheme registration, generated CRD + deepcopy. - Chart: `strategy-kafka-default.yaml` (gated on the bucket name), a `cozy-default` binding, read RBAC on `kafka.strimzi.io/kafkas`, and the Strimzi Kafka client image value. - Docs: a Kafka section in `docs/operations/backup-classes.md`. - Example + e2e: `examples/backups/kafka-metadata/` and `hack/e2e-chainsaw/kafka-metadata/` (`kafka-3-metadata-roundtrip`), an honest round-trip that proves a per-run `retention.ms` sentinel survives both restores. The original `packages/apps/kafka` chart is left untouched (it has no legacy backup mechanism). Validated: `go build ./...`, `go vet`, `go test ./internal/backupcontroller/...` (pure helpers plus a reconcile Kind-gate test and the RestoreJob cleanup regression test), `helm unittest` on the package (which now renders `strategy-kafka-default.yaml` and asserts its kind/name/`artifactURITemplate`), the `select-install`/`select-e2e` bats suites, `helm template` of the strategy CR, `bash -n` of the rendered script, and `make generate` (clean). The end-to-end round-trip (`kafka-3-metadata-roundtrip`) is enabled and mapped like the sibling backup suites, so CI builds this branch's controller and runs it live against a fresh sandbox; it has not been merged-and-run before, so that first live run must go green before merge. ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. This PR adds a new `strategy.backups.cozystack.io/Kafka` CRD, a Go driver, chart wiring, docs and examples. It does not add or rename a `packages/apps`/`packages/extra` package, does not change any `ApplicationDefinition`, `values.schema.json`, version enum, `release.prefix`, output Secret/Service, the `Package`/`Plan`/`RestoreJob` CRDs the provider types by hand, any `hack/*.mk` anchor or make-target behaviour, node prerequisites, labels/annotations, or telemetry metrics. A brand-new `strategy.backups.cozystack.io` CRD travels to the Terraform provider as opaque JSON (it types only `Plan`/`RestoreJob`/`Package`), so it keeps working untouched. None of the downstream triggers match. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(backups): add a Kafka topic-metadata backup strategy bound by cozy-default — backs up topic definitions and configs (partitions, replication factor, non-default configs) via the Kafka Admin API, with in-place and to-copy restore; message data, consumer offsets, ACLs, quotas and KafkaUsers are out of scope ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added Kafka topic-metadata backup and restore support, excluding message data. - Supports ad-hoc and scheduled backups, in-place restoration, and restoration to a separate Kafka cluster. - Recreates missing topics and restores compatible configurations while validating partition and replication-factor constraints. - Waits for Kafka readiness and cleans up backup artifacts after completion. - Backup and restore jobs now enforce configurable deadlines to prevent stalled operations. - **Documentation** - Added guidance covering Kafka metadata backups, restoration limitations, and S3 configuration. - **Tests** - Added end-to-end coverage for backup, restoration, validation, and cleanup scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
19adffb515 |
test(opensearch): exercise the TLS rollout on a cluster
No suite created an OpenSearchCluster anywhere in the tree, so the whole class of failure where the chart renders correctly and the cluster then refuses the workload had no coverage at all. Rendering proves the manifests; only a cluster proves that securityadmin authenticates against the CA the chart handed the operator and that the security configuration actually applies. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrey Kolkov
|
405eb25095 |
fix(backups): harden Kafka metadata backup
- Fail closed on topic listing: set -o pipefail, capture kafka-topics --list --exclude-internal exit separately, and abort on failure so an unreachable broker can no longer produce an empty backup reported as Succeeded. An empty list after a successful call stays legitimate. - Check each kafka-topics --describe exit and validate the parsed partition count / replication factor before writing a record, so a timed-out describe cannot write a corrupt orders\t\t\t line. - Store topic configs one per line with key and value in separate tab-delimited fields, sourced from kafka-configs --describe (one config per line), so a value containing a comma (cleanup.policy=compact,delete) round-trips; on restore each config is applied with the value bracketed only when it contains a comma. - Detect topic divergence on restore instead of a silent --if-not-exists no-op: grow partitions with --alter when the backup asks for more, and fail loudly (naming the topic and both values) when it asks for fewer partitions or a different replication factor. A restore that cannot reach the recorded state no longer reports Succeeded. - Add KafkaStrategyKind to the RestoreJob cleanup-dispatch no-op group so deleting a Kafka RestoreJob does not run a pointless Velero cleanup; add a regression test alongside the siblings, plus a reconcile Kind-gate test and a rendering test that actually renders the strategy template. - Wire the kafka-metadata Chainsaw suite into both selectors (select-install suite_to_source, select-e2e src_to_suites) so editing the kafka app or the suite selects it and the bats unit tests pass. - Pin the client image to the operator's default broker version (kafka 3.9.1) with a digest, and align the example default; fix the values comment. Point the RabbitMQ comment references (which shipped in the CRD description) at the Job strategy actually mirrored, and trim the in-pod comments that rendered into the manifest. - Document the Strimzi Topic Operator caveat: for a topic owned by a KafkaTopic CR the operator reconciles its config from the CR, so an in-place config restore is durable only for out-of-band topics. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
Myasnikov Daniil
|
186547d835 |
test(e2e): run the merge-gating lanes on Talos containers
The srv1-srv3 nodes of both lanes that gate a merge -- the same-repo one in pull-requests.yaml and the fork one in e2e-fork.yaml -- run as Talos containers instead of QEMU guests, so a tenant worker sits at L2 rather than L3. Measured in CI on commits where both substrates ran the same tree: whole-job wall clock down 24-49 minutes, essentially all of it in Chainsaw, while the two tenant-Kubernetes suites go from 1143-2040s to 466-809s and stop depending on the runner's vgif flag. The node-join soft-red gate goes with the substrate, which is what it was for. It existed because nested virtualisation on the shared runners degraded far enough that a worker registering in minutes elsewhere could miss any deadline this test could afford; #3513 established that the discriminator was the host's kvm_amd vgif flag, and removing a nesting level removes what it acted on. So hack/e2e-node-join-soft-red.sh, the SOFT-RED-node-join.txt marker and the soft_red job outputs are gone from all four workflows, and a missed deadline is an ordinary red again. The ::warning annotation survives, because the reason a run went red should be legible without opening the log, and so does the 124-versus-everything -else test, because "the workers were slow" and "the wait never ran" send a reader to different places. Done by swapping the substrate inside the existing e2e jobs rather than by promoting a separate container job. Those jobs carry more than a substrate -- the GitHub App token, the report-overrun guard, the images list, the SSH breakpoint -- and promoting a job that never had them would have dropped four features without saying so. Three properties of container mode drive the design, and each fails quietly rather than loudly. machine.kernel.modules is a silent no-op -- kernel_module_spec.go returns early on ModeContainer with no error and no event -- so the host loads openvswitch and zfs before the nodes start and the workflow step asserts both rather than trusting them; a missing module surfaces an hour later looking like a CNI or a storage regression. Docker gives a privileged container its own tmpfs /dev seeded once at creation, so /dev/kvm works but ZFS zvols created later never appear and linstor-csi fails ControllerPublishVolume; the compose file binds the host devtmpfs, which is also why talosctl cluster create docker can never carry LINSTOR. And there is no ARP VIP, so 192.168.123.11 stands in for the QEMU lane's .10. The QEMU machinery stays in the tree and keeps running: nightly.yaml and e2e-tag.yaml still call `make prepare-env`, so DRBD, the replicated StorageClass, its Immediate binding mode, the tenant StorageClass propagation that only applies to remotely-accessible classes, and the Cozystack Talos node image with its extensions all keep nightly and release-candidate coverage. None of them is exercised per pull request any more. That is the trade. Nothing on the PR path builds or downloads a nocloud disk now, so build-talos drops the talos-nocloud step and the talos-image artifact. build-talos itself stays: finalize consumes its digest fragment. One property this does not fix, recorded beside the runner class because it will be read off a red run eventually: a container node reports the HOST's CPU and memory as capacity, and hack/e2e-container-up.sh counters that with kubelet systemReserved. That corrects the scheduler and not what a pod reads from /proc, so a workload sizing itself from /proc/cpuinfo or /proc/meminfo is configured for the whole runner. Also carried here because the same files carry them: the ghcr.io pull-through mirror removal (see #4007 -- whichever lands first makes the other a no-op), the authoritative HelmRelease readiness gate, the backup-credential preflight, the prepull guard against an image-less container fragment, and folding the two standalone OIDC suites into the latest-version tenant cluster. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
ad961f6de1 |
test(e2e): cover webhook admission after namespace churn
Exercise ingress and pod admission immediately after deleting a tenant namespace, and route ingress-nginx changes to the gateway suite. Assisted-By: GPT-5 <noreply@openai.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
adf231513f |
fix(ci): correct two reachability defects in the selection graph
Both are cases of the walk reaching the wrong set of suites for a package, and both were found by enabling the site-router suite on #3426, which is the first suite in the tree to land downstream of kubevirt-cdi. `vm-disk-application` now maps to the `vminstance` suite. It mapped to a `vm-disk` suite that does not exist, which intersect_suites() drops, so the source reached nothing runnable and neither did kubevirt-cdi above it: every CDI change ran all 21 suites. The vminstance suite creates a VMDisk and asserts the DataVolume behind it (hack/e2e-chainsaw/vminstance/vmdisk.yaml, vmdisk-vmi.yaml), so it is the suite that covers both. A CDI change now selects `vminstance` alone. `cozystack.cozystack-basics` joins `cozystack.cozystack-engine` as a propagation hub excluded from the reverse-dependency walk. Every edge into it exists so a namespace or a platform-wide policy is in place before the dependent installs -- kubevirt-cdi says exactly that in its own source -- which is install ordering, not behaviour. The damage runs opposite to the engine's: the engine fans one change out to every app, while basics narrows down instead. It reaches no suite today so it escalates, but it sits upstream of kubevirt-cdi, so the first suite to land under CDI silently converts the platform's namespace-and-policy package from the full run to that one suite. On #3426 that is 22 suites down to `site-router` alone, with nothing in the output to say coverage was lost. Dropping the reverse edges keeps the package reachable, stops it propagating, and leaves a change to it running everything through the per-path escalation. Both rules have a test, each verified to redden with its rule removed: the mapping test falls back to the full-suite escalation, and the hub test -- which seeds the hazard by giving redis-application an edge onto basics, since no suite sits downstream of it in this tree yet -- selects `redis vminstance` instead of everything. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
73bc4c0ae3 |
fix(ci): build the backstop's reason list without a blind pipeline
`unmatched=$(echo … | tr … | sort -u | grep -v '^$' | paste -sd ' ' -)` takes its exit status from paste, so a failure in tr, sort or grep is invisible under set -e. Demonstrated with a sort stub that exits 1: the pipeline reports rc=0 and yields an empty string, so the line becomes `no runnable suite is named by ''`. That is the same last-command blindness this script already handles twice -- for the suite-list find and for the two yq indexes -- and the comments there explain why each was a bug. Selection stays fail-safe because the escalation is decided before the message is built, so the only casualty is the reason line; but the reason line is a contract as of this change set, and a contract that degrades in silence is what the change set exists to remove. Replaced with the shell's own split plus a `case` membership test: no external command, nothing that can half-succeed, dedup preserved, and the same idiom resolve_suites already uses a few lines up. A test feeds two unresolved directories with one repeated and pins the whole line, so a regression to a form that can emit a partial or empty name turns the file red rather than printing a worse message. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
c28d7804af |
fix(ci): enumerate the inert .gitattributes instead of matching by name
The rule was broader than its justification. It read `(^|/)\.gitattributes$`, so it claimed every present and future .gitattributes anywhere, while the argument for it was about content: every one in this tree holds nothing but `linguist-generated` markers, which reach no build, chart or test. The filename does not carry that argument. .gitattributes can also set `filter`, `eol`, `working-tree-encoding` and `export-subst`, each of which changes what lands in the working tree and therefore what gets built, so a by-name rule would eventually make a live file inert with nothing to notice it. Nothing is lost by narrowing, because only three files ever reached the rule: api/.gitattributes and internal/crdinstall/manifests/.gitattributes are escalated by full_suite_pattern first, so entries for them would be dead. Measured against the base selector, those three are exactly the paths whose classification this branch changes, and they still select nothing. Same trade as the workflow enumeration a few lines up, and the same upkeep, which the comment now states: after adding or moving one, re-check with `git ls-files '*.gitattributes' '.gitattributes'` and confirm the contents are still only linguist markers. A .gitattributes not on the list is classified by whatever rule its path falls under -- the graph inside a package, the unclassified fall-through at the root -- and both fail safe. A test pins that root case, so a future widening back to a by-name match turns the file red. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
b2702621d3 |
fix(ci): decide the per-app bats rule per path, not after the merge
Two defects in the hack/e2e-apps/ arm added by the previous commit. The arm set trigger_any=1 for whatever basename it derived and left the verdict to the final intersection. That makes the escalation depend on the REST of the diff: hack/e2e-apps/monitoring-oidc-system.bats alone emptied the selection and the backstop escalated, but the same file beside packages/apps/redis/templates/redis.yaml selected `redis` with nothing on stderr, where the pre-change selector ran all 21 suites. That is the merge-before-escalate shape #3330 removed from the graph walk, reintroduced one rule over. The basename is now membership-tested against the suite list on the spot and an unmatched one sets trigger_full=1 and names the file. POSIX case matches `/` with `*`, so the arm also sees nested paths, and the unanchored capture turned hack/e2e-apps/fixtures/postgres.bats into the "suite" fixtures/postgres -- right outcome by accident, reason line printing a path where a suite name belongs. The capture is anchored to one segment, and what the arm does with anything that is not a top-level <name>.bats is now stated rather than left to fall out of a regex: it is shared material for those suites the way hack/e2e-chainsaw/_lib/ is for the Chainsaw ones, so it escalates for the same reason. Both claims that said otherwise are corrected -- the arm's own comment and docs/agents/e2e-testing.md both asserted the deferred escalation was equivalent to the old fall-through, which held only for a single-file diff. The backstop comment drops the e2e-apps clause it gained last commit, since that route no longer reaches it. The header's "four of the seven" becomes count-free: this commit adds two escalating branches, so a hardcoded total was already stale. Tests: the mixed-diff case is the regression pin (the isolated case passes with the escalation deferred and cannot see the bug), plus a nested-path case and a non-.bats case. Both fixes are mutation-verified -- restoring either defect turns the file red. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
4d4f5edf92 |
fix(ci): escalate only the e2e bats files, not the whole unit lane
full_suite_pattern took every hack/*.bats, so all 63 of them ran the whole Chainsaw suite -- including the 60 that are the unit lane and that the e2e sandbox never executes. The root Makefile is the authority on the split and draws it at exactly the prefix this narrows to: BATS_UNIT_FILES := $(filter-out hack/e2e-%.bats,$(wildcard hack/*.bats)) The 60 files it keeps run under `make unit-tests`; the three it filters out -- e2e-prepare-cluster, e2e-install-cozystack, e2e-test-openapi -- are the ones packages/core/testing's recipes execute inside the sandbox, and those still escalate. So no Chainsaw suite can regress from an edit to one of the 60. Inert here is not a green gate with nothing behind it. `make unit-tests` is gated on pull-requests.yaml's `code` output, computed in the workflow as "any changed path outside docs/" and never from select-e2e.sh, so a bats-only pull request still runs the whole unit lane. The `e2e` job reads the same output, so it still installs the platform and runs the OpenAPI tests before skipping Chainsaw. Two tests: one pins the rule on both sides of the prefix and asserts an inert bats file beside an app path does not mask the selection, and one walks every hack/*.bats in the tree and requires the selector's verdict to agree with the Makefile's split, so a file named to fit neither lane surfaces here rather than on the next PR that touches it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
33428671fa |
fix(ci): classify hack/e2e-apps/ and packages/tests/ in select-e2e
Three path classes matched no rule in hack/select-e2e.sh and escalated to all 21 Chainsaw suites through the unclassified fall-through -- 7 of the last 150 merged pull requests between them. hack/e2e-apps/<name>.bats now maps to the <name> suite, off the basename exactly as the per-suite rule takes the name off a hack/e2e-chainsaw/<app>/ directory. Not marked inert: what is left in that directory after the Chainsaw migration is wired to nothing, and inert would bake that orphan status into the rule and go quietly wrong the day a lane runs those files again. A basename no suite carries drops out of the intersection at the bottom of the script and escalates through the backstop, which is the outcome those paths already had, so mapping optimistically loses nothing and gains the correct answer for every name that is a suite. packages/tests/ and every .gitattributes join inert_config_pattern, which is what the script's own header says to do with a genuinely inert path rather than widening the fall-through. packages/tests/ is a helm-unittest fixture chart: changing a test OF cozy-lib does not change cozy-lib, no PackageSource lists those paths as a component, and nothing installs them -- a change to the library itself still escalates through packages/library/. .gitattributes is matched by name wherever it lives because every one in this tree is a linguist-generated marker, which reaches no build, chart or test; one under a tree full_suite_pattern already covers still escalates, since that pattern is checked first. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
3c3f9450ae |
fix(ci): name the cause on every select-e2e escalation
Seven branches in hack/select-e2e.sh escalate to the full Chainsaw suite and four of them did it in silence: the shared hack/e2e-chainsaw/_lib/ and .chainsaw.yaml case, the full_suite_pattern match, a packages/ path no PackageSource claims, and the empty-selection backstop. "Run everything" is the same 21 suite names whichever rule produced it, so a reason line is the only thing that tells the seven apart. full_suite_pattern is the commonest cause of a full run by a wide margin -- 88 of the last 150 merged pull requests, 68 of them with no other cause -- so the usual answer to "why did this pull request run the whole suite" was the one the log never carried, recoverable only by re-deriving the selection by hand. Each new line names the path that caused it rather than only the rule, and the backstop names the directly-selected names it could not resolve. They go to stderr, never stdout: stdout is the suite list and both e2e lanes parse it, so a reason line there would be read as a suite name. A unit test asserts each of the four messages. A reason that regresses to silence changes no selection, so it is invisible to every other test in the file. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
d0990299ed |
feat(clickhouse): let the backup sidecar trust a private S3 CA, unblocking the backup e2e round-trip (#3385)
## What this PR does Gives the ClickHouse `clickhouse-backup` sidecar a way to trust an S3 endpoint whose certificate is signed by a private CA, and uses that to make the ClickHouse backup/restore e2e round-trip actually run in CI. This actualizes #2600, whose feature code and example scripts have since landed on `main` independently, and whose remaining contribution — the e2e test — was still in the retired BATS format. #2600 can be closed once this merges. ### 1. `backup.endpointCA` on the ClickHouse chart The sidecar had **no TLS/CA surface at all**: bucket coordinates and credentials arrive as `S3_*` env, and nothing let it verify an endpoint signed by a private CA. Cozystack's own in-cluster SeaweedFS is exactly that case — it serves `:8333` behind the self-signed "SeaweedFS CA" — so a ClickHouse release could only ever back up to a publicly-trusted endpoint. MariaDB (`storage.s3.tls.caSecretKeyRef`) and Postgres (barman `endpointCA`) have had a CA field all along; this closes the gap for ClickHouse. `backup.endpointCA` takes `{name, key}`. When set, the chart mounts that Secret into the sidecar and points `SSL_CERT_DIR` at it. Empty — the default — renders no env, no mount and no volume, so existing releases are byte-for-byte unchanged. **Why `SSL_CERT_DIR` and not `AWS_CA_BUNDLE`.** Both are honoured by `altinity/clickhouse-backup:2.7.4`; I measured them against a self-signed endpoint and public AWS S3 in turn. Go reads `SSL_CERT_DIR` *in addition to* the system bundle, so both endpoints verify. `AWS_CA_BUNDLE` *replaces* the SDK's trust pool: the private endpoint verifies, and public S3 then fails `x509: certificate signed by unknown authority` — it would silently break a release that also talks to a public endpoint. `S3_DISABLE_CERT_VERIFICATION` was rejected outright, since it drops verification rather than trusting one more CA. ### 2. The e2e suite, and what it was really testing Two Tests added to the existing `hack/e2e-chainsaw/clickhouse/` suite (in name order, mirroring the `etcd`/`mariadb`/`postgres` shape — see the layout rationale below for why they live here rather than in a new directory): - **`clickhouse-1-backup-contracts`** (CI, `tenant-test`, ~98s): a backup-enabled ClickHouse asserted against the *running* cluster — the `clickhouse-backup` sidecar on the `ch-backup-api` `:7171` port, and the `<release>-backup-api-auth` Secret whose keys the sidecar's `API_USERNAME`/`API_PASSWORD` actually resolve `valueFrom`. Each is a silent break if a chart bump renames it. Needs no object store. - **`clickhouse-2-backup-roundtrip`** (CI, `tenant-root`): `Bucket` → `Altinity` strategy + `BackupClass` → source ClickHouse with a sentinel row → `BackupJob` → in-place `RestoreJob` → to-copy `RestoreJob` into a second instance, driving `examples/backups/clickhouse/run-all.sh` so test and docs cannot drift. The round-trip was originally gated behind `CLICKHOUSE_E2E_S3_ROUNDTRIP` and **skipped in CI** — its `PASS (1.01s)` meant the gate fired, not that a backup happened. `backup.endpointCA` removes the reason for the gate, so the suite now adopts the same three preconditions mariadb/postgres rely on: run in `tenant-root` (an isolated tenant's Cilium egress allowlist blocks its Pods from reaching `tenant-root`'s SeaweedFS), target the in-cluster endpoint instead of the `Bucket`'s external ingress URL (an unroutable placeholder in CI), and trust its certificate via the copied CA. It also pre-cleans with `cleanup.sh`, because `tenant-root` is shared and a leftover `Succeeded` `BackupJob` would satisfy the harness's wait without a backup ever running. Both backup Tests live in the existing `hack/e2e-chainsaw/clickhouse/` suite rather than a new suite directory, so no selector mapping is added: `hack/select-e2e.sh` already derives `clickhouse` both from `packages/apps/clickhouse/**` (the `*-application` rule) and from `examples/backups/clickhouse/**` (the harness path), and both paths therefore select the suite that actually runs the harness. That alignment is the reason for the layout — a separate `clickhouse-backup/` directory would have made an edit to the harness select a suite that cannot exercise it. `hack/select-e2e.sh` and `hack/select-install.sh` are otherwise untouched (the only change is a comment recording that invariant), and the existing `backup example harness edit selects its app suite` case in `hack/select-e2e_test.bats` covers the mapping. ### Verification - `helm unittest` — 17 pass, including 5 new `endpointCA` cases: the unset default renders nothing, the env/mount/volume wiring, a custom key still projected as `ca.crt`, composition with `useSystemBucket`, and `backup.enabled=false` renders nothing. - The CA-trust behaviour was measured against the real image, not assumed: without a CA knob the sidecar reports `x509: certificate signed by unknown authority`; with `SSL_CERT_DIR` the handshake succeeds. - `hack/select-e2e_test.bats` (16) and `hack/select-install_test.bats` (17) pass; `select-install.sh --validate` reports graph OK. - `make generate` is idempotent, so the pre-commit job's `git diff --exit-code` stays clean. - The contracts Test passed in CI on earlier runs of this branch. The round-trip only became CI-live with this work, so it still needs a green E2E run before merge — the branch has since been rebased onto current `main` to get one. ### Screenshots Not applicable — no UI changes. ### Downstream repositories Walked the trigger map against the diff, file by file. The load-bearing change is the new `backup.endpointCA` field in `packages/apps/clickhouse/values.yaml` / `values.schema.json`; everything else is `hack/e2e-chainsaw/`, `examples/backups/`, one docs paragraph, and regenerated artifacts, none of which appear in the map. - [ ] No downstream repository is affected by this change - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/18 — "add, remove or rename a field in an app's `values.schema.json`" reaches the provider's hand-maintained schema, model and expand/flatten pair, so the new nested object is not surfaced until it is added there. Opened as an issue rather than a PR because the natural moment is alongside that repo's next `api/apps/v1alpha1` pin bump, which is a maintainer call. Not ticked, with reasons rather than silence: **cozystack/website** — the trigger is *adding, renaming or removing a package*, and this adds a field to an existing one. `clickhouse` is already in the website `Makefile`'s app list and its reference page is generated from this package's `README.md`, so the regenerated `backup.endpointCA` rows are carried by the existing release-tag automation with no manual PR. **cozystack/external-apps-example** — vendors `hack/package.mk` and `update-crd.sh`, neither of which this PR touches. **cozystack/ansible-cozystack**, **ccp**, **talm**, **cozyhr**, **cozy-proxy**, **cozystack-telemetry-server**, **examples** — no installer values, `hack/` layout, namespace, variant, node-prerequisite, annotation, label or metric in their trigger rows is touched here. ### Release note ```release-note feat(clickhouse): add `backup.endpointCA` so the clickhouse-backup sidecar can verify an S3 endpoint whose certificate is signed by a private CA, such as Cozystack's in-cluster SeaweedFS. Leave it unset for publicly-trusted endpoints; existing releases are unaffected. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional private S3 endpoint CA support for ClickHouse backup/restore sidecars via `backup.endpointCA`, including configurable Secret name and CA bundle key (default `ca.crt`). * Sidecars mount the CA and extend trust while leaving public endpoints unchanged. * **Documentation** * Updated ClickHouse chart docs, operational backup guidance, and backup/restore examples to cover CA setup, overrides, and cleanup. * **Tests** * Added helm-unittest and end-to-end coverage validating CA rendering/mounting behavior and ClickHouse backup/restore workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
587a7b9fbd |
fix(ci): decide E2E coverage per changed path
The full-suite escalation for a changed package that reaches no suite was
read off the merged selection, so it depended on the whole diff rather
than on the path that needed it. Any other changed path contributing a
suite name made the selection non-empty and the escalation vanished with
nothing in the output to record it: edit a platform component, adjust the
Chainsaw test beside it, and the component ran that single suite instead
of everything. Over one values.yaml per package under
packages/{apps,system,extra}, 82 of the 159 escalate on their own and
stop escalating once an unrelated suite edit joins the diff.
Coverage is now decided per changed path. Each path resolves through its
owning sources to the suites that exist, and a path reaching none of them
forces the full run whatever the rest of the diff selected. The unit is
the path rather than the source: system/postgres-operator belongs to two
PackageSources where one reaches no suite, so deciding per source would
escalate on that half and run everything for every change to it.
The escalation names its cause on stderr, as every other escalation in
this script already does. Most packages reach no suite of their own, so
this is the branch a full run usually takes; without a line saying which
sources bought it, "why did this run everything" has no answer in the log.
The walk moves into resolve_suites() to be called once per path. POSIX sh
has no `local`, so the function writes into the caller's scope and is
safe only inside the $( ) its single call site wraps it in; a test pins
that the call count stays one.
Two entries leave src_to_suites. `postgres-application` restated what
stripping the suffix produces and `external-dns` restated what carrying
the source name through produces, so both made the exception list read as
longer than the set of real exceptions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
0980e755eb |
fix(ci): map E2E suites from sources not named after an app
src_to_suites() covered PackageSource names ending in -application plus external-dns, and the filter after the graph walk dropped every other source it reached. cozystack.kuberture and cozystack.securitygroup- controller are neither, so both were walked to and then discarded: a CNI change ran neither of the two controller suites most likely to regress from it, and a change to either package reached no suite at all and escalated to the full run instead of selecting the one that covers it. Every reached source now maps through the table. That table is the inverse of select-install.sh's suite_to_source(), and a test walks every suite through both, so the next source name fitting neither convention fails there rather than quietly making its own suite unreachable. The cilium test measured the gap rather than hid it, asserting the 19 suites the selector could reach and naming the two it could not. The subtraction is removed rather than inverted: kept as an exclusion list it would stay green if the walk stopped reaching them for an unrelated reason. What replaces it is a second assertion against a tree holding one suite the graph reaches and one no source names, which an escalation cannot satisfy. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1abc653cfb |
fix(ci): treat a disabled chainsaw suite as inert in test selection
Editing hack/e2e-chainsaw/backup/chainsaw-test.yaml.disabled selects the whole Chainsaw suite. The per-suite rule reads the suite name off the directory and ignores the suffix, so the change selects "backup"; the final intersection against the suites that actually exist finds no such suite and empties; and the safety net for an empty selection escalates to the full run. A disabled suite is registered nowhere and executed by nothing, which makes its edits the one file class that provably cannot regress a test. The mapping had it backwards: they bought the most expensive outcome the selector has. Rare in practice, and paid on every commit by anyone grooming or re-enabling such a suite. Classify *.disabled as inert ahead of the per-suite rule, and record it alongside the other rules that select nothing so the header stays an accurate enumeration. The safety net keeps its job for paths no rule has looked at, which is what it was added for. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
37a395363f |
fix(ci): stop select-e2e from deciding on inputs it never checked
Three holes on the same seam: the script builds a list, never looks at whether the command that built it succeeded, and then decides from the empty result. The dependency indexes are built as `$(build_owners_index | sort -u)`, where the function is a single yq. A pipeline carries the status of its last command, so the script sees sort's zero and `set -e` never fires: a missing binary or a malformed source leaves the index empty, every changed path falls through to escalation, and CI runs the full Chainsaw suite on every pull request. The direction is safe and that is the problem — an empty index and "no owners matched" print the same thing, so the symptom is a full run, which reads as the conservatism the selector is supposed to apply. Selection can stay switched off indefinitely with nobody signalled. Check each half's status on its own and name yq once on stderr before taking the fallback the empty index would have reached anyway. The suite list one line up has the identical shape and a worse consequence: `$(find ... | sed | sort)` reports sort, never find, and errors went to /dev/null. That list is what every escalation prints. So a find that failed produced an empty list, and "escalate to the full suite" then emitted a blank line — which both e2e lanes read as "skip Chainsaw" before posting the required "E2E Tests" status green. The one outcome that means run everything delivered a run of nothing, on the pull request that earned a full run. Capture find on its own, check it, let its errors through. A failing find is not the only way to end up with an empty list, and the escalations are not its only reader, so the check sits once where the list is built. The examples/backups/<app>/ rule escalates nothing and still membership-tests against that list: with it empty the test fails, the app is not selected, and an edit to a backup harness that should run its suite reports nothing to run — output identical to a legitimately empty selection, on a path no escalation guard would ever see. An empty list is a broken enumeration, a moved directory or a wrong working directory, not a project without tests. Refuse to answer at all. Both lanes run the step under `bash -e`, so a non-zero exit fails the job rather than falling through to the empty selection this is here to prevent. Each is covered by a test that stubs the failing component and asserts both the exit status and that the script names what broke. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
8e273a7341 |
docs(ci): record what escalating on the non-PR workflows costs
Three of the four workflows on the escalation list cannot be exercised by the run they trigger. `e2e-fork` runs on `workflow_run`, which always takes the default-branch copy; `e2e-tag` runs on `workflow_call` from a tag push; `nightly` runs on a schedule. Only `pull-requests.yaml` is executed from the pull request's own head, so only there does escalating test the edit rather than the tree around it. That argument existed in the reference doc and was removed by this branch rather than answered, and the list it argued against is what replaced it. Measured over six months of main, 26 commits touch one of the three with no other escalating path; each now pays a full Chainsaw run it used to skip, on a suite whose flake rate is the reason people avoid running it for nothing. Keeping them is defensible: the rule stays one idea rather than two, and dropping them leaves those files inert, which reads as an oversight rather than a decision. What was not defensible was paying it silently. The trade now sits next to the pattern that makes it, with the number, so the next person weighing it starts from measurement instead of rediscovering the mechanism. The reference doc points there rather than carrying its own copy. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
55f3c79e9f |
docs(ci): stop pointing at the inert list for paths it does not hold
Four places said an empty selection means the inert pattern matched, and sent the reader to `inert_config_pattern` to see which paths those are. Documentation, dashboards and `*.md` are dropped before that pattern is ever consulted, and a backups example whose app has no suite is dropped where backup examples are matched. A reader checking the pattern for any of them finds nothing and has to reconstruct the order by reading the script. Name the rules that produce an empty selection instead, and keep the claim to the one the lanes rely on: empty is a decision some rule reached, not a path nothing looked at. The same bullet claimed every path is classified. One is not. The inert pattern makes `.github/` inert as a whole directory and the workflows that run the suite are exempted from it by name, so a workflow added under `.github/workflows/` reads as inert and the fall-through never sees it. Today's enumeration is correct and I checked it against `rg -l test-chainsaw`, so this is not a live hole, but the sentence saying it cannot exist is what makes the reminder next to that list look decorative. Record it as the guard it is. Separately, the fork lane's rationale for appending a newline described a mechanism this branch removes: the reader now guards its own loop, so the terminator is no longer what keeps the last path alive. Say that it is now the text-file contract and a second line of defence across two files, which is a reason to keep it and a different reason than the one written there. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
548dfa9520 |
docs(ci): say what an empty selection actually means
Four places on this branch describe an empty selection as "classified inert", against a list of three outcomes that every path is said to land in. There is a fourth: an edit under examples/backups/<app>/ where the app has no Chainsaw suite selects nothing, decided inline where backup examples are matched rather than through the inert list. The rule is deliberate and explained where it lives; it just is not one of the three the summary enumerates. The property the two lanes actually depend on is weaker and true: an empty selection means some rule decided the change cannot affect a suite, never that nothing looked at the path. State that instead, in the header that defines the classes and in the privileged lane's comment that consumes the result. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
cc522e7def |
fix(ci): classify the last path when the list has no trailing newline
The classifier this branch introduces is only total if every line reaches the loop that applies it. POSIX read assigns the final line and then returns non-zero when the input does not end in a newline, so a plain `while read` drops it. The fall-through that escalates an unrecognised path lives inside the loop body, so the dropped line is never classified at all and the selection comes back empty. Both lanes read empty as "skip Chainsaw" and post the required status green. The shape that matters is a list whose only unclassified path is last: `docs/x.md` followed by an unrecognised path with no terminator selects nothing, where the same list with a terminator escalates to all 21 suites. The safety net this branch is built around is bypassed by exactly the input it exists to catch. No caller triggers it today, because `git diff --name-only` terminates its output and the fork lane appends a newline explicitly. That makes it caller discipline documented in one of two callers, which the next caller will not read. Guard it where the assumption lives. Both new cases are red without the guard and the twenty-five existing ones stay green, so they pin this behaviour rather than the loop's general shape. They clean up inline instead of through an EXIT trap: that trap replaces the one the bats binary installs for its own bookkeeping, and a test failing under it can print no TAP line at all, which is the opposite of what a regression pin is for. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
ae4e9e55cb |
fix(ci): classify every TIA path and escalate the unclassified
`select-e2e.sh` silently selected nothing for a changed path it did not recognise outside `packages/`. Both e2e lanes read an empty selection as "skip Chainsaw" and then post the required "E2E Tests" status green, so an unrecognised path produced a green merge gate with no suite run. The worked example is `hack/common-envs.mk`, which supplies the tag, push and output flags of every image in the tree and matched nothing: `full_suite_pattern` covered `hack/*.sh` and `hack/*.bats` but not `hack/*.mk`. `pkg/`, `tools/`, `go.mod`, `hack/lib/` and `hack/buildkitd.toml` were in the same position, as was `.github/workflows/e2e-fork.yaml` while `pull-requests.yaml` was already escalated. A blanket "empty means full suite" fallback would be the wrong fix: it escalates the inert paths the design deliberately skips. Instead every path now lands in one of three classes by an explicit rule — the full-suite list for shared build inputs and the e2e harness, an inert list for paths that cannot affect what e2e exercises, and the PackageSource graph for `packages/(apps|system|extra)/`. A path matching none of them escalates and logs why, so the next path someone adds fails safe instead of failing open. The two lists are checked in that order, so a specific escalation beats a broad inert directory: `.github/` is inert, the workflows that run the suite are not. `*.md` and `docs/`/`dashboards/` stay ahead of both, so a README under `packages/core/` does not inherit that tree's escalation. Ten tests cover both sides of the classification. Mutation-checked: reverting the fall-through fails the unclassified-path test, dropping `.github/` from the inert list fails the non-e2e-workflow test, and swapping the two list checks fails the existing pull-requests.yaml test. Note the per-entry escalations are belt-and-braces — removing `hack/*.mk` from the pattern still escalates via the fall-through, which is the point of having it. Closes #3392 Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
82aa967674 |
test(e2e): correct the round-trip budget and its stale comments
Second review pass on this branch.
1. The op timeout equalled the inner sum instead of exceeding it: the comment
enumerated 3900s of inner waits and then set 65m, and 3900s IS 65m — so there
was zero headroom for the applies and SQL calls between those waits, and the
claim of "room" was false. Sized the inner budgets against what CI actually
takes instead (the bucket suite completes in ~37s and a backup-enabled
ClickHouse plus Keeper reaches Ready in ~98s, measured on run 30353894332):
bucket claim/access 300->180 and the three job waits 600->480, summing to
3300s, under a 60m op timeout. Shrinking the inner budgets rather than
raising the ceiling also keeps this suite closer to its postgres and mariadb
siblings on a 180-minute job cap.
2. The strategy Pod's securityContext comment still justified root with "Pod
runs apk add at startup", which the previous commit made conditional — the
default path now runs the platform's pre-baked image and installs nothing.
Scoped the note to the alpine fallback and said explicitly that the field set
matches cozy-default-altinity, runAsNonRoot deliberately unset in both.
3. clickhouse-backup.yaml still referenced clickhouse-backup-{1-contracts,
2-roundtrip}: the file moved unchanged, so the Test rename never reached its
own header and a reader following either name found nothing.
4. wait_sts_ready's timeout diagnostic could not print anything. It selected
clickhouse.altinity.com/chi=${name#chi-}, which yields
clickhouse-<app>-clickhouse-0-0 while the label carries the CHI name
(clickhouse-<app>), and 2>/dev/null then swallowed kubectl's "No resources
found". Ask for the StatefulSet's own first replica by name instead and let
both streams through; verified it now surfaces the Pod events that explain
why the replica never came up.
Also dropped the clickhouse bats case added in the previous commit: it asserted
what the generic examples/backups/<app> -> <app> mapping already guarantees, so
it did not lock the invariant its comment claimed. That invariant is now
recorded where it belongs — in the mapping's own comment in select-e2e.sh.
The dedicated ch-backup-client image this example now inherits is tracked in
#3420; noted there that the command: override is load-bearing in two files.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrey KolkovandClaude Opus 4.8
|
c1fd7db51d |
test(e2e): fold the clickhouse backup Tests into the app's own suite
Review findings on the previous two commits. 1. TIA no longer selected the suite that runs the harness. select-e2e.sh maps examples/backups/<app>/** to suite <app>, so with the round-trip living in a separate clickhouse-backup/ dir an edit to the harness selected `clickhouse` — which did not execute those scripts — while the suite that does was left out. Fixed the way postgres/mariadb/etcd already do it: both Tests move into hack/e2e-chainsaw/clickhouse/ as clickhouse-1-backup-contracts and clickhouse-2-backup-roundtrip. That also deletes every special case the earlier commits added (src_to_suites, suite_to_source, and the bats case that guarded them — both scripts are now byte-identical to main), and the suite list in hack/e2e-chainsaw/README.md stays correct because no dir is added. The bats case is replaced by one asserting the invariant that actually matters: an examples/backups/clickhouse edit selects the suite that runs it. 2. The docs claimed a fail-fast the harness did not have, and the budget was sized as if it did. wait_for_field polled to the full timeout with no terminal-value escape, and the HelmRelease/StatefulSet waits were bare `kubectl wait` with no existence backstop and no Stalled fail-fast. Ported wait_for_field's fail_value parameter and wait_hr_ready from the postgres helpers, added wait_sts_ready, and routed all eight waits through them. Verified against a fake kubectl: a Failed BackupJob now bails in 3s instead of 600s, a Stalled HelmRelease immediately. The inner budgets sum to 3900s, which the old 30m op timeout did not cover — a hang would have been SIGKILLed with no diagnostics, the failure mode .chainsaw.yaml calls strictly worse — so the timeout is now 65m with the arithmetic written out. 3. The strategy Pod ran `apk add --no-cache curl jq` on every backup and restore. Harmless while nothing in CI ran it; a per-PR dependency on the Alpine CDN once it does, and broken outright air-gapped — which is exactly why the shipped strategy bakes curl+jq into its image. Step 01 now reads that image off cozy-default-altinity and drops the apk add, falling back to alpine only when the platform strategy is absent. Also bounded the two polling curls with --max-time; a hung connection previously stalled the loop until the outer timeout. 4. 91-scenario-user-backup.md still documented the old tenant-test default. 5. .chainsaw.yaml claimed all app tests run in tenant-test, which the postgres and mariadb round-trips already contradicted before this branch. The round-trip header now also states its scope plainly: it exercises the example's strategy, not the shipped cozy-default-altinity. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
d8e003f4cb |
test(e2e): add clickhouse altinity backup chainsaw suite
Port the intended ClickHouse Altinity backup/restore e2e coverage from cozystack#2600 (never-merged hack/e2e-apps/backup-clickhouse.bats) to the current Chainsaw approach, reworked on origin/main where the Altinity backup feature and examples/backups/clickhouse already landed. Two Tests, etcd-suite shaped: - clickhouse-backup-1-contracts (CI): a backup-enabled ClickHouse, asserting the chart<->Altinity-strategy contract against the running cluster — the clickhouse-backup sidecar on the ch-backup-api :7171 port and the populated <release>-backup-api-auth Secret the strategy Pod authenticates with. Needs no reachable object store. - clickhouse-backup-2-roundtrip: the full Bucket -> BackupJob -> Backup -> in-place + to-copy RestoreJob round-trip, driving examples/backups/clickhouse/run-all.sh. Gated out of CI (CLICKHOUSE_E2E_S3_ROUNDTRIP=1) because kind's self-signed SeaweedFS S3 endpoint cannot be TLS-validated by the in-Pod sidecar. Wire clickhouse-application -> "clickhouse clickhouse-backup" in hack/select-e2e.sh so ClickHouse chart edits exercise the backup suite too. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey KolkovandClaude Fable 5
|
50fb3c87c2 |
test(postgres): run the backup round-trip e2e in CI under TIA
Ungate postgres-2-backup-roundtrip: instead of an opt-in env flag it now runs whenever TIA selects the postgres suite (postgres changes, and every release cut), like the other chainsaw tests. The blocker was network placement, not capability: the S3 client is the barman-cloud sidecar inside the Postgres Pod, and the isolated e2e tenant's Cilium egress allowlist blocks it from tenant-root's seaweedfs — so the Test runs in tenant-root (same-tenant egress) against the in-cluster S3 endpoint, trusting its self-signed cert via the copied seaweedfs-system-ca-cert. This is exactly the recipe validated end to end on a real cluster. Also map examples/backups/<app>/ edits to that app's suite in hack/select-e2e.sh (the round-trip executes those scripts as its harness; previously such edits were silently ignored), with bats coverage for both the mapped and the no-suite case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
dd2f89d208 |
test(sdn): port the securitygroup and serviceexposure suites to Chainsaw
Main added securitygroup.bats (#2922 membership model) and serviceexposure.bats (#3081 ExposureClass/ServiceExposure) after the Chainsaw migration branch was cut. - securitygroup: two Tests — the aggregated-API projection round-trip (backing CNP marker label, empty-value membership endpointSelector, lineage fromApp + membership fromSG peers, 1:1 rule translation, the view hiding storage internals, unmarked-CNP invisibility via an error op, delete propagation) and the controller membership-label stamp/clear lifecycle. Only the attachments annotation stays a jq script (it is a JSON string). Chainsaw auto-cleanup covers every fixture even on mid-test failure — the bats had no teardown hook and leaked its fixtures. - serviceexposure: the externalIPs backend resolve/Ready/assignedIPs path with the no-MetalLB-pool negative as an error op (ordered after resolvedBackend so it cannot false-pass), and the finalizer-removal test collapsed onto the delete op itself, which waits for full disappearance. - selection: kubernetes-application now also maps to the two OIDC render-side suites (they exercise the same app chart — a gap main's bats selection had too); the new dirs enter all_apps discovery automatically. Selector unit tests extended and green (13/13 under cozytest). - README: suites list updated. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Fable 5
|
9297e51170 |
Merge origin/main into feat/e2e-chainsaw-example
Reconcile the Chainsaw e2e migration with ~500 commits of main drift: - keep main's run-kubernetes.sh helpers (kubectl_wait_retry, tenant drain, LINSTOR pool-free wait, SC-fallback probe) under the branch's _lib/ rename with the Chainsaw finally/snapshot deltas re-applied - resolve modify/delete on 10 drifted hack/e2e-apps/*.bats as deleted; their post-branch assertion drift is re-ported into the Chainsaw suites in follow-up commits - drop the 4 new main-side bats (kubernetes-oidc-*, securitygroup, serviceexposure) from _lib/ relocation; ported as Chainsaw suites in follow-up commits - docs/agents/e2e-testing.md: merge main's plan-job TIA wording with the Chainsaw suite phrasing Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
IvanHunters
|
e0597e3d99 |
fix(platform): make *-rd sources depend on cozystack-engine (#3041)
## Problem On an unlucky Flux reconcile order, a `*-application` PackageSource that ships a `system/*-rd` component can install before the `applicationdefinitions.cozystack.io` CRD exists. Every `*-rd` chart renders an `ApplicationDefinition` (`cozystack.io/v1alpha1`); that CRD is installed by the `application-definition-crd` component of `cozystack.cozystack-engine`. When an `*-rd` HelmRelease renders first, it hard-fails with `no matches for kind "ApplicationDefinition" in version "cozystack.io/v1alpha1"`, and that cascades into a broad install timeout — surfacing even on changes that have nothing to do with ordering, because it is a latent race. ## Fix Add `- cozystack.cozystack-engine` to the `dependsOn` of every `*-application` source that ships a `system/*-rd` component (29 sources). `gateway-application` already carried this edge — it was added earlier for the TenantGateway CRD, the same bug class, but only for that one source. A new invariant test fails if any source with a `system/*-rd` component is missing the edge. ## Why The ordering was previously implicit, relying on Flux happening to reconcile `cozystack-engine` before the `*-application` sources. Making the edge explicit removes the race deterministically. No dependency cycle is introduced: `cozystack-engine` depends only on `networking` and `cert-manager` (plus keycloak in its oidc variant), none of which reach back to any `*-application`. ## Test - New generic guard `hack/check-applicationdefinition-crd-ordering.bats` (auto-discovered by `make bats-unit-tests`, runs in CI via `make unit-tests`): parses every source and fails if any PackageSource variant with a `system/*-rd` component lacks the `cozystack.cozystack-engine` edge — so future sources are covered, not just today's set. - `helm unittest packages/core/platform`: 13 suites / 66 tests pass; the source edits do not disturb existing assertions. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved dependency ordering across several platform packages to ensure related components install in the correct sequence. * Prevented end-to-end test selection from being overly broad when only ordering-related dependencies change. * **Tests** * Added checks to catch missing dependency requirements and to verify test selection behavior stays accurate. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
e22d84f2c8 |
test(e2e): migrate app suite from BATS to Chainsaw and wire into CI
Replaces the per-app BATS E2E suite (hack/e2e-apps/*.bats) with declarative Kyverno Chainsaw tests under hack/e2e-chainsaw/, extending this branch's pilot (postgres + bucket) to all 18 app suites. - Declarative ports assert on status/conditions and concrete fields; the repeated "timeout ... until kubectl get" + "kubectl wait" boilerplate collapses into a single polling assert with structured failure diffs and automatic events/describe/podLogs capture on failure. - Inherently imperative suites keep their logic in script steps: openbao init/unseal, kuberture split-horizon probes, vminstance, the gateway admission/impersonation cases, and kubernetes-latest/previous, which wrap the relocated hack/e2e-chainsaw/_lib/run-kubernetes.sh verbatim. - Gateway tests derive the tenant apex from the namespace host label at runtime, so they are host-independent. CI: the e2e job now runs "chainsaw test hack/e2e-chainsaw/" via a new test-chainsaw target and uploads the JUnit chainsaw-report.xml; the chainsaw binary is added to the e2e-sandbox image. install-cozystack and test-openapi stay BATS. The per-app 3-retry loop is dropped, since assertion polling replaces the fixed-timeout flakiness it papered over. Verified against a dev cluster (DB/app, storage and VM suites pass). Service port assertions use the ports[*].port projection form because Chainsaw v0.2.15 mis-evaluates JMESPath number-literal port filters. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
02eeb9e0d5 |
ci(e2e): keep the cozystack-engine ordering edge out of test-impact selection
Every *-application source declares dependsOn cozystack.cozystack-engine so its *-rd HelmRelease waits for the ApplicationDefinition CRD to register before it reconciles. That is a universal install-ordering edge, not a behavioral one. The test-impact selector reused the same dependsOn graph to pick which e2e app suites to run for a diff. Because the engine transitively depends on postgres-operator, keycloak and cert-manager, a change to any of those fanned out through the engine to every app, defeating test-impact analysis. Exclude cozystack.cozystack-engine as a propagation hub in the reverse-dep walk so the engine stays reachable but no longer pushes selection to its app-dependents. A change to the engine itself still triggers the full suite via the existing no-app-descendants safety net. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |