admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-11-09 14:02:56 +09:00
Code Issues Packages Projects Releases Wiki Activity
7,168 Commits 568 Branches 294 Tags
b4fbaf00a23691fee2fcc09638fbeca4584fae3c
Commit Graph
7168 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
IvanHunters b4fbaf00a2 test(e2e): keep gateway and redis-tls op ceilings above their own waits (#4643) 
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

Two Chainsaw script ops had a timeout smaller than the waits inside
them. When the op ceiling fires first, Chainsaw kills the shell: the
script never prints its own error, and a cleanup trap never runs.

The gateway listener-set step could poll for 660s under a 6m op. Its
cleanup trap also deleted two tenants with no limit on the wait. Each
delete now runs under an outer `timeout`, because kubectl's own
`--timeout` does not cover discovery or a stalled request. A tenant
delete gets 300s, the same 5m that `.chainsaw.yaml` gives a delete op.
The parent is deleted only after the child is gone, because deleting a
parent while its child is still uninstalling wedges the parent's
cleanup. A failed tenant delete fails the step with kubectl's own error,
so a leaked tenant shows up here and not in a later suite. The op is now
24m, which covers the polls plus those bounds.

The redis-tls CA-projection step has 30s read caps that add up to well
over its 5m op. Raising the op would not hold, because several reads run
in loops. So all reads now share a 240s step budget. No read starts
after the budget runs out, and a read that is still running gets cut off
when it does. A read that hits its bound, or finds the budget already
spent, says which read it was.

Two exec checks in the same step blamed the 15s redis-cli cap for any
failure, including a refused exec or a missing container. They now print
the exit status and what the exec printed.

The step also runs under bash now, for pipefail. The sandbox's `/bin/sh`
is dash, which rejects `set -o pipefail`. Chainsaw 0.2.15, the version
the sandbox image installs, supports a `shell:` field on script ops.
Without pipefail, a failed Secret read took base64's zero exit status
and was reported as a wrong source name.

The private-key check read the projection a second time, so a failed
second read made it pass. It now checks the value the step already
fetched.

I added `hack/chainsaw-op-budgets.bats` to guard all of this. It holds
the gateway op above its polls plus its delete bounds, keeps the
child-before-parent order, holds the redis budget under its op with
every kubectl call going through the clamped helper, and runs the redis
step's script against a stubbed kubectl to check each failure message
and the clamp. On main, 11 of its 12 tests fail. The one that passes is
the healthy-stub control.

Closes #4114

Closes #4250

Closes #4246

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff touches two Chainsaw suites and adds one unit test file under
`hack/`. It moves or renames nothing a downstream repository reads.

### Release note

```release-note
test(e2e): the gateway listener-set and redis-tls projection steps now fail on their own errors instead of being killed by the op timeout, and redis-tls reports the real cause of a failed read or exec.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Added automated checks to verify gateway operations have sufficient
time to complete and cleanup deletions are bounded and correctly
ordered.
* Expanded gateway scenarios to cover cleanup failures: if child-tenant
deletion fails or times out, the parent is left in place; deletion
failures are reported.
* Updated Redis TLS checks with bounded read times and a shared
execution limit. Failure reports include command output and status, and
checks cover private-key material in the decoded projection.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-05 14:07:37 +03:00
IvanHunters 1ca603a7b2 test(e2e): wait for the S3 gateway to accept a key before using it (#4694) 
## What this PR does

The bucket e2e suite fails now and then because the S3 gateway rejects a
key that COSI already reports as granted. The error is `The access key
ID you provided does not exist in our records.` from `mc alias set`. In
job 109130131391 the BucketAccess assert finished at 21:23:43.57 and the
first `mc alias set` failed at 21:23:44.82. In job 109396174094 cap1 and
cap2 were accepted, and cap3 was rejected two seconds into the step.

Granted access is not the same as a usable key. In seaweedfs-cosi-driver
v0.3.1, `DriverGrantBucketAccess` writes the new key into
`/etc/iam/identity.json` on the filer and returns. The S3 gateway in
SeaweedFS 4.45 reloads that file later, from a filer metadata
subscription (`weed/s3api/auth_credentials_subscribe.go`). In our chart
the driver writes through `seaweedfs-filer` and the two gateways follow
`seaweedfs-filer-client`, so the change may have to pass through the
other filer replica first.

The suite now waits until the gateway accepts the key before it writes.
Every alias in both write steps goes through `await_key`, which uses the
same port-forward as the writes and so asks the same gateway pod. It
waits only while the answer is the InvalidAccessKeyId text, for at most
60s. Any other error fails at once, and so does an attempt cut off at
its 15s cap. When the window runs out, the step fails with a message
that names the BucketAccess.

One probe costs at most 77s, so both steps go from a 5m to an 8m op
timeout. The inner budgets now sum to 409s and 441s, and the budget
comments say so.

I checked the probe locally against a fake `mc`:

```text
case=ok rc=0 attempts=1 elapsed=0s
  Added `a` successfully.
case=lag rc=0 attempts=3 elapsed=5s
  Added `a` successfully.
case=other rc=1 attempts=1 elapsed=0s
  FAIL: mc alias set for BucketAccess bucket-test-admin failed (exit 1) with an error other than the key not being loaded yet
case=never rc=1 attempts=31 elapsed=60s
  FAIL: S3 gateway still rejects the key of BucketAccess bucket-test-admin after 60s of probing (the gateway never loaded it, or the driver never wrote it)
```

A hanging `mc` fails on its first attempt after 15s with exit 124.
`chainsaw lint test` from v0.2.15 accepts the file.

The read-only upload check gets fixed along the way. Before, an unloaded
`ro-user` key also made the upload fail, and that failure passed as the
expected denial.

One thing I left as it is: `await_key` is copied into both steps,
because chainsaw steps share no shell, and moving it to `_lib/` would
run every suite on each edit.

Closes #4580

### Screenshots

Not applicable, no UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
test(e2e): the bucket suite waits for the S3 gateway to accept a new key before writing with it, instead of trusting BucketAccess readiness.
```
 2026-10-05 14:07:16 +03:00
IvanHunters 858ca4d0a3 test(e2e): cover monitoring OIDC modes with a Chainsaw suite (#4678) 
## What this PR does

Monitoring OIDC gets live-cluster e2e coverage again. It had none since
the two pre-Chainsaw BATS suites were removed without a port. This adds
a Chainsaw suite in `hack/e2e-chainsaw/monitoring/` that ports what they
asserted, plus the in-place switch from an inline config to a
`secretRef` that nothing covered.

The suite takes one Monitoring release in `tenant-test` through three
modes in a row: System, CustomConfig with an inline config, and
CustomConfig with a `secretRef`. It creates that release itself and
never reads the tenant-root monitoring instance, so it keeps working if
#4023 or #4479 turns root monitoring off.

What each phase checks:

- System mode: cozystack-api admits `spec.oidc` and both HelmReleases go
Ready. The Grafana CR points `auth.generic_oauth` at the cozy realm, has
no `role_attribute_path`, and has `skip_org_role_sync` and
`oauth_allow_insecure_email_lookup` forced. The KeycloakClient is
confidential and references the generated Secret, which holds 32
alphanumeric characters. The audience scope is bound to this release's
clientId. There is no chart-owned `KeycloakRealmGroup`. The users-Job
carries the users and the mode, and `grafana-deployment` reads the
client secret from the Secret.
- Switch to inline: the Grafana CR takes the tenant's map, keeps the
forced keys and drops every System-only key. The users-Job reruns with
the new list. The KeycloakClient, the audience scope and the client
Secret are pruned, and the Deployment stops referencing that Secret.
- Switch to `secretRef`: the inline `auth.generic_oauth` block is gone
from the Grafana CR. The operator Secret is mounted at
`/etc/grafana/oidc` on both the CR and the Deployment. This checks the
mount only and does not prove Grafana loads the file, see #4679.
- After the install and after each switch, neither release may show a
remediation cycle. That means no `failed` or `uninstalled` snapshot, and
the latest revision must be exactly 1, then 2, then 3. The revision
check is there because helm-controller truncates history after the
previous deployed snapshot. An upgrade that failed, rolled back and then
passed on retry leaves no failed entry, only extra revisions.

I did not assert `GF_PATHS_CUSTOM_INI`. I could not find anything in
Grafana that reads it (#4679), and pinning it would make possibly dead
wiring part of the contract.

The coverage split with the render tests is this.
`packages/system/monitoring/tests/oidc_test.yaml` keeps the
fresh-install render of each mode, including the CustomConfig negatives
(no Keycloak objects, no client Secret). This suite covers what a render
cannot see: admission, the Deployment the Grafana operator builds, and
what a switch between modes prunes or leaves behind. The Keycloak
objects are checked as written, so creating the client in Keycloak is
not asserted. Removing it is, because the operator's finalizer holds the
KeycloakClient until the client is gone from Keycloak.

Before the first apply the suite also waits until the tenant's
`cozystack-values` carries `oidc-enabled: "true"`. The outer release
copies `_cluster` into the inner release once, so a Monitoring applied
right after the tenant is created can render from stale values and fail
the System-mode guard for the full 15m. That is what the first CI run on
this PR hit.

The Keycloak CRDs are required, not probed for. The e2e install always
enables platform OIDC, so a missing CRD is a broken install. The removed
suites skipped in that case, and their probe could not tell an absent
API group from a present one.

The selector picks the suite up by name from
`cozystack.monitoring-application` and `cozystack.monitoring`. A change
to `system/monitoring`, `extra/monitoring`, `monitoring-rd` or
`grafana-operator` now runs only this suite instead of the full run. The
tenant-root Grafana bring-up stays covered by the install BATS while it
exists. One selector unit test relied on `cozystack.monitoring` reaching
no suite, so it now renames that source in its copy of the graph.

The second commit keeps Keycloak changes on every suite that drives a
Keycloak client. Before this PR a change to `system/keycloak` or
`system/keycloak-operator` escalated to the full run. With the
monitoring suite it would select monitoring alone, because
`monitoring-application` is the only source that depends on
`keycloak-operator`, and `kubernetes-latest` with its OIDC lifecycle
would drop out. `src_to_suites()` now maps both to `kubernetes-latest
monitoring`. The mapping has to be explicit because
`kubernetes-application` has no `dependsOn` on `keycloak-operator`
(#4680). A `postgres-operator` change now also reaches
`kubernetes-latest`, through Keycloak's database. The selector comment
and the e2e doc no longer use `system/postgres-operator` as the example
of a path with a source that reaches no suite.

Cost: one monitoring stack bring-up, two in-place upgrades and teardown.
I expect 10 to 20 minutes on every full run. The selector change makes
the CI run on this PR a full run, so the suite's own timing is in the
chainsaw report. A red run can spend up to 16m on each of four waits,
since each one sits a minute above the 15m timeout of the release it
waits on.

The suite cannot run locally, so I checked it this way:

- `chainsaw lint test` at v0.2.15, the sandbox version, passes.
- I ran the suite's own steps against a local envtest apiserver.
Chainsaw applies the real manifests, and a stand-in for helm and the
operators renders the chart from the stored Monitoring spec. With the
chart intact all nine steps pass. Each of 30 single breakages of the
chart or of the simulated controllers turns the expected step red.
- `hack/select-e2e_test.bats` and `hack/select-install_test.bats` pass.
The new Keycloak selection case fails without the mapping and when
`kubernetes-latest` is dropped from it.

A browser login through Keycloak is out of scope, same as in the removed
suites.

Left for later:

- KeycloakClient and scope `status` after the operator creates them in
Keycloak is not checked.
- `hack/e2e-chainsaw/README.md` lists suites by hand. I added
`monitoring`, but `backup`, `kafka-metadata` and `kubernetes-proxmox`
were already missing.
- The `operator diff selects all dependent app suites` selector test
could also check `monitoring`.
- The comment on the Keycloak mapping says both suites wait for the
finalizer to remove the client. The monitoring suite does wait.
`kubernetes-latest` only reads once after its upgrade.
- The ready checks compare `observedGeneration` with
`metadata.generation` through `contains()`. Both are fields of the same
object, not a number literal, so the v0.2.15 literal gotcha should not
apply, but no live run has passed this suite yet.
- The inline fixture carries no `enabled: "true"`, so that phase checks
a disabled OAuth block. The asserts do not depend on it.

Closes #4051

Closes #3785

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added end-to-end coverage for Monitoring installations using system
OIDC, inline custom configuration, and Secret-based configuration. The
tests verify readiness, rendered resources, configuration cleanup, and
upgrade history.
* Updated test selection so Keycloak-related changes also run the
Monitoring suite.
* **Documentation**
* Added Monitoring to the documented Chainsaw suite list and clarified
how changed paths determine test coverage.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-05 14:07:10 +03:00
IvanHunters 6e4c9c4b6a fix(ci): harden the PR labeler's title parsing and map recurring scopes (#4666) 
## What this PR does

Fixes three problems in `.github/workflows/pr-labeler.yaml`. Each one
either killed the job or put a PR in the wrong area, and none of them
showed up anywhere.

- Both maps are now looked up with `Object.hasOwn`. A title typed or
scoped `constructor`, `toString`, `hasOwnProperty` or `__proto__` used
to resolve to an `Object.prototype` member and crash the step on
`l.startsWith`, which left the PR with no labels at all.
- Every leading `[Backport ...]` prefix is now stripped, not only the
first one. With two prefixes, the second was read as a bracket scope, so
the real scope and the `kind/*` label were both lost. The `area/release`
label hid the problem by keeping `area/uncategorized` from firing.
- Scopes that recur three or more times over the full PR history now map
to an area. So does each area's own name (`networking`, `storage`,
`database`, `virtualization`, `testing`, `dependencies`).

Scope-to-area decisions:

- `area/database`: `database`, `etcd-operator`, `opensearch`,
`opensearch-operator`
- `area/networking`: `networking`, `kubeovn`, `multus`, `kilo`,
`hetzner-robotlb`, `ouroboros` (it rewrites the CoreDNS Corefile to fix
hairpin NAT, so it is networking, not platform)
- `area/virtualization`: `virtualization`, `vm`, `kubevirt-operator`,
`kubevirt-cdi`, `kubevirt-instancetypes`
- `area/platform`: `operator`, `cozystack-operator`, `controller`,
`cozystack-controller`, `cozystack-basics`, `flux-shard-operator`,
`migrations`, `scheduler`, `cozystack-scheduler`, `reloader`, `bootbox`,
and `lineage`. I put `lineage` here on purpose: no existing label covers
it more precisely, and it is too narrow for an area of its own.
- `area/storage`: `storage`, `piraeus-operator`,
`backupstrategy-controller`
- `area/release`: `changelog`, `changelogs`
- `area/ci`: `workflows`, `labeler`
- `area/dependencies`: `dependencies`, `renovate`
- `area/api`: `registry` (the cozystack-api storage layer)
- `area/dashboard`: `dashboard-controller`
- `area/monitoring`: `monitoring-agents`
- `area/keycloak`: `keycloak-configure`
- `area/testing`: `testing`
- New `area/dx`, added to `.github/labels.yml`: `dx`, `hack`, `cozypkg`.
These are developer tooling (hack/ scripts, package Makefiles, cozypkg).
`area/build` covers image builds and `area/testing` covers tests, and
neither describes this work.

Left unmapped on purpose:

- `http-cache`: the application was removed from the tree.
- `openbao`: already mapped on main.
- `apps`, `system`, `docs`: catch-all scopes that don't point to one
area.
- `fix`, `bugfix`: malformed titles.
- `security`, `rbac`, `governance`, `codeowners`, `maintenance`,
`images`, `readme`, `telemetry`, `cluster-autoscaler`: no single area
fits any of them.

I did not add spelling normalisation. `kubeovn` got its own row, since
it is the only alternate spelling that recurs.

A new contract test, `hack/pr-labeler-contract.bats`, pulls the
workflow's own `script:` block out of the file and runs it under node
with stubbed `context`, `core` and `github`, so it tests the shipped
code, not a copy. On main it is red on the prototype titles (`THROW:
l.startsWith is not a function`), on the double backport (only
`area/release`), on the newly mapped scopes and on the area-name scopes.
It also checks that every map target exists in `labels.yml` and that no
key is written twice. The suite fails rather than skips when node is
missing. The older node-based suites skip, so this is the first one that
needs node on the unit-test runner.

Some things stay out of this PR. The scope table in
`docs/agents/contributing.md` is a summary. It has no row for `area/dx`
and does not mention that an area's own name works as a scope.
Separately, the singular `migration` still maps to `area/virtualization`
as it did before, while the new `migrations` maps to `area/platform`.

A backport PR whose scope is unmapped still never gets
`area/uncategorized`, even with a single prefix. The `area/release`
added by the strip satisfies the fallback check. This behaviour is the
same on main and is not changed here.

The test that checks map targets against `labels.yml` passes by itself
if the map literal cannot be extracted. The duplicate-key and area-name
tests go red in that case, so the suite as a whole still fails.

Closes #4125, closes #4124, closes #4122

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(ci): the PR labeler no longer crashes on titles scoped like Object.prototype members, handles nested backport prefixes, and maps more recurring scopes, including a new area/dx.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added the `area/dx` label for issues and pull requests related to
developer tooling.
* **Improvements**
* Pull requests can now be categorized using a broader set of scope
aliases.
* Repeated leading backport prefixes are recognized and route the pull
request to the release area.
* Label mapping now avoids treating inherited property names as valid
mappings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-05 14:06:47 +03:00
IvanHunters 02fbe76418 fix(ci): close three blind spots in the e2e test selectors (#4658) 
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

The e2e selectors miss Chainsaw suites that have no `chainsaw-test.yaml`
or `.yml` directly in their directory. Chainsaw v0.2.15 also runs a
directory that holds only numbered step files such as `01-install.yaml`,
and its discovery walks every subdirectory of the path the runner hands
it. A suite in either layout ran only when nothing was selected. No
narrowed selection named it, the full-suite list left it out, and
`select-install.sh --validate` never asked for its mapping. Now
`hack/select-e2e.sh` and `hack/select-install.sh` list a top-level
directory as a suite when Chainsaw would load a test from it or from
anywhere below it. No suite in the current tree uses either layout, so
today's suite list stays the same.

A file placed directly in `examples/backups/` (a shared `common.sh`, for
example) selected nothing, because it fell through to the inert
`examples/` rule. It now escalates to the full suite, same as
`examples/backups/_lib/`. `README.md` there stays inert through the
`*.md` rule.

The `test-chainsaw` comment in `packages/core/testing/Makefile` said
Chainsaw discovers every `chainsaw-test.yaml`, which was too narrow. I
fixed it, and the matching sentences in `docs/agents/e2e-testing.md`
too.

Every behaviour change has a bats test that fails against the scripts on
main.

Known gaps I left for a follow-up, none of them a wrong answer today:

- The `full_suite_list` helper in `hack/select-e2e_test.bats` repeats
the selector's `find`/`sed`, so a bug in that shared pattern would pass
every full-suite assertion. The step-file and nested tests compare
against literal lists, which covers the new layouts.
- The selectors are a little wider than Chainsaw: a broken
`chainsaw-test.y*ml` symlink, or one with no Test in it, still lists its
suite. That errs toward running more.
- A few test comments describe what used to happen rather than why the
rule exists.

Closes #4630

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(ci): the e2e test selectors now see Chainsaw suites made of numbered step files or nested below a suite directory, and run the full suite for an edit to a file placed directly in examples/backups/.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* E2E suite discovery recognizes supported manifest and numbered-step
files at any depth and maps them to the top-level suite.
* Changes to files directly under `examples/backups/` trigger full-suite
selection.
* Suite selection reports an error if discovery fails or finds no
suites.
* **Documentation**
* Clarified supported suite layouts, disabled suite-file handling,
discovery and escalation rules, and per-path coverage guidance.
* **Tests**
* Added coverage for nested suites, numbered steps, unmapped suite
layouts, and backup-path selection.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-05 14:06:18 +03:00
IvanHunters 9485f9118b test(tests): make negated BATS assertions able to fail (#3790) 
## What this PR does

A hundred and twelve assertions across twelve BATS suites begin with
`!`. A hundred and seven of them pass no matter what they find. POSIX
exempts a command whose return value is inverted with `!` from `set -e`,
so the failing status is dropped where it is produced, and
`hack/cozytest.sh` closes each test function with `return 0`, so it
never resurfaces as the test's own status either. The other five are
followed by `|| { ... exit 1; }`, which reads the inverted status and
does work; they are rewritten too, because telling the two apart needs
the tail of every statement parsed, and a guard that guesses wrong there
is green on a real one.

Switching to the `bats` binary does not fix the hundred and seven. Its
ERR trap carries the identical exemption: `bash -c 'set -eE; trap "echo
TRAP" ERR; ! true; echo REACHED'` prints `REACHED`, runs no trap and
exits zero. The only negated assertion `bats` reports is one standing
last in the test body, where the inverted status happens to be what the
function returns. Two of the hundred and seven stood last; the rest did
not.

What was lost is specific rather than diffuse. `!` is how you spell
"this must not appear", and an absence is what no other assertion in a
test covers. The affected suites assert that a migration ran nothing
destructive, that no image was copied, that a flag was not passed, that
a backup cleanup never flushed the master, that an object does not
exist. On that half of their contract they held for any input.

Each site is now written as `if cmd; then echo "FAIL: ..."; false; fi`,
the form already used in the files where this had been noticed one at a
time, and each carries a message naming what must not have happened.
`hack/bats-no-negated-assert.bats` scans every `hack/**/*.bats` and
refuses the old form, so a new one cannot be added silently. Inverted
conditions (`if !`, `while !`, `until !`) are untouched, because `set
-e` is not what decides a condition. `[ ! -f x ]` is untouched for a
different reason and remains an ordinary failing assertion: the `!`
there is an operator of `test`, not negation of a command, so nothing
exempts its status.

The guard pins its own premise instead of describing it, on both sides.
One test asserts the runner still passes a bare negated assertion; a
second, that neither `set -e` nor the ERR trap sees an inverted status;
a third, that the prescribed `if cmd; then ...; false; fi` does fail,
under both runners, and *at* the form rather than somewhere before it,
since a run that died early would satisfy a bare status check and pin
nothing.

Every rewrite here rests on the third test. If `false` inside a `then`
body ever stopped propagating, the first test would still pass, because
a negated assertion would still be vacuous, while every converted
absence went silently green. A ban that prescribes a replacement has to
test the replacement too.

Every claim the guard makes is pinned by removing the thing it claims.
The recursion is exercised *through* the audit function against a
fixture in a subdirectory, not through a second copy of its `find`
expression: a copy keeps passing while the function loses its recursion,
and an audit that returns empty reads as a clean tree. Recursion matters
because `BATS_UNIT_FILES` is a non-recursive `wildcard`, so `make
unit-tests` never runs a suite placed in a subdirectory.
`hack/e2e-apps/` held such suites until they moved to Chainsaw and is
gone now, which is why the test pins against a fixture and not a real
directory.

The rule's surface was checked in the other direction too. The shape
appears in no `hack/**/*.sh` script as an assertion: the one hit,
`hack/testdata/install-wait/call-shapes.sh`, is fixture text that
`hack/run-kubernetes-install-wait_test.bats` parses, not a command
anything runs. It appears in no Chainsaw `script:` step either, so
`.bats` is the whole surface rather than the part that was convenient to
guard.

The scan is lexical, so its header names the statement boundaries it
owns rather than implying it understands shell: `;` `&` `&&` `||` `{`
`(`, the `)` closing a case pattern, and the words `then` `do` `else`,
each measured against the runner before being added. A keyword may
legally be preceded only by whitespace or by `;`, since every other
separator in front of `then`, `do` or `else` is a shell syntax error,
checked with `sh -n` rather than recalled. The scan therefore admits
exactly those two, and that part of the class is closed rather than
patched one spelling at a time. All three keywords are vacuous after
either spelling, so missing one is a silent miss, the single direction
this scan must not err in, given that its whole design trades noise for
safety. For the same reason it takes `!(cmd)` with no space before the
subshell, which is vacuous on dash and bash alike, and reads through a
`time` or `time -p` prefix, which is vacuous under `bats` and under
cozytest wherever `/bin/sh` is bash.

Five over-counts remain and are deliberate, each named in the header: `(
! cmd )` does propagate its inverted status and is reported anyway; so
would a `!` opening a statement after a redirection `&`; `x=$(true || !
false)` is reported although the `||` reads the inverted status; a
condition continued by a trailing `&&` instead of a backslash is
reported, because the strip only recognises a condition whose end it can
see; and `time ! cmd` is reported although on a dash `/bin/sh`, which
has no `time` keyword, it runs an external `time` that fails. Each costs
one rewritten line, where the opposite error costs a silent pass. They
are enumerated rather than engineered away on purpose, because a removed
error leaves no trace while a listed one tells the next reader what they
are looking at. The scan also errs *quiet* in at least two places, named
here rather than in the header. A file cut off midway through a line
continuation loses its last buffered statement, but that input is not a
runnable test, since `sh -n` rejects it with `unexpected end of file`.
And `time -- ! cmd`, which bash accepts, is vacuous under `bats` and not
peeled by the scan; nothing in the tree uses it, and I stopped adding
spellings there rather than chase every prefix bash allows. The test
naming the shapes claims only what its fixture holds. A lexical scan
over shell cannot promise it has enumerated the language, and a name
that promised it would be false whether or not a gap existed today.

The branch name is narrower than the subject and I am leaving it as it
is rather than churn it. This is not a defect in `hack/cozytest.sh` that
moving to the `bats` binary would fix. It is a POSIX rule both runners
inherit, and moving to `bats` would close two of the hundred and seven
sites.

One suite went red for real once its assertions could fail.
`seaweedfs-guard-parity` forbids four mutable discriminators by grepping
each chart, and both charts carry a template comment block arguing why
those signals were rejected, so the grep matched the argument and would
have reported a chart for documenting the rule it follows. The charts
are correct; the assertion was not. It now strips `{{/* ... */}}` before
matching. Two details there are load-bearing and neither is obvious. The
strip removes the comment *span*, not the line that opens it: dropping
the line is shorter and silently loses a discriminator sharing a line
with a comment, which is a chart classifying on a forbidden signal and a
test reporting that it does not. And the strip reports when the file
ended inside a comment, and the checks refuse to read its output when it
did, because every `{{/*` is read as an opener, so one inside a quoted
string (these charts carry long `fail (printf ...)` messages) would
swallow the rest of the file and leave the checks reading nothing.

One commit widens six comments that explained this trap as applying to a
"negated pipeline". It is not about pipelines, and the narrower wording
invites the conclusion that the other shapes are safe. Comment text
only; each of those files already uses the working form.

The last commit makes the one absence check in
`hack/e2e-install-cozystack.bats` that already used the `if` form tell a
missing object from a failed read. It asserted that the version
configmap was gone on `kubectl get`'s exit status alone, and that status
is 1 both for a NotFound and for an api-server the test cannot reach.

### Verification

Every unit suite this branch touches was run through `hack/cozytest.sh`
on the current base and is green. `migration-seaweedfs-db-adopt` passes
15 of 15 on macOS only when its temp directory sits under `$HOME`. Its
container cannot write into a bind mount under `/var/folders`, and
`/usr/bin/mktemp -d` ignores `TMPDIR`, so the run needs GNU `mktemp`
first on `PATH` as well. Measured on Linux before the rebase, it passed
15 of 15 as is.

The six sites that reached main after this branch was first written, in
`redis-backup-strategy` and `proxmox-e2e-fixture_test`, were each shown
to be able to fail: with the condition inverted, the suite goes red at
that assertion and prints its own `FAIL:` message. The nine sites in
`migration-seaweedfs-db-adopt` carry a two-sided check as well: with the
pattern swapped for one the same test proves is present, the rewritten
form fails the suite and the original `!` form passes it.

The guard itself is green under cozytest with bash and with dash as
`/bin/sh`, under the `bats` binary, and with mawk, gawk and BSD awk.
Each branch of its separator class and of the `time` and `!(cmd)`
handling was removed in turn, and each removal turns the shapes test
red.

`e2e-install-cozystack.bats` is an e2e suite and runs only in the E2E
workflow, so its one changed absence check is not exercised locally.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff file by file. The change touches
eighteen existing BATS files, adds one, and adds a convention to
`docs/agents/e2e-testing.md`. It does not touch
`hack/e2e-prepare-cluster.bats`, which is the node contract
`ansible-cozystack` and `talm` restate; it moves and renames nothing
under `hack/` and changes no make target's behaviour, which is what
`ccp` gates on; and it touches no package, schema, enum, default, CRD,
label, annotation or metric.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```

Closes #3565
 2026-10-05 14:05:42 +03:00
IvanHunters 94abf6940b feat(kubernetes): let kubevirt-csi detach without the HotplugVolumes gate (#4451) 
## What this PR does

This lets kubevirt-csi detach volumes without the deprecated KubeVirt
`HotplugVolumes` gate. The default stays as it is, so nothing changes at
runtime yet. Part of #4032.

The driver got a switch upstream for exactly this, and I bumped to it.
kubevirt/csi-driver#382 added `--enable-vmi-hotplug-fallback`, merged as
cb4770e4e475. With it set to false, the driver no longer calls the
VMI-level `removevolume` on a VMI owned by a VM. virt-api rejects that
call once `HotplugVolumes` is off.

Our wrapper now passes the flag to upstream and no longer has its own
hot-unplug path. `unpublishHotplugVolume` was a backport of
kubevirt/csi-driver#184, and the previous pin already contained that
change. Upstream detaches from the VMI when the VM is gone, then waits
for the VMI status to drop the volume before it returns. So the
wrapper's re-check never found the volume attached. Adding the ownership
check there would have been dead code.

Closes #4450

The chart leaves the flag at its default of true and does not render it.
The driver image in the chart is stamped only at release preparation, so
between releases main runs the previous release's binary, which does not
know the flag. Go's `flag` package exits on an unknown flag. Rendering
`=true` would crash-loop the CSI controller for no effect. The right
value depends on the KubeVirt CR, not on the tenant, so the comment next
to `HotplugVolumes` in `kubevirt-cr.yaml` points at the controller
Deployment. The follow-up PR drops the gate and renders the flag as
false.

Nothing changes on the node side. Main already pins f5de10d39556, and
cb4770e4e475 is that commit plus kubevirt/csi-driver#382.

A Go unit test runs an unpublish through the wrapper during a teardown,
with the flag on, off with a VM-owned VMI, and off with an orphaned VMI.
It checks that the flag reaches upstream and that only upstream makes
the VMI-level call. Upstream's own tests cover the ownership rule. A
second test holds that an NFS volume only releases the node from its
CiliumNetworkPolicy and never reaches the upstream hot-unplug. A
helm-unittest pins that the chart renders no such argument. I haven't
built the image or run e2e. Volume attach and detach on a tenant cluster
in e2e is the real check.

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(kubernetes): kubevirt-csi supports --enable-vmi-hotplug-fallback, so it can detach volumes without the KubeVirt HotplugVolumes feature gate. The flag stays true for now and behaviour is unchanged.
```
 2026-10-05 10:46:12 +03:00
Aleksei Sviridkin 091b8549c2 build(images): run the tests of the packages the patches touch (#4326) 
## What this PR does

Twelve first-party images that are built from an upstream tree with
patches on top now run `go test` on the packages those patches touch.
The step sits in the builder stage, right after the patches are applied.
Before this, ten of them ran `go build` only, and `go build` never
compiles a `_test.go`, so a test carried inside a patch was proven once
by hand and never again. The other two, opensearch-operator and
kube-ovn, ran a `go test` line written by hand, which a patch reaching a
new directory would slip past; the loop replaces both lines.

The package list is read from the patches themselves with `git apply
--numstat`, so a new patch is covered without an edit to the Dockerfile.
Each package is tested from its own directory, which finds nested
modules: in objectstorage, `internal/` is a module of its own and the
root module cannot name `./internal/runtime`. An empty list runs
nothing. The list is read into a variable before the loop, because a
`for` loop drops the exit status of a command substitution, and a patch
that git cannot read would otherwise give an empty list and a green
step.

A few images need more than the plain loop:

- opensearch-operator skips `controllers/`. A patch touches it, but its
suite starts an envtest control plane from etcd and kube-apiserver
binaries the builder does not have. The loop runs with `-race` and
replaces the hand-picked `go test -race` line, which also covered
`pkg/reconcilers/util`, a package no patch touches.
- kube-ovn drops its `go test -run '^TestVMIMigrationSchedulingRetry$'`
line. The test it named is copied into `pkg/controller`, which a patch
touches, so the loop runs it along with the rest of that package.
- redis-operator also runs `go vet -tags integration` in each touched
directory. Two of the files its patches edit are upstream integration
tests behind that tag, and `go test` compiles none of them.
- cluster-autoscaler and kubevirt-cloud-provider export `GOOS` and
`GOARCH` for the target in the whole builder stage, so their tests unset
both. A local `make image` builds amd64 and arm64 by default, and a test
binary for the other architecture runs on the build host only under
binfmt emulation. CI is not affected either way: the jobs that build
package images each pin one `PLATFORM` that matches the runner's
architecture.

The first run of the step on cluster-autoscaler found the carried
`fix-downscale.diff` out of step with the upstream tests it leaves in
place. The patch removes the `maxSize` refusal from `SetSize`, and one
case of `TestNodeGroupIncreaseSizeErrors` still expected that refusal.
The patch now drops that case and carries
`TestNodeGroupDeleteNodesAboveMaxSize` instead. That test covers the
downscale the patch is named for: `DeleteNodes` lowers a group through
`SetSize`, so with the refusal in place a group standing above its max
size could not shed a node. Against the unpatched `SetSize` the new test
fails. The image's behaviour does not change, and scale-up stays bounded
by the core scale-up orchestrator, which skips a node group already at
its max size.

CI builds all twelve images, and the step reports `ok` or `no test
files` for every package it tests. Locally, the redis-operator,
opensearch-operator and kube-ovn builder stages pass. The
cluster-autoscaler package was checked from the
`cluster-autoscaler-1.32.0` tag tarball, because the Dockerfile's full
clone of kubernetes/autoscaler does not survive my network. As negative
controls, an undefined call added to a redis integration test fails `go
vet -tags integration` while `go test` still passes, and the new
autoscaler test fails without the patch.

Left out on purpose, and the Dockerfile says so: `multus`. Its only
touched package, `pkg/multus`, creates network namespaces in its tests,
which a build container cannot do (`mount --make-rshared /var/run/netns:
operation not permitted`). `cilium` (C, no compile step) and
`piraeus-server` (Java) are out of scope.

Closes #4318

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff: Dockerfiles under
`packages/*/images/` and one patch file. No values key, schema, make
target, `hack/` file or documented workflow changes.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```
 2026-10-03 04:56:00 +03:00
Aleksei Sviridkin cc731e3105 test(e2e): keep redis-tls failures legible under the op ceiling 
The CA-projection step caps each kubectl at 30s, but more capped reads
run than its 5m op can cover, so a degraded apiserver could spend the
ceiling and leave nothing but "signal: killed". The reads now share a
step budget held under the op, and a read that hits its bound or finds
the budget spent says which read it was.

Two exec checks reported the 15s redis-cli cap for any failure, sending
a reader after a slow redis when the exec was refused or the container
was missing. They now report the exit status and what the exec
printed. The step also runs under bash with pipefail: under dash a
failed Secret read took base64's zero status and surfaced as a wrong
source name. The private-key check reads the projection already
fetched, since a second read that failed made it pass.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-03 01:42:53 +03:00
Aleksei Sviridkin a057d738d1 test(e2e): give the gateway listener-set op room for its own waits 
The step's polls can wait 660s under a 6m op, and its cleanup trap
deletes two tenants with no bound on the wait. When the op ceiling
fires first, Chainsaw kills the shell: the poll that was running never
reports, the trap never runs, and both tenants stay behind for every
later suite.

The deletes keep waiting, child before parent, since a parent deleted
while its child is still uninstalling wedges its own cleanup. Each
delete runs under an outer timeout with a kill grace, because kubectl's
--timeout bounds only the wait after the request and not discovery or
a stalled request. The parent is deleted only once the child is gone.
A failed tenant delete fails the step with kubectl's error, its exit
status and the tenant left behind, rather than passing with the leak.
The op is sized above the polls plus those bounds, and a unit guard
holds it to that sum and the cleanup to that order.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-03 01:42:53 +03:00
IvanHunters 309875ab62 test(controller): seed the DataVolume informer for controller-runtime 0.25 (#4729) 
## What this PR does

`main` does not build `internal/controller` tests, so `Unit & controller
tests` is red on every open PR. #4380 added DataVolume watch tests that
set `FakeInformer.Synced` and lock `SyncedLock`. #4473 then moved to
controller-runtime v0.25.1, where `FakeInformer` keeps its sync state in
a private channel and exposes only `Synced()` and `HasSynced()`. Each PR
was green on its own base.

The tests now seed the DataVolume informer into
`FakeInformers.InformersByGVK` and call `Synced()` when they want it
synced. Seeding is needed because `FakeInformers` builds every informer
it creates itself already synced, so there is no way to get an unsynced
one from it otherwise. The unsynced cases still assert that no reader is
installed, so they would fail if the informer came up synced.

`go vet ./...` is clean and `go test ./internal/controller/` passes.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Improved automated checks for workload monitoring across different
data synchronization states. These changes strengthen validation of
watch behavior without changing functionality visible to users.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-03 01:36:33 +03:00
Aleksei Sviridkin 2eeaa59055 test(controller): seed the DataVolume informer for controller-runtime 0.25 
controller-runtime 0.25 made FakeInformer's sync state a private
channel, so the DataVolume watch tests no longer compile: they toggled
the exported Synced field and its lock. FakeInformers also builds every
informer it creates already synced, so an unsynced informer has to be
seeded into InformersByGVK and marked synced with Synced() later.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:29:03 +03:00
Aleksei Sviridkin 89a1fe52d9 build(multus): say why the image build runs no tests of the patched package 
The other patched Go images now test the packages their patches touch
in the builder stage. The one package the multus patch touches,
pkg/multus, creates network namespaces in its tests, which a build
container is not allowed to do, so that image keeps building without a
test step. The Dockerfile records the reason, so the absence reads as a
decision and not as an omission.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:02 +03:00
Aleksei Sviridkin f51be95430 build(objectstorage-controller): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Each package is tested from its own directory rather than as a ./...
path of the root module, because internal/ is a module of its own and
the root module cannot name ./internal/runtime.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:02 +03:00
Aleksei Sviridkin b2a5f12e73 build(kubernetes): run the tests of the packages the patches touch 
The cluster-autoscaler and kubevirt-cloud-provider image builds ran go
build only, and go build never compiles a _test.go, so nothing ran the
tests of the code the patches change. Run go test in the builder stage
on the packages the patches touch. The list is read from the patches
themselves, so a new patch is covered without an edit to the Dockerfile.

Both builder stages export GOOS and GOARCH for the target, so the tests
unset them: a local make image builds amd64 and arm64 by default, and a
test binary for the other architecture runs on the build host only
under binfmt emulation.

The first run of that step on cluster-autoscaler found the carried
downscale patch out of step with the upstream tests it leaves in place:
SetSize no longer refuses a size above maxSize, while one case of
TestNodeGroupIncreaseSizeErrors still expected that refusal. The patch
now drops that case, and carries a test for the downscale it is named
for instead. DeleteNodes lowers a group one node at a time through
SetSize, so with the refusal in place a group standing above its max
size could not shed a node at all. The new test deletes a node from
such a group and expects the replica count to drop; against the
unpatched SetSize it fails with the refusal the patch removes. Scale-up
past maxSize stays bounded by the core scale-up orchestrator, which
skips a node group already at MaxSize.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:01 +03:00
Aleksei Sviridkin 070d235d1c build(kube-ovn): run the tests of the packages the patches touch 
The image build ran one test by name, the one shipped next to the
migration patch, and go build never compiles a _test.go, so the rest of
the tests in the packages the patches change never ran.

Run go test in the builder stage on the packages the patches touch. The
list is read from the patches themselves, so a new patch is covered
without an edit to the Dockerfile. It replaces the hand-picked go test
line: the shipped test lives in pkg/controller, which a patch touches,
so the loop runs it with the rest of that package.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:01 +03:00
Aleksei Sviridkin 59c3174b44 build(kamaji): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:00 +03:00
Aleksei Sviridkin 0b00b69267 build(capi-providers-cpprovider): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:00 +03:00
Aleksei Sviridkin 1ffc8580a4 build(keycloak-operator): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:01:00 +03:00
Aleksei Sviridkin e2757ba87b build(redis-operator): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Two of the files the patches edit are upstream integration tests behind
the integration build tag, which go test compiles none of, so each
directory is also vetted with that tag. Doing it per derived directory
rather than for the one directory those tests live in today keeps a
later patch to a tagged test elsewhere covered.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:00:59 +03:00
Aleksei Sviridkin 0a606815ab build(kilo): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:00:59 +03:00
Aleksei Sviridkin a3a7d66bdf build(linstor): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:00:58 +03:00
Aleksei Sviridkin a73b91e232 build(bucket): run the tests of the packages the patches touch 
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:00:58 +03:00
Aleksei Sviridkin 293a780938 build(opensearch-operator): run the tests of the packages the patches touch 
The image build ran go test on two package trees listed by hand, and go
build never compiles a _test.go. A patch that reaches a directory
nobody listed brings tests that are proven once, by hand, when the
patch is cut, and nothing notices them failing after the next version
bump.

Run go test in the builder stage on the packages the patches touch. The
list is read from the patches themselves, so a new patch is covered
without an edit to the Dockerfile. It replaces the hand-picked go test
line, keeping its -race flag: two lines testing the same packages is
one too many. The hand-picked line also covered pkg/reconcilers/util,
which no patch touches.

controllers/ is left out. The stall-tracing patch touches it, and its
suite starts an envtest control plane from etcd and kube-apiserver
binaries the builder does not have, so testing it fails the build.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 22:00:57 +03:00
Aleksei Sviridkin bf03c84711 chore(deps): move to the Kubernetes 1.37 libraries without the apimachinery fork (#4473) 
## What this PR does

This moves the root module's `k8s.io/*` from v0.35 to v0.37.1 and
controller-runtime from v0.23 to v0.25.1, and drops the
`k8s.io/apimachinery` fork on the way. Supersedes #3001, #4467 and
#4472; those were a stack of the same commits.

The four commits are meant to be read one by one, and each builds and
passes the tests on its own:

1. **Plain watches stop getting the initial-events-end bookmark.** The
apiserver defaults `sendInitialEvents=true` onto any watch with an empty
or `0` resourceVersion and leaves bookmarks off, and our aggregated
storages took that flag alone as a WatchList request. From
`k8s.io/apiserver` v0.36 the watch handler compiled into cozystack-api
panics on that bookmark, so `kubectl get -w` on any cozystack-api
resource would die after the bump. The marker now needs both flags, the
same condition the watch cache uses. There are tests per storage and one
through the real handler chain.
2. **The apimachinery fork goes.** Every apps kind shares the
`Application` Go type, and stock `Scheme.ConvertToVersion` picks the
first kind registered for it. The fork patched that; the apps group now
encodes through a serializer that keeps the kind the object carries. A
handler-chain test covers create, get, list, update, watch, apply and
table for two kinds and fails without it.
3. **MutatingAdmissionPolicy stays out of admission.** From v0.36 the
plugin is on by default and refuses every write until it has synced
`mutatingadmissionpolicies`, which cozystack-api has no RBAC to list and
which clusters before 1.36 do not serve. Without this, every create,
update and delete would fail after the bump. It is also re-applied in
`Complete`, since `--disable-admission-plugins` replaces the list.
4. **The bump itself**, with the regenerated clients, openapi and
strategy CRDs.

Commits 2 and 4 are where the review effort goes; 1 and 3 are small.

Known changes in behaviour:

- Without the fork, the row objects of a Table requested with
`includeObject=Object` carry the first registered app kind. kubectl asks
for that only with `--sort-by`, and it sorts on fields, not on the kind.
Nothing at this layer can fix it, because the handler converts through
`APIGroupInfo.Scheme`, a concrete `*runtime.Scheme`. Admission webhooks
and ValidatingAdmissionPolicy still see the right kind.
- The alpha PodSpec field `workloadRef` became `schedulingGroup`
upstream. Once the new strategy CRDs are applied, the apiserver prunes
`workloadRef` from stored strategies it serves, and their next write
erases it. It only ever worked behind an alpha feature gate.

The branch is rebased onto the Renovate rollup (#4447), with `go mod
tidy` and codegen rerun on top of it.

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
chore(deps): cozystack-api and the controllers build against the Kubernetes 1.37 libraries without the apimachinery fork. Plain watches no longer receive the WatchList initial-events-end bookmark, and objects in Table rows requested with includeObject=Object may carry another app's kind.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Application API responses now preserve the requested resource kind
across operations, including lists and watches.
* Backup strategy resources support additional Kubernetes pod settings,
including probe protocols, volume ownership options, and scheduling
groups.
* Backup strategy schemas include updated Kubernetes pod fields and
descriptions.
* **Bug Fixes**
* Initial-event watch bookmarks are sent only when requested; plain
watches continue without an ending bookmark.
* Mutating Admission Policy is disabled by default, including when
admission plugin options are customized.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 21:45:51 +03:00
Aleksei Sviridkin 9809b22280 fix(vm-disk): report DataVolume readiness through WorkloadsReady (#4380) 
## What this PR does

Nothing on a vm-disk application tells whether CDI ever finished
populating the disk. Its WorkloadMonitor has no pods to count, and
`Ready` follows the Helm release. So an import or clone that never
completes can leave a green application over a disk nobody can use.
cozystack/cozystack#2908 is one way to get there: a clone into a
StorageClass that cannot place a replica where the source data sits. The
expensive part there was the silence, not the stuck clone.

The disk state was in the DataVolume phase all along, and nothing read
it. With this change the WorkloadMonitor in cozystack-controller reads
it. The monitor lists the DataVolumes its selector matches and reports
not operational while any of them is in flight, Failed, Unknown or has
no phase. An upload counts as in flight until it completes. That
surfaces on the application as `WorkloadsReady=False` with reason
`DataVolumeNotReady` and a message that names the DataVolume and its
phase, for example `DataVolume vm-disk-x is ImportInProgress`.

`Ready` still follows the HelmRelease, as for any other kind
(`convertHelmReleaseToApplication` in
`pkg/registry/apps/application/rest.go`), and for a vm-disk it no longer
waits on the DataVolume at all. So the disk state has to be visible
outside `Ready`, and this PR puts it where operators look by default:

- `kubectl get` on any application kind gets a `WORKLOADS` column next
to `READY`. It shows the `WorkloadsReady` status, or `<none>` when the
application has no WorkloadMonitor.
- The console badge turns amber and reads `DataVolumeNotReady` while
`Ready` is True and `WorkloadsReady` is not True for that reason. A
failing `Ready` still wins. The console shows only the disk case: other
kinds need a separate decision, because a stopped VM legitimately has no
pod and reports `WorkloadsReady=False` too.

A Helm readiness gate on the DataVolume would be the obvious
alternative, and it is not used on purpose. helm-controller runs that
wait inside its reconcile, so an importing disk would hold one of the
shard's worker slots for the whole install window, and a disk stuck for
good would keep taking the slot back. cozystack/cozystack#4145 has the
details. The monitor costs no slot, keeps watching after install, and
covers existing disks without anyone writing to them.

Without any gate, the install would still wait on the DataVolume.
helm-controller's default poller falls back to a resource's `Ready`
condition for kinds it has no rule for (`checkReadyCondition` in
fluxcd/cli-utils `pkg/kstatus/status/status.go`), so a vm-disk install
could hold a worker slot for the whole install timeout. The
ApplicationDefinition therefore gets a `healthCheckExprs` entry for
DataVolume with `current: "true"`, the same idiom postgres-rd uses for
its ScaledObject. With it Helm never waits on the DataVolume. The entry
is pinned by a helm-unittest in `packages/system/vm-disk-rd`, which gets
a `test` target for it.

Release settings reach a HelmRelease only when the application is
created or updated, so an existing disk gets the entry on its next
update. The reconciler that syncs definitions onto existing releases
copies chartRef, valuesFrom and labels only. That affects any kind with
release settings, so it is filed as cozystack/cozystack#4409 and not
fixed here. `WorkloadsReady` covers every disk right away, since the
monitor needs no write to the application.

CDI copies the DataVolume labels onto the PVC it creates and makes the
DataVolume its controller, so the monitor also publishes a `pvc-<name>`
Workload for the disk. That Workload reads not operational while its
DataVolume is not ready, even though the PVC is already Bound, so
`kubectl get workloads` agrees with `WorkloadsReady`. While the
DataVolumes cannot be read it keeps the verdict it has, like the
monitor.

The WorkloadMonitor CRD now requires at least one selector label. An
empty selector would match every Pod, PVC, Service and DataVolume in the
namespace. No chart renders one.

When an application has several monitors that are not operational and
only some of them name a cause, `WorkloadsReady` carries the named
messages and reason only. A vm-disk has one monitor, so it does not hit
this.

### Which phases count as ready

Settled phases pass: `Succeeded`, `PendingPopulation`,
`WaitForFirstConsumer`, `Paused`. A disk with `bindImmediately: false`
on a `WaitForFirstConsumer` class rests in `PendingPopulation` or
`WaitForFirstConsumer` until a VM consumes it. A multi-stage import
rests in `Paused` until its next checkpoint, and the vm-disk chart
renders no checkpoint source. These report ready while holding no data,
because that is where such a disk is supposed to sit.

`UploadReady` is not one of them. CDI enters it when the upload server
pod is running and ready, and leaves it for `Succeeded` only when that
pod succeeds (`updateStatusPhase` in
`pkg/controller/datavolume/upload-controller.go` at v1.66.1), so the one
phase covers a disk nobody has uploaded to and a half-uploaded one. A
`source: upload` disk therefore reports `WorkloadsReady=False` from
creation until `virtctl image-upload` completes.

A phase the controller does not know also reads as ready, so a CDI
release that adds or renames a phase cannot turn working disks red. The
cost goes the other way: a renamed in-flight phase silently reads as
settled. The list wants a re-read whenever the CDI version moves. It
matches the named phases of the CDI v1.66.1 API, the version vendored in
`packages/system/kubevirt-cdi-operator`.

### Clusters without CDI

CDI comes with the IaaS bundle, while cozystack-controller runs
everywhere, and CDI can be installed while the controller is running. A
watch on a kind whose CRD is absent never syncs, and then the whole
controller does not start. So the watch is not set up at startup. A
runnable adds the DataVolume watch once the kind is served. It waits for
the first reconcile, because only from then on does a new watch start at
once, and then checks discovery right away and every 30s after that. It
installs the DataVolume reader only after the DataVolume informer has
synced within a timeout, because a reconcile context has no deadline and
a List on an informer that never syncs would block every WorkloadMonitor
reconcile. The watch source is started once: a controller-runtime `Kind`
source adds an event handler to the informer and never removes it, so on
a timeout the next check waits on the source it already has instead of
starting another one. RBAC needs no change: the controller's ClusterRole
already has read on everything.

When the DataVolumes cannot be read, because the reader is not installed
yet or the List fails, the monitor still publishes its replica counts
and keeps its last DataVolume verdict, and a List error is returned so
the request is retried and recorded as a `DataVolumesUnavailable`
Warning event on the monitor. The status of a monitor without
DataVolumes is not affected by a failing List. A disk whose verdict is
already recorded does not read as ready in the window after a controller
restart. A monitor with no recorded verdict yet, which is every monitor
right after the upgrade that brings this change and any disk created
right after a restart, reads ready until the DataVolume watch has
synced. Because the watch starts on the first reconcile rather than on
the next discovery tick, that window is the time to sync the DataVolume
informer plus at most one 5s requeue, not a 30s poll interval. The
window is not zero, since a monitor reconciled in it is published before
its DataVolumes can be read. From then on, a reconcile that runs before
the watch has synced is requeued after a few seconds, so the DataVolume
events that queue it are not lost. A pass that changes nothing in the
monitor status does not write it, so that requeue sends no status writes
to the API server. Once the reader is installed, every monitor that
holds a DataVolume verdict is queued once more. So a verdict whose
DataVolume was deleted while the controller was down is recomputed then,
not at the periodic resync.

### Tests

The table test in `workloadmonitor_controller_test.go` covers the named
phases of CDI v1.66.1, plus Failed, Unknown, an empty phase and a
missing status. A second table checks the message and reason for an
importing, a failed, a phaseless and a populated disk, and a third test
lists two stuck DataVolumes in both orders and expects one message. It
also covers a DataVolume outside the selector, the kind not being
served, a List error that still publishes the replica counts, and a
missing reader that keeps the last verdict. The watch tests check that
the watch starts on the first reconcile and not before it. A source that
never syncs leaves no reader and is retried on the same source, with the
reconciles in between still requeued. A List failure records the Warning
event, the sync window records none, and an unchanged pass writes no
status. A manager that stops during the wait gets the cancellation back,
and a reconcile that runs while the watch syncs is requeued without an
error. A table test covers the PVC Workload: importing, populated, a PVC
with no DataVolume controller or a controller of another kind or group,
and the unread cases. A helm-unittest in
`packages/system/cozystack-controller` pins the selector's
`minProperties`. `rest_table_test.go` pins the `WORKLOADS` column,
`rest_workloads_test.go` the condition message and reason, and
`status.test.ts` the console badge, including a halted VM that stays
green.

In `hack/e2e-chainsaw/vminstance` the existing VMDisk scenarios now wait
for the DataVolume to be Ready, then for `Ready` and `WorkloadsReady` on
the VMDisk itself. That replaces the wait on the HelmRelease. Those
asserts would pass even if the monitor ignored DataVolumes, so a new
scenario imports from an `.invalid` URL. It first waits for the importer
to have failed, which CDI reports on the DataVolume as `Running=False`
with reason `Error`, and then for `Ready=True` with
`WorkloadsReady=False` and reason `DataVolumeNotReady` on the VMDisk.
Without the first wait the VMDisk assert passed in about a second, on a
DataVolume that had no phase yet. The e2e runs on a fresh install only,
so nothing here exercises an existing release across an upgrade.

### What it does not do

#2908 lists possible fix directions, and none of them is here. The
cross-class clone still never completes, it just stops being invisible.
Keep the issue open.

While the DataVolume watch syncs, every monitor is still requeued. Only
the ones whose selector reaches a DataVolume need it, but the controller
cannot tell which those are before it can read DataVolumes.

Closes #4449

### Screenshots

There is no screenshot: I had no running console to take one from. The
badge change is pinned by the vitest cases in
`packages/system/dashboard/images/console/apps/console/src/lib/status.test.ts`:
a VMDisk whose `WorkloadsReady` has reason `DataVolumeNotReady` goes
amber, and a halted VMInstance with the generic reason stays green.

### Downstream repositories

I walked the trigger map against the diff.
cozystack/terraform-provider-cozystack is affected: its `wait_for_ready`
reads only `Ready`, so on a `cozystack_vmdisk` it returns before the
disk is populated once this lands. That needs a decision on the provider
side, so it is an issue there. The release-time bot regenerates
`packages/apps/vm-disk/README.md` into the website's virtualization
pages, and `vm-disk` is in the `VMS` list that generator reads, so the
website needs no follow-up. The ApplicationDefinition keeps its kind,
plural, prefix and path.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
https://github.com/cozystack/terraform-provider-cozystack/issues/59
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(vm-disk): report whether CDI finished populating a disk in the WorkloadsReady condition, which names the stuck DataVolume and its phase. kubectl get shows WorkloadsReady in a new WORKLOADS column for every application kind, and the console badge turns amber while a disk is not populated, so a disk whose import or clone never completes no longer looks healthy. Ready of a new or updated vm-disk no longer waits for the DataVolume, so wait on WorkloadsReady to know the disk is populated.
```
 2026-10-02 19:47:02 +03:00
IvanHunters dad5924b21 fix(computeplane): do not grant tenants the cluster admin kubeconfig (#4722) 
## What this PR does

A tenant user could read the admin kubeconfig of their own ComputePlane
cluster, which the module promises never to hand out. The module deploys
the `kubernetes` app into the tenant namespace as
`computeplane-cluster`. The secrets allowlist kept the Secret out of
`tenantsecrets`, but the chart's dashboard Role still gave the tenant's
`use` subjects get/list/watch on
`computeplane-cluster-admin-kubeconfig`. So a plain `kubectl get secret`
worked.

The fix drops that one rule for the ComputePlane cluster only. The
kubernetes chart now skips it when the internal value
`withholdAdminKubeconfig` is set, and the computeplane module sets that
value inline in the cluster HelmRelease. The tenant can't reach those
values, because the module reads only `nodeGroups` and passes an
allowlisted subset of it to the pool releases. The value is kept out of
`values.yaml` and the schema, same as `clusterReleaseName` in
kubernetes-nodes.

A regular Kubernetes app keeps the rule, so the dashboard still offers
its kubeconfig. Only the v1.7.0 alphas ship the module. After this
lands, the `kubectl auth can-i` check can go back into the e2e suite in
#4713.

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(computeplane): tenant users no longer get read access to the ComputePlane cluster's admin kubeconfig Secret through the kubernetes app's dashboard Role.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Security**
* Computeplane dashboards no longer grant access to the cluster’s admin
kubeconfig Secret. Other dashboard resource permissions remain
available. This limits access to sensitive cluster credentials through
the dashboard while preserving the service-related permissions needed
for dashboard resources.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 18:11:49 +03:00
Aleksei Sviridkin ded2354b65 fix(tenant): skip Cilium policies on isp-hosted without Cilium (#4710) 
## What this PR does

The tenant chart no longer fails to install on a hosted cluster without
Cilium. The isp-hosted variants install no CNI of their own, but the
chart rendered its Cilium policies unconditionally, so `tenant-root`
failed with `no matches for kind "CiliumNetworkPolicy" in version
"cilium.io/v2"`. Everything that depends on `cozystack-basics` then
never converged. This also unblocks the isp-hosted e2e lane, which fails
on exactly this.

The guard is keyed on the platform variant. The platform already passes
it to every tenant as `_cluster.bundle-name`. It covers every Cilium
object the chart renders, including the shared-central tracing egress
policies.

- On variants where Cozystack installs Cilium itself, nothing changes
and a missing CRD still fails the install.
- On `isp-hosted` and `isp-hosted-slim`, the policies render only when
the host serves both Cilium policy kinds, since a hosted cluster may run
Cilium.

The render on Cilium variants is byte-identical to `main`. I checked it
with `helm template` over all four Cilium variants and an absent
`_cluster`, with optional services on and off, with and without the
Cilium API, for root, child and grandchild tenants.

I didn't use the capability check alone because it fails open.
helm-controller upgrades a deployed release only when the chart or the
values digest changes, so a change in API discovery never triggers a
re-render. That is `VerifyRelease` in `internal/action/verify.go` at
v1.5.0, the version Cozystack ships. If the Cilium CRDs were not
discoverable at install time on a Cilium cluster, a capability-only
guard would leave the tenant without policies until the next chart or
values change. An upgrade rendered in that window would delete existing
ones.

On isp-hosted without Cilium, tenants get no network isolation from
Cozystack. Isolating them is up to the host. I didn't fall back to
standard `NetworkPolicy`, because the host CNI may not enforce it. The
tenant policies also use Cilium features that don't map one to one: the
`world` and `kube-apiserver` entities, clusterwide policies, selectors
on ancestor namespace labels. A partial translation would look like
isolation without really being one. The tenant README now documents
this, including that Cilium added to the host later takes effect only on
the tenant's next chart or values change. The threat model and the
self-assessment list it under non-goals.

The new helm-unittest suite fails on `main` for the hosted cases without
Cilium. Each guard mutation I tried turns at least one case red: no
variant check, no capability check, `and` changed to `or`,
`isp-hosted-slim` dropped from the list.

I didn't add an explicit "require Cilium" value, since the Cilium
variants already behave that way. One thing to keep in mind: the list of
hosted variants now lives in the platform bundle and in the tenant
helper, so a new hosted variant has to go into both. A platform
`_cluster` flag saying whether Cozystack manages the CNI would remove
that, but it's a platform change.

Closes #3387

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The tenant README change reaches the website through the release bot
that regenerates the managed-apps pages from each package README.

### Release note

```release-note
fix(tenant): the tenant chart no longer fails to install on the isp-hosted variants when the host has no Cilium. There, the Cilium network policies are created only if the host serves the Cilium policy API; otherwise tenants get no network isolation from Cozystack. Other variants are unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Tenant network-isolation policies are rendered for `isp-hosted` and
`isp-hosted-slim` only when the required Cilium APIs are available. If
Cilium becomes available later, policies are applied after a subsequent
chart or values change. Tracing policies also require the Cilium APIs.
* **Documentation**
* Clarified when Cozystack provides tenant network isolation and that
isolation is the host’s responsibility when the required Cilium APIs are
unavailable.
* **Tests**
* Added coverage for policy rendering across bundle types and Cilium API
availability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 16:02:55 +03:00
IvanHunters 11910e0b2c fix(kubernetes-nodes): refuse a resources.cpu that cannot be a core count (#4386) 
## What this PR does

A worker pool's CPU value could land in the VM as something KubeVirt
cannot accept. `resources.cpu` is typed as a quantity, but on the
kubevirt substrate the template wrote it unchanged into
`domain.cpu.cores`, which is `uint32` in KubeVirt and an integer in the
KubevirtMachineTemplate CRD. With `resources.cpu: 500m` the chart
rendered `cores: 500m`. That string was truthy, so it also counted as
explicit sizing and dropped the pool's `instanceType` from the VM, and
then the API server refused the template anyway.

The render now decodes the value as YAML, the same way `cores: <raw>` is
decoded, and stops unless the result is a whole number from 1 to
4294967295. `500m`, `"1.5"` and `"1e10"` now fail at render time with an
error naming the field, before the value reaches the VM template or the
instancetype decision. `"0"` and `"-2"` were already refused on `main`
and still are.

Only inputs that could not produce a working pool are refused. Every
value that rendered a pool before renders the same bytes now, so the
content-hashed template keeps its name and nothing rolls. That includes
`2`, `"2.0"`, `"+2"` and `"1e1"`, and also two odd cases I kept on
purpose because pools may already carry them. An unquoted `0` is falsy,
so the pool renders without `domain.cpu` and is sized by its
instancetype, as on `main`. A leading-zero value such as `"010"` is
still emitted as written and keeps the reading it gets on `main` today,
where YAML treats it as octal. I checked both against a render of
`main`, and the tests pin the template names that `main` produces for
them.

This is where kubernetes-nodes and vm-instance now differ. Since #4405,
vm-instance refuses a leading zero and an unquoted `0`, and this chart
does not. Stored worker pools can carry these values, because migration
56 copies `resources` verbatim, so refusing them here without a
migration would fail those pools' releases on their next upgrade. #4554
tracks the migration and the refusal after it.

The proxmox substrate is not checked. It converts the quantity to
millicores and rounds up to whole cores, so `500m` is a valid size there
and becomes one core. The field description now says what the value
means on each substrate.

Memory is left alone. It goes to `domain.memory.guest`, which is a
`resource.Quantity`, so a quantity string is valid there.

The issue suggested coercing to an integer, as vm-instance did before
#4405. I didn't, because `int64` turns `500m` into `0` without an error.

The new helm-unittest suite covers each input form, both preserved
cases, the upper bound, the instancetype case and the proxmox path.

Two neighbouring descriptions still say only "CPU available.":
`nodeGroups[].resources.cpu` in `packages/extra/computeplane` and in the
parent `kubernetes` chart. computeplane passes `resources` straight into
this chart, so the new check applies to its pools as well. I left both
alone to keep this change to one chart.

Closes #4150

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(kubernetes-nodes): on the kubevirt substrate, a worker pool whose resources.cpu cannot be a whole vCPU core count (for example 500m, 1.5 or a value above 4294967295) now fails at render time with a clear error, instead of rendering a VM template that is refused later or dropping the pool's instanceType. Values that rendered a working pool before render unchanged.
```



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* KubeVirt CPU settings now accept whole-core values from 1 to
4,294,967,295. Fractional, millicore, negative, and larger values are
rejected; an unquoted `0` is treated as unset.
* Proxmox rounds fractional and millicore CPU values up to whole cores.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 15:57:11 +03:00
IvanHunters fb7e780517 fix(postgres): reassign dropped roles' objects to an unprivileged role (#4574) 
## What this PR does

Objects owned by a role the Postgres init-job drops no longer end up
owned by the `postgres` superuser. This covers a user removed from
`users` and an orphaned managed `<db>_admin` or `<db>_readonly` role.
Before dropping such a role the job reassigned everything it owned to
`postgres`, and objects a tenant role created should not become
superuser-owned.

They now go to `cozystack:orphaned`, a NOLOGIN role that is granted
nothing. The init-job creates it and resets its attributes on every run.
The `:` in the name is outside the charset allowed for user and database
names, so values cannot declare it.

There is a trade-off. Grants and memberships of the dropped role do not
carry over, so a view or function it owned that ran with its owner's
privileges can lose access it had through them. To keep such objects
working, keep the user in values or move the objects to another owner
before removing it. `docs/operations/backup-classes.md` is updated
accordingly.

Role names read from the catalog now reach SQL as psql variables instead
of being spliced in by the shell. The queries are echoed after
substitution, so the job log shows which role is dropped.

Tested with helm-unittest and by running the rendered init script
against PostgreSQL 13 and 18.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(postgres): objects owned by a user removed from `users`, or by an orphaned managed database role, are now reassigned to the unprivileged `cozystack:orphaned` role instead of the `postgres` superuser. A view or function of the removed role that ran with its owner's privileges can lose access it had through that role's grants or memberships.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Clarified that after restore, the first initialization releases
undeclared managed databases without deleting their data and removes
users not declared in the chart.
* Removed users’ objects are reassigned to a restricted, non-login role.
Grants and memberships are not preserved; functions use invoker rights,
and PostgreSQL 15+ views are set to use invoker security.
* Added guidance for finding eligible objects still owned by the
PostgreSQL superuser for manual reassignment.
* Declare a user in `users` to keep it; the chart will generate a new
password.
* **Bug Fixes**
* Removed users’ objects are reassigned across connectable, non-template
databases before their users are dropped.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 15:56:49 +03:00
IvanHunters 339be8d42d fix(vm-default-images,kubernetes-worker-image,vm-disk): wait for golden images to import and refuse cloning a failed one (#4392) 
## What this PR does

Both golden image catalogs, `vm-default-images` and
`kubernetes-worker-image`, went Ready before any image had imported.
Each one renders a DataVolume per image with immediate bind, so every
import starts at install, but neither PackageSource had
`healthCheckExprs`. The HelmRelease went Ready as soon as helm applied
the DataVolumes, and an image that failed or stalled stayed green.

This PR makes both releases wait for the imports. Each component's
`install:` block now has a DataVolume health check. `current` holds only
on `Succeeded`, and `failed` holds on `Failed`. Any other phase,
`Unknown` included, matches neither, and flux keeps waiting. A
DataVolume with no `status` key makes the CEL evaluation error out. Flux
reports that as Unknown and keeps waiting too.

The other settled phases are safe to treat as not ready for these
charts. `PendingPopulation` and `WaitForFirstConsumer` are where a
volume waits for a consumer, and immediate bind removes that wait.
`UploadReady` and `Paused` belong to upload and multi-stage sources, and
both charts render only `source.http`.

There is no per-component timeout, so I did not set one. The window is
the operator-wide `--helmrelease-install-timeout`, 10m by default. The
operator sets `RetryOnFailure` on both install and upgrade, so a
download that outlasts the window is retried as an upgrade until the
import finishes. Nothing uninstalls the DataVolumes in between, and the
import keeps running.

One image that never imports keeps its whole catalog not Ready until the
image imports or its entry is removed. There is no time limit, and I
kept it that way on purpose. A group/kind health check cannot tell one
image from another, so it cannot be made per image. A per-component
timeout would be an API change for one opt-in package. Both charts'
`values.yaml` document this. Nothing on `main` lists either package in
`dependsOn`, and the images that did import stay usable in the meantime.
Tenant worker pools look up each golden directly and never wait on the
catalog's status.

The price is a helm-controller slot. The operator labels the generated
HelmRelease with `cozystack.io/package`, plus `cozystack.io/privileged`
when privileged. The flux-shard-operator acts only on HelmReleases that
already carry `sharding.fluxcd.io/key`. So both catalogs go to the
system helm-controller, which picks up releases without that key and
runs with `--concurrent=5`. A DataVolume in `Failed` ends each attempt
at once and costs next to nothing. An import that stalls without
reaching `Failed` is the expensive case, for example an importer that
keeps retrying an unreachable URL. Each attempt waits out the full
window, and the next one starts 30s later. So one of those five slots
stays busy about 95% of the time for as long as the image is stuck.
#4145 describes the same cost on the shard controllers.

The status change alone does not protect a vm-disk, because vm-disk
clones from a golden without checking it. A golden still importing is
fine, since CDI holds the clone until the source is populated. A golden
whose import has Failed is never populated, so the clone would wait
forever. vm-disk now refuses to render a new disk on a Failed golden and
names the image. This matches the check the worker pools already have. A
disk that already exists is exempt, so a golden that fails later does
not break that disk's upgrades. The check fires only when CDI sets
Failed, and at CDI v1.66.1 an import reaches Failed only on a checksum
mismatch. The goldens carry no checksum, so a broken URL keeps the
importer retrying and the new disk still waits, with only the catalog's
not-Ready status pointing at the image. #4564 tracks that gap. A golden
that does not exist at all is a separate gap, tracked in #4548.

Tests pin the expressions for both catalogs. A helm-unittest case checks
them on the rendered PackageSource. A Go test builds the HelmRelease
spec from the shipped file and checks that they arrive with the poller
wait strategy. The repo has no harness that evaluates flux CEL, so the
phase behaviour above is argued from flux's evaluator and not exercised
by a test. The vm-disk check has helm-unittest cases for a Failed, a
Succeeded, and a still-importing golden, for a render with no cluster,
and for an existing disk.

Closes #4169

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(vm-default-images, kubernetes-worker-image): the golden image catalogs now report Ready only after their images have finished importing, and a failed or stalled import keeps the catalog not Ready. A vm-disk created from a golden image whose DataVolume is in phase Failed is now refused with an error naming the image, instead of waiting forever on a clone that cannot complete.
```
 2026-10-02 15:44:28 +03:00
Aleksei Sviridkin 0a3546e126 test(e2e): tell a missing object from a failed read 
The assertion that the no-delete guardrail can be bypassed decides on
`kubectl get`'s exit status alone. That status is 1 for a NotFound and
1 for a connection, auth or discovery failure, so an api-server the
test cannot reach reads as proof the configmap is gone.

`--ignore-not-found` collapses the NotFound alone into success with
empty output, which leaves the status free to mean a failed read.
Read the two separately: a non-zero status fails the test naming the
read that did not answer, non-empty output fails it naming the object
that must not be there.

Capture stdout only. kubectl writes deprecation and throttling notices
to stderr on a call that succeeded, and folding those into the value
under test would report an object that is not there.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:09:20 +03:00
Aleksei Sviridkin cd448fdc04 test(bats): widen the comments describing the errexit exemption 
Six suites explain why they avoid `! cmd` by saying errexit is
suppressed for a "negated pipeline". The exemption is not about
pipelines: a bare `! true`, a negated `[ ... ]`, one inside a `for`
body or a brace group, one in a case branch, and one following `;`,
`&`, `&&` or `||` are all exempt too. Anyone reading the narrower
wording could conclude the other shapes are safe.

One of the six carries the sentence twice, and the second copy is
split across two physical lines -- `` `!` `` ending one and
`pipeline)` opening the next -- so a line-oriented search for the
phrase does not see it. Prose wraps; a grep for it has to fold the
lines first.

Comment text only; every one of these files already uses the working
form the comment recommends.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:09:20 +03:00
Aleksei Sviridkin 5137d98e71 test(bats): refuse a negated assertion across every suite 
A negated assertion is the natural way to spell "this must not
appear", and an absence is what nothing else in a test covers, so the
errexit exemption lands where it is least likely to be noticed and
invisible in a green run. Switching runners does not lift it: the ERR
trap the bats binary reports through carries the same exemption, so
neither runner sees a negated assertion that has any statement below
it.

Add a suite that scans every hack/**/*.bats, subdirectories included,
and fails on a statement whose first token is `!` -- naming the file,
the line the statement opens on, and the form to use instead.

Statement boundaries are the POSIX separators, each measured against
the runner before being admitted: `;` `&` `&&` `||` `{` `(`, the `)`
closing a case pattern, and the words `then` `do` `else`. A keyword may
legally be preceded only by whitespace or by `;` -- every other
separator in front of one is a shell syntax error, checked with `sh -n`
rather than recalled -- so the scan admits exactly those two and that
part of the class is closed rather than patched a spelling at a time.
Both spellings are vacuous under the runner, so missing either is a
silent miss, the one direction this scan must not err in. For the same
reason the scan takes a `!` with no space before its subshell,
`!(cmd)`, which is vacuous on dash and bash alike, and reads through a
`time` or `time -p` prefix, which is vacuous under bats and under
cozytest where /bin/sh is bash. A scan that wants `!` and then
whitespace as the first token misses both. On a dash /bin/sh `time` is
not a keyword and `time ! cmd` fails, so there it is an over-count.
Five over-counts remain, all erring loud, each named in the header:
they are enumerated rather than removed, because a removed error
leaves no trace while a named one tells the next reader what they are
looking at.

Inverted conditions are untouched because errexit is not what decides
a condition. `[ ! -f x ]` is untouched for a different reason and
stays a normal failing assertion: the `!` there is an operator of
`test`, not negation of a command, so nothing exempts its status.

Each claim the suite makes is pinned against removing the thing it
claims, and the separator class is measured branch by branch rather than
only at its newest entries. The recursion is exercised through the audit
function on a fixture in a subdirectory rather than through a second
copy of its find expression, because a copy passes while the function
loses its recursion -- and an audit that returns empty reads as a clean
tree. The condition-strip carries fixtures that put the negation after
an `&&` inside the condition, where only the strip can save it. The
continuation fold is pinned by a fixture whose negation sits on the
continued line, so folded and unfolded name different lines. Both
keyword spellings, the `time` prefix and `!(cmd)` have fixtures of
their own. The subshell over-count carries a fixture for both of its
halves -- that the scan reports the shape, and that the shape really
does fail.

Both halves of the premise are pinned, not just the half about the
banned form. A test asserts the runner still passes a bare negated
assertion; a second asserts the prescribed `if cmd; then ...; false; fi`
does fail, under both runners, and fails AT the form rather than
somewhere before it. Every converted absence rests on that second
half, and its failure mode is asymmetric: were `false` in a `then`
body to stop propagating, the first test would still pass while every
conversion went silently green.

The rule joins the E2E conventions doc as its own section, and the
inline list in AGENTS.md that an agent may read instead of the doc --
it belongs beside the EXIT-trap ban for the same reason, both leaving
a green suite that checks nothing.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:09:20 +03:00
Aleksei Sviridkin e5a1ca853f fix(ci): run every Keycloak client suite on a Keycloak change 
With a monitoring suite in the tree, a change to system/keycloak or
system/keycloak-operator stopped escalating to the full run and
selected monitoring alone, because monitoring-application is the only
source that declares the keycloak-operator edge. That dropped
kubernetes-latest, whose OIDC lifecycle also creates a KeycloakClient
and waits on the operator's finalizer to remove it.

The mapping is explicit because kubernetes-application has no dependsOn
on keycloak-operator. A postgres-operator change now reaches
kubernetes-latest too, through Keycloak's database.

The selector comment and the e2e doc no longer name
system/postgres-operator as a path with a source that reaches no
suite, which stopped being true when monitoring got its suite.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:03:15 +03:00
Aleksei Sviridkin 472f962eff test(e2e): cover monitoring OIDC modes with a Chainsaw suite 
Monitoring OIDC lost its live-cluster coverage when the two pre-Chainsaw
BATS suites were removed without a port. Render tests pin each mode on a
fresh install, but not admission by cozystack-api, the Deployment the
Grafana operator builds from the CR, or what an in-place mode change leaves
behind on a running release.

The suite drives one tenant-test Monitoring release through System,
CustomConfig with an inline config, and CustomConfig with a secretRef,
so a single stack bring-up covers all three and the upgrades between
them: the cozy-realm client, scope and client Secret are pruned when
System mode is left, and the inline generic_oauth block is gone once the
secretRef mount arrives.

The monitoring package sources now map to this suite, so a change to
system/monitoring, extra/monitoring, monitoring-rd or grafana-operator
selects it alone instead of escalating to the full run, and the selector
test that relied on cozystack.monitoring reaching no suite seeds a
source that does not.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:03:00 +03:00
Aleksei Sviridkin 40dac09aaa test(bats): spell absences so they can fail 
An assertion written `! cmd` cannot fail a test. POSIX exempts a
command whose return value is inverted with `!` from errexit, so the
status is dropped where it is produced. hack/cozytest.sh then closes
each test function with `return 0`, and the bats binary's ERR trap
carries the same exemption -- it reports only a negated assertion
standing last in the body, where the inverted status happens to be
what the function returns.

Every absence these suites assert was written that way: a command that
must not reach the log, an image that must not be copied, a flag that
must not be passed, an object that must not exist. On that half of
each suite's contract the tests held for any input.

Rewrite each as `if cmd; then echo "FAIL: ..."; false; fi`, the form
already used where the trap had been noticed one file at a time, and
give every one a message naming what must not have happened. The
message states the property, not the pattern: where the forbidden
string is a substring of a required value -- an https endpoint the
test requires in its http form -- naming the string would describe the
opposite of what is asserted.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:01:49 +03:00
Aleksei Sviridkin a72236687c test(seaweedfs): read the naming guard's logic, not the prose about it 
The parity suite forbids four mutable discriminators -- claim
timestamps, ready-replica counts and the two birth-order flags -- by
grepping each chart's whole file. Both charts carry a template comment
block arguing why those signals were rejected, so the grep matched the
argument and would have reported a chart for documenting the rule it
follows. It never did report: the checks were spelled `! grep`, a form
POSIX exempts from errexit, so they could not fail whatever they found.

Strip the {{/* ... */}} spans before matching and let the loop fail
through `false`, naming the signal it matched.

The span and not the line. Dropping every line that opens a comment is
shorter and silently loses a discriminator that shares its line with
one -- `{{- $x := .creationTimestamp }} {{/* why */}}` leaves with the
comment and the checks find nothing. The two spellings agree on
today's charts, so a fixture pins the difference rather than trusting
it.

Have the strip say when it ended inside a comment, and ask before
reading what it returned. Every `{{/*` is read as an opener, including
one inside a quoted string -- these charts carry long `fail (printf
...)` messages -- and a lone opener swallows the rest of the file,
leaving the checks reading nothing. A canary token cannot cover that:
it only proves whatever precedes IT survived, and the block most
likely to swallow is the last one in the file.

It says so on stdout rather than through an exit status, because
hack/cozytest.sh rewrites every `}` at column 0 into `return 0; }` --
the strip's own closing brace included -- so a status it returned
would never reach the caller.

Counting delimiters cannot stand in for the state the strip already
carries. Go template comments do not nest, so the first `*/}}` closes
the block and `{{/* the opener is spelled {{/* */}}` is a comment Helm
renders, while its two openers against one closer read as an imbalance
that is not one.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 15:01:49 +03:00
Aleksei Sviridkin 64f43f4f77 fix(computeplane): do not grant tenants the cluster admin kubeconfig 
The computeplane module deploys the kubernetes chart into the tenant
namespace, and that chart's dashboard Role lets the tenant's "use"
subjects read <release>-admin-kubeconfig. Withholding the Secret from
the tenant secrets view does not help when RBAC grants it directly, so
any tenant user at "use" level or above could fetch the cluster-admin
credential of a cluster the module promises to keep operator-owned.

The kubernetes chart now drops that rule when the internal value
withholdAdminKubeconfig is set, and the module sets it in the cluster
HelmRelease's inline values. The tenant cannot reach those values: the
module reads only nodeGroups from them and splices an allowlisted
subset into the pool releases, and Flux applies spec.values over
valuesFrom. A regular kubernetes app keeps the rule, so its dashboard
can still offer the kubeconfig.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:36:57 +03:00
Aleksei Sviridkin 9c9a8c225d test(seaweedfs): lower the volume size limit on a live instance and test the chart hand-off (#4707) 
## What this PR does

This adds an e2e case that lowers the SeaweedFS volume size limit on a
live instance and checks that the bucket still takes writes. It also
adds a unit test for the hand-off between the SeaweedFS app chart and
the data-plane chart.

The new `seaweedfs` suite creates its own instance in `tenant-test`,
directly through the API, so the limit can change without racing the
tenant module. It installs at 200 MB and writes into one bucket until
every volume of the bucket sits between 20 and 200 MB. Then it patches
the resource to 20 and waits for both releases and the master rollout.
After that a write to the same bucket has to land, and the bucket has to
have a new volume. If the master never picks up the new limit, the write
goes to an old volume and the second check fails. 200 and 20 stand in
for the shipped 1000 and 400, because filling a bucket into that band at
real sizes would cost gigabytes in the sandbox.

The suite also checks the live master StatefulSet. The limit from the
resource has to be in the master command before and after the change,
and all four `WEED_MASTER_VOLUME_GROWTH_COPY_*` counts have to stay. The
same hand-off is checked on every PR by
`hack/seaweedfs-chart-seam.bats`. It renders the app chart, takes
`spec.values` from the `seaweedfs-system` HelmRelease and renders the
data-plane chart with them. The expected default comes from the app
chart's values, so this test does not pin 400 a second time. I checked
in a scratch copy that it goes red when the app chart's
`volumeSizeLimitMB` key or its `master:` block is renamed.

A new suite directory broke two cases in `hack/select-e2e_test.bats`, so
`seaweedfs-application` in `hack/select-e2e.sh` now maps to the new
suite too. That is one line plus its comment.

About the suite order question in #4302: this suite runs on its own
instance, so it takes nothing from tenant-root's capacity and does not
move the margin of `bucket-1-capacity`.

Left for later:

- A generic helper for the two-chart shape (render a chart, take
`spec.values` from a HelmRelease, render the target). This PR covers
only the SeaweedFS pair.
- The `status.history` remediation guard on the upgrade the suite
triggers. A failed upgrade that is retried and then succeeds would pass
here without notice.

The suite has not run on a cluster yet. Locally `chainsaw lint` passes.
I also ran the StatefulSet and HelmRelease asserts with `chainsaw
assert` against rendered objects. They pass on the expected state and
fail on a wrong limit, a missing `COPY_2`, a rollout in progress and a
stale generation, both on the StatefulSet and on the HelmRelease.

Closes #4302

Closes #4285

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
test(e2e): add a seaweedfs suite that lowers the volume size limit on a live instance and checks that the bucket still takes writes, and a unit test for the hand-off between the SeaweedFS app and data-plane charts.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Added end-to-end coverage for SeaweedFS bucket resizing, including
verifying that writes succeed after the volume-size limit changes.
- Added SeaweedFS to the available and automatically selected end-to-end
test suites.

- **Documentation**
- Updated local test prerequisites to note the required tools for
SeaweedFS tests.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 14:08:12 +03:00
Aleksei Sviridkin 10a6cf2a0b test(tests): make absence-only helm-unittest cases fail when they check nothing (#4664) 
## What this PR does

Twenty-one helm-unittest cases asserted only that something is absent
from a document, and could pass without ever looking at the document
they name. This PR makes each of them fail when that happens.

1. Thirteen cases picked their document by position: mongodb (4), nats
certmanager (3), opensearch (3), qdrant certmanager (2), seaweedfs
cluster_scoped_names (1). When a template gains or reorders a document,
the index points at some other object that also lacks the field, and the
case stays green. These now use `documentSelector` on `kind` or
`metadata.name`, like the earlier harbor conversion.
2. Seven cases had no selector at all, on templates wrapped in a
top-level `if`: kubernetes cilium_gateway_alpn, mongodb restore,
linstor-scheduler admission, seaweedfs s3_traffic_distribution and three
in vertical-pod-autoscaler. If a values change turns the condition off,
the render is empty and the absence check passes on nothing. A
`documentSelector` turns an empty render into `document not found`.
3. The redis null-namespace case checked that the in-cluster name is
present, but never that the external name is absent. It now asserts that
neither `my-redis.` nor `my-redis.example.com` (the suite-level host
that `_namespace: null` must override) is in the SAN list.

The `kubernetes cluster` entry in the #3634 catalog no longer applies,
since `packages/apps/kubernetes/tests/cluster_test.yaml` has been
removed from main. I did not add the `my-redis.redis.` check that #4621
mentions, because the per-engine SAN it refers to has not landed.

Each case was checked by mutation. I broke the template in a way the old
case could not see, then ran the chart's `make test` with the new test
file and with the one from main. For group 1 I removed the guard and put
an extra document in front of the target. For group 2 I turned off the
top-level condition. For group 3 I let the empty-host branch emit the
bare SAN. Every time the main version stayed green and the new one went
red, and the fixed cases were the only extra failures. All templates are
back to their committed bytes.

Follow-ups, not in this PR:

- The same absence-only shape has appeared in more cases since #3634 was
filed: mongodb `tls_test.yaml` (8 cases), mariadb `tls_test.yaml` (6),
redis `tls_test.yaml` (2 more), kubernetes
`kamaji_managed_args_test.yaml` (2), and one each in kafka
`dashboard-resourcemap_test.yaml`, cert-manager-issuers
`solver_test.yaml` and seaweedfs `volume_resize_hook_test.yaml`. They
need a separate issue. The four platform `bundles_proxy_protocol*` cases
are #3864, handled in #3929.

Closes #3634

Closes #4621

### Screenshots

Not applicable, test-only change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Updated chart validation to identify rendered resources by their type
or name rather than relying on document order, improving consistency
across Kubernetes applications.
* Expanded Redis TLS checks to verify that certificates omit generic and
external hostnames when no namespace data is provided.
* Kept existing expected behavior checks for gateway, database,
messaging, search, storage, and autoscaling resources.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-10-02 14:08:07 +03:00
Aleksei Sviridkin ace69cb075 docs(controller): say what a nil DataVolume reader means 
The field comment still said DataVolumes do not count while the reader
is nil, and that the reader appears once the kind is served. Neither
holds: the reader is installed only after the informer has synced, and
until then a monitor keeps its stored verdict, which still decides
Operational.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:38 +03:00
Aleksei Sviridkin a6a32d79cd fix(controller): recompute stored DataVolume verdicts once the reader is in 
A reconcile that runs before the DataVolume watch starts finds no
reader, keeps the monitor's stored verdict and is not requeued. The
watch's initial replay then delivers only the DataVolumes that still
exist, so a monitor whose DataVolume was deleted while the controller
was down kept reporting it not ready until the periodic resync, hours
later.

Installing the reader now queues every monitor that holds a DataVolume
verdict, through the controller's own queue, so each is recomputed
against the synced cache. Monitors without a verdict are left alone:
the replay already reaches every DataVolume they could select. If the
monitors cannot be listed, the next discovery poll retries.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:37 +03:00
Aleksei Sviridkin 9f72f0bda9 feat(api): refuse a WorkloadMonitor with an empty selector 
An empty selector matches every Pod, PVC, Service and DataVolume in the
namespace, so one such monitor goes non-operational on any import there.
No chart renders one, but the schema marked the field required without
requiring any label, and the API accepted it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:37 +03:00
Aleksei Sviridkin 694f4f2ff9 fix(controller): report a disk's PVC Workload by its DataVolume 
CDI copies the DataVolume labels onto the PVC it creates, so the vm-disk
monitor also publishes a Workload for that PVC, and its readiness was
bind state alone. Mid-import the PVC is already Bound: the Workload read
operational while the monitor, and the Application's WorkloadsReady,
said the disk was not populated.

A PVC whose controller is a DataVolume the monitor finds not ready now
reads not operational. While the DataVolumes cannot be read, its
Workload keeps the verdict it has, as the monitor does.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:36 +03:00
Aleksei Sviridkin 366b5e80f4 perf(controller): skip the monitor status write when nothing changed 
While the DataVolume watch syncs, every WorkloadMonitor of every kind
is requeued every few seconds, and an informer that cannot sync keeps
that window open. Each pass wrote status unconditionally, so a cluster
with many monitors sent the API server a steady stream of writes that
changed nothing. Comparing against the stored status first leaves the
requeue as cache reads only.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:36 +03:00
Aleksei Sviridkin dd72583dfd fix(controller): record an event when DataVolumes cannot be read 
A failed DataVolume List keeps the monitor's last verdict and retries,
and the only trace of it was a controller log line. A monitor with no
verdict yet then reads operational while its disks are unreadable, with
nothing a tenant can see saying why. A Warning event on the monitor
makes the failure visible where the bucket metrics failure already is.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:36 +03:00
Aleksei Sviridkin beeb7994cc fix(controller): start the DataVolume watch source only once 
A controller-runtime Kind source adds an event handler to the informer
when it starts and never removes it; cancelling it on a sync timeout
only ends its context. Building a new source on every failed sync
attempt therefore left one more handler on the DataVolume informer per
retry, each enqueueing the same events, until the controller restarted.

The source is now started once and later attempts only wait for the
informer to sync. Because the started source keeps delivering events
between attempts, the syncing flag stays set until the reader is
installed, so a reconcile queued in that gap is requeued as well.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:35 +03:00
Aleksei Sviridkin a9b276f99b fix(controller): requeue monitors reconciled while the DataVolume watch syncs 
Starting the DataVolume source replays every DataVolume to its
handler at once, and the handler queues the matching monitors, while
the reader is installed only after the source has synced. A monitor
reconciled in that window saw no reader, kept its last verdict and
returned with nothing left to queue it again, so a disk that failed
or finished while the controller was down could keep a wrong
WorkloadsReady until the next unrelated event.

Such a reconcile still publishes the replica counts and now asks to
be requeued after a few seconds, until the reader is in place or the
sync attempt gives up. It is a requeue rather than an error because
the window is expected: an error would count as a reconcile failure
and log a line for every monitor in the cluster while it lasts.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:35 +03:00
Aleksei Sviridkin f447ab8ebd docs(vm-disk): explain which condition reports the disk populated 
Ready follows the Helm release; whether CDI finished is in
WorkloadsReady. Say so and where an operator sees it, say that a
disk created before this change keeps a release that may wait on the
DataVolume until its next update, say that an upload disk reads not
ready until the upload completes, and say that a disk waiting for its
first consumer counts as ready, so it is not read as broken.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-10-02 14:02:34 +03:00