cozystack

0 Fork 0

mirror of https://github.com/cozystack/cozystack.git synced 2026-11-06 03:57:52 +09:00

Author	SHA1	Message	Date
Aleksei Sviridkin	9620be0103	feat(postgres): add per-user createRole and bypassRls attributes (#3505 ) ## What this PR does - Adds `users.<name>.createRole` to the postgres chart, which sets the `CREATEROLE` role attribute on a managed login user. - Adds `users.<name>.bypassRls`, which sets `BYPASSRLS` on the same user. The two attributes are needed together to provision a privileged service role, see below. - Sets `createrole_self_grant` as a role-level default alongside `createRole`, without which the attribute is granted but not usable. - Refuses `createRole` on PostgreSQL 15 and older, where it is an escalation path rather than a bounded privilege. - Makes the init job record the roles it manages in a superuser-only marker instead of trusting role comments and refuses to adopt a role it did not create once `createRole` has been used, see below. - Adds `tests/user_createrole_test.yaml`: defaults, both explicit values for each attribute, composition with `replication` and with each other, the `createrole_self_grant` set and reset paths, and the version guard on v15 and v13. Adds `tests/managed_roles_test.yaml` for the marker and the adoption checks. Today a tenant that needs to create its own roles has no way to do it through the chart, because role attributes are not inherited through role membership: granting `CREATEROLE` to the generated `<db>_admin` group role has no effect on the users that are members of it, and `<db>_admin` is `NOINHERIT` anyway. The attribute has to be set on the login user itself, which only the init script can do. The remaining option is to hand the tenant the `postgres` superuser, which also gives it `COPY ... FROM PROGRAM`, arbitrary file access and command execution inside the instance pod. These flags are the narrow version of that: the tenant manages its own roles, and the superuser stays internal. Both attributes are reconciled on every init-script run, like `replication`, so setting one back to `false` emits the `NO...` form and revokes it. ## Why `bypassRls` belongs in the same PR `CREATEROLE` on its own does not let a tenant reproduce a role layout that uses row-level security, which is the common case this feature exists for. PostgreSQL only lets a role create other roles carrying `BYPASSRLS` if it holds the attribute itself, so `CREATE ROLE service_role ... BYPASSRLS` fails with `Only roles with the BYPASSRLS attribute may create roles with the BYPASSRLS attribute`. Exposing `createRole` without `bypassRls` therefore ships a feature that stops one statement short of the goal. `bypassRls` is not gated on a PostgreSQL version, because it carries no escalation of its own: `BYPASSRLS` waives row-level security inside the tenant's own databases, and the database owner already reads every row in them. It grants no reach outside the tenant's data. ## Why `createrole_self_grant` is set with the attribute This is the non-obvious half. PostgreSQL 16 grants a `CREATEROLE` holder only `ADMIN OPTION` on the roles it creates, governed by `createrole_self_grant`, whose default is empty. `ADMIN OPTION` allows granting the role onward but not `SET ROLE` to it, and creating an object owned by another role requires `SET`. The attribute therefore looks functional -- `CREATE ROLE` succeeds -- while the very next statement a tenant writes, `CREATE SCHEMA ... AUTHORIZATION <new role>`, fails with `must be able to SET ROLE`. Every tenant granted `createRole` would hit this and have to discover the parameter, so the chart sets it as a role-level default and resets it when the attribute is withdrawn. ## Why the init job stops trusting role comments and role names The init job runs as the superuser, and it trusted two things a CREATEROLE holder controls for the roles it creates. It picked the users and roles to drop by their comment, and on PostgreSQL 16 a CREATEROLE holder may `COMMENT ON ROLE` every role it created, so it could hand the chart for deletion a role the chart never created. It also adopted any role that already existed under a declared name, so a role created under a name an administrator declares later stayed under its creator's control, for example `debezium` before `users.debezium.replication: true`, or `newdb_admin` before `databases.newdb`. Neither case can be caught from the catalog. Creating a role through an intermediate role and then dropping that role leaves the squatted role with no row in `pg_auth_members` at all, while the objects, role settings and default privileges planted in it stay. On 17.11 such a `debezium` had zero memberships, and after the chart adopted it `migrator` could still run code as `debezium` with `REPLICATION` through a function it had planted. The chart therefore keeps its own record: - Chart-managed roles are members of `cozystack:managed`, a NOLOGIN role that only the superuser can grant. On 16+ the membership is `WITH INHERIT FALSE, SET FALSE`. The name is outside the username and database charsets. The delete passes act only on these members. On its first run the chart marks every role it had commented before, when no role could yet hold CREATEROLE through it. - Once any user has held `createRole`, recorded as `cozystack.createrole_granted` on the `postgres` database (the same mechanism as `cozystack.adopted_restore`), the chart refuses to adopt an existing role it did not mark, both for users and for `<db>_admin`/`<db>_readonly`. The init job fails with the role's name, and a platform administrator must drop the role. Before that point adoption works as before, so a fresh install that declares `users.app` still adopts the role CNPG bootstraps. ## The PostgreSQL 16 floor on `createRole` The floor is a security boundary, not a cosmetic check. Before PostgreSQL 16, `CREATEROLE` allowed granting membership in any non-superuser role, so a user holding it could grant itself `pg_execute_server_program` and reach command execution on the instance, which is exactly what not exposing the superuser is meant to prevent. I verified both sides on a real cluster: on 15.15 `GRANT pg_execute_server_program TO <createrole_user>` succeeds and `COPY ... FROM PROGRAM 'id'` then returns `uid=26(postgres)`; on 16.11 and 18.1 the same grant is refused because it now requires ADMIN option on the role. PostgreSQL 16 restricted `CREATEROLE` to roles the grantee administers, which is what makes the attribute safe to expose. The guard uses the same fail-fast `SECURITY:` idiom as the blocked `postgresql.parameters` list in `templates/db.yaml`. Note that a grant obtained on PostgreSQL 15 survives a major upgrade: after moving an instance to 16.11 the user kept its membership in `pg_execute_server_program` even though new attempts were refused. Operators who exposed `CREATEROLE` out of band on an older major should audit and revoke memberships in `pg_execute_server_program`, `pg_read_server_files` and `pg_write_server_files` before relying on the floor. ## Verification The marker, adoption and `createRole` paths were rendered and run against live PostgreSQL 13, 15, 16 and 17 containers. An upgrade from the previous chart marks the roles it already managed and removes a dropped user. What happens to a dropped role's objects comes from #4574 and is covered by its tests. A fresh install adopts the CNPG `app` role in the same run that grants `createRole`. A forged comment is ignored, and squatted `debezium` and `newdb_admin` roles stop the init job with nothing granted to them. The orphaned-role pass still drops the roles of a removed database, and every re-run is clean. Reverting each part of the fix in turn fails `tests/managed_roles_test.yaml`. Beyond the chart unit tests, I replayed a real migration checklist against a live PostgreSQL 17.7 instance, connected as the managed login user and never as the superuser: creation of four roles with their attributes including `CREATEROLE` and `BYPASSRLS`, `CREATE SCHEMA ... AUTHORIZATION` landing on the requested owner, trusted extensions into a named schema, per-role and per-database `search_path`, per-role `statement_timeout` and `idle_in_transaction_session_timeout`, RLS including `FORCE ROW LEVEL SECURITY` verified end to end (a plain role sees zero rows, the `BYPASSRLS` role sees them all), and `SECURITY DEFINER` functions. All of it succeeds without the superuser. The one thing that remains out of reach is `CREATE EVENT TRIGGER`, which PostgreSQL restricts to superusers and no chart flag can change. Other attributes and the predefined roles that a managed-service admin role usually wants (`pg_monitor`, `pg_read_all_settings`, `pg_maintain`) can follow the same per-user pattern in a later PR, minus the three that grant OS access (`pg_execute_server_program`, `pg_read_server_files`, `pg_write_server_files`). ### Screenshots No UI change. ### Downstream repositories `packages/apps/postgres/values.schema.json` gains two fields, which per the trigger map reaches `cozystack/terraform-provider-cozystack`: its postgres resource carries a hand-maintained schema plus an expand/flatten pair for `users`, with no codegen from `values.schema.json`, so `createRole` and `bypassRls` will be invisible to Terraform users until they are added there by hand. I searched that repository and found no open PR or issue covering it (the only open PR is a dependabot Go module bump). I have not filed the follow-up, so I am leaving the box unticked rather than claiming work that does not exist. Happy to open the issue or the provider PR if a maintainer confirms that is the preferred route. I walked the rest of the map against the diff and found nothing else: no package is added, renamed or removed, no version enum changes, no default changes, `release.prefix` and the output Secret and Service names are untouched, and no CRD outside the regenerated `packages/system/postgres-rd/cozyrds/postgres.yaml` is affected. The `README.md` parameter table is regenerated here and postgres is already in the website generator's app list, so the website path stays automated. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: not filed yet, see the note above - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(postgres): add `users.<name>.createRole` and `users.<name>.bypassRls`, which grant the CREATEROLE and BYPASSRLS attributes to a managed user so a tenant can create and manage its own roles, including roles that bypass row-level security, without the postgres superuser. `createRole` also sets `createrole_self_grant` on the user so that roles it creates can own objects. `createRole` requires PostgreSQL 16 or newer: on older majors CREATEROLE can be used to grant oneself pg_execute_server_program and reach command execution on the instance, so the chart refuses to render the attribute there. The init job now records the roles it manages as members of a superuser-only `cozystack:managed` role instead of trusting role comments. Once `createRole` has been used on an instance, it refuses to adopt an existing role it did not create. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * PostgreSQL user configuration now supports `createRole` and `bypassRls` privileges. * These privileges can be combined with replication access. Enabling `createRole` also configures the role’s self-grant behavior. * `createRole` requires PostgreSQL 16 or newer; configuring it on older versions is rejected. * Documentation * Updated configuration schemas, examples, and guides to describe the new options and PostgreSQL version requirements. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-06 20:03:34 +03:00
Aleksei Sviridkin	f5325425d6	fix(vm-instance): supply cloud-init network-data to netplan guests (#4306 ) <!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does A guest whose interfaces are rendered by netplan ignores two things the platform already puts on the wire for it, and this adds an opt-in `netplanDhcp` value that supplies the cloud-init network-data making it ask for both. `virt-launcher` starts a DHCP server on every bridge-bound interface, secondary ones included — `newDHCPConfigurator` in `pkg/network/setup/launcher/configurator.go` gates only on `iface.Bridge != nil`, and `BridgeConfigGenerator.Generate` serves the address the CNI put on the pod-side interface along with that link's MTU. The chart renders `bridge: {}` for every network, so a VM joined to a VPC subnet is offered its reserved address and its MTU. cloud-init's fallback network configuration covers a single interface, so on Debian and Ubuntu nothing ever sends a DHCPDISCOVER on the second NIC and the interface stays inert, while AlmaLinux (NetworkManager auto-profile) and Windows come up. That asymmetry is the tell: with no DHCP server at all, none of the three would work. The same DHCP server advertises the launcher pod's search domains as option 119, and `UseDomains=` defaults to `no` in systemd-networkd, so a netplan-managed link keeps none of them. A cluster domain under `.local` then reaches the mDNS exclusion in resolved and every lookup answers `REFUSED`, which is #2085. `dhcp4-overrides: use-domains: true` is enough on its own: the domains on the wire already carry whatever cluster domain the cluster actually uses, so nothing needs to be hardcoded here and #2372 does not collide with this. ### This is a regression, not a missing feature `packages/apps/virtual-machine` carried exactly this, added in `c63fb203b` ([apps] virtual-machine: add vpc support, 2025-10-21) as `templates/secret-network-config.yaml` plus a `networkDataSecretRef`, gated on `.Values.systemDisk.image`. `08a5f9890` (feat(platform): remove virtual-machine application, 2026-02-12) deleted the application and the template with it, and `vm-instance` never had one — `git log --all -S networkDataSecretRef -- packages/apps/vm-instance/` is empty. The gate could not be ported verbatim because `systemDisk.image` went away in the same split, which is the likeliest reason nobody did. #2166 is the other regression from that migration. `docs/vm-external-vlan.md` has since documented the absence as a property of the chart and prescribed a `write_files` + `netplan apply` workaround, so this PR updates that page too. ### Why it is opt-in The network-data renders only when `netplanDhcp` is `true`, and the value defaults to `false`. Keying it on `instanceProfile` instead cannot work: the profile defaults to `ubuntu` whatever image the disk holds, and it is a KubeVirt preference rather than a declaration of the guest's network stack. cloud-init passes a v2 configuration through verbatim only for the netplan renderer; sysconfig, NetworkManager and eni turn each `match` stanza into an interface literally named `cozy-en` or `cozy-eth`, so a non-netplan guest handed this configuration loses DHCP on its real NIC. Two cases where the value must stay off, both stated in its description and in `docs/vm-external-vlan.md`: - A guest not rendered by netplan (AlmaLinux, Rocky, Alpine), for the reason above. - A guest that configures a NIC statically through netplan, as the external-VLAN recipe does. Netplan writes one networkd file per definition ID and systemd-networkd applies only the first `.network` file in lexical order that matches a link, so `10-netplan-cozy-en.network` shadows `10-netplan-enp2s0.network` and the static address is never applied, whatever number the user's netplan file carries. Reproduced with `netplan generate` on Ubuntu 24.04. Matching by MAC with a v1 configuration would be renderer-independent and would not shadow a named interface, but it needs the chart to know the MACs at render time; `lookup` cannot supply them on a first install, so the chart would have to pin them, which is a larger change that overlaps #4038. ### What this does not do A VM that leaves `netplanDhcp` unset renders exactly as before: no network-data Secret, and no cloud-init disk where there was none. Turning the value on for a VM with neither `cloudInit` nor `sshKeys` adds a cloud-init disk, and because KubeVirt uses the pinned firmware UUID as the NoCloud instance-id, a guest that has never seen that id treats its next boot as a first boot (new SSH host keys); the value's description says so. On a VM that already has a cloud-init disk the instance-id does not change, and it is derived from the namespace, the name and `cloudInitSeed`, so recreating the VM keeps it too; cloud-init applies network configuration per instance by default, so such a guest picks the network-data up once `cloudInitSeed` is changed. Static addressing is still `userData`. ### Tests `tests/network_data_test.yaml`, 6 cases: the rendered network-data asserted in full (both properties, not just the presence of the key), the VM carrying network-data with no user-data requested, user-data and network-data side by side, a VM on the default profile rendering neither the Secret nor a cloud-init disk, user-data alone when the value is off, and the value rather than the profile deciding. With the templates gated on the `ubuntu` profile again, the default-profile case and the value-off case go red. `helm unittest` green on all four suites of the package (32 tests). `make generate` output is committed (`values.schema.json`, `README.md`, `api/apps/v1alpha1/vminstance/types.go`, `packages/system/vm-instance-rd/`). `helm lint` fails on this package identically on an untouched tree (`_cluster.scheduling` nil pointer, `cozy-lib` dependency metadata), so it is not a signal here. ### Screenshots Not a UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> <!-- Trigger map walked file by file against the diff. `docs/vm-external-vlan.md` documents the absence this PR removes, which is the "make a documented workaround obsolete" trigger, so the website copies of that page need the same edit and issue 702 carries it. `netplanDhcp` is a new field in `values.schema.json`, which the provider models by hand, and issue 62 there carries it. No package added, renamed or removed; no hack/ or ApplicationDefinition change; the cozy-proxy annotations in templates/service.yaml are untouched. --> - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/issues/702 - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/62 - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(vm-instance): a new `netplanDhcp` value, off by default, makes the chart supply cloud-init network-data that requests DHCP on every ethernet interface and accepts the DNS search domains offered over DHCP. A netplan guest (Ubuntu, Debian) attached to a VPC subnet then configures that interface on its own instead of leaving it unaddressed, and cluster service names under the cluster domain resolve inside the guest. Leave it off for guests not rendered by netplan and for guests that configure a NIC statically through netplan. VMs that do not set it are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * Added an opt-in setting to request DHCP on Ethernet interfaces in supported virtual machines while retaining DHCP-provided DNS search domains. * The setting is off by default. Enabling it can add a cloud-init disk and may interfere with static network settings or guests that do not use netplan. * Documentation * Updated setup guidance with considerations for enabling DHCP and configuring static networking on VLANs without a DHCP server. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-06 19:48:34 +03:00
Aleksei Sviridkin	bcda5c893a	feat(kubevirt)!: drop HotplugVolumes for declarative volume hotplug (#4453 ) ## What this PR does This turns off the deprecated KubeVirt `HotplugVolumes` gate and moves the platform to declarative volume hotplug. It changes how every volume hotplug on the platform works, so the title carries `!`. Closes #4032. This PR is stacked on #4451 and has to merge after it. #4451 adds the kubevirt-csi flag this PR flips. The replacement gate is already on in KubeVirt 1.9, and the old gate is the only thing keeping it off. `DeclarativeHotplugVolumes` is Beta at v1.9.0, and Beta gates are on by default. virt-controller skips `handleDeclarativeVolumeHotplug` while `HotplugVolumes` is on, so dropping the old gate is what switches the new one on. The kubevirt-csi fallback goes off in the same change. With the old gate off, virt-api rejects the VMI-level `removevolume` on a VM-owned VMI, so the chart now passes `--enable-vmi-hotplug-fallback=false`. The comments next to the gate and the flag point at each other. This needs the rebuilt kubevirt-csi image. A driver built before #4451 does not know the flag and exits on it at startup. The image is stamped at release preparation and ships in the same release as this change, so only a `make apply` from main before then would crash-loop the CSI controller. What changes at runtime on an upgrade: - The VM-level `addvolume` and `removevolume` calls now edit `VM.spec` directly. virt-controller then makes the VMI hotplug volumes match it, instead of going through `VM.status.volumeRequests`. - Volumes hot-plugged through the VM subresource are in `VM.spec`, so they stay attached. - During a tenant teardown, when the VM goes before its VMI, the driver no longer calls the VMI subresource. It waits for the VMI status to drop the volume, and the garbage collector deletes the VMI with its VM. Upstream waits up to two minutes. If the VMI is still there after that, the unpublish fails and is retried. A new platform migration keeps old VMI-only volumes attached. On its first pass virt-controller removes every hotpluggable VMI volume that is missing from `VM.spec`. Until July 2025 kubevirt-csi attached volumes at the VMI level (the Role then granted `virtualmachineinstances/addvolume`), and live migration carries such a VMI across KubeVirt updates. Without help, a tenant node not recreated since then would lose a mounted disk on upgrade. Migration 59 copies each such volume and its disk from the VMI into the VM template before anything else changes. Adding a hotplug volume to the VM spec is a valid live update in KubeVirt, so no VM needs a restart. It runs as the platform chart's pre-upgrade hook, before the upgrade re-points `cozystack-packages`. The KubeVirt CR and the tenant kubernetes charts both render from that source, so the gate and the flag cannot change before the migration has finished. It only touches VMs that need it, logs each one, and a second run changes nothing. The migration also makes ephemeral hotplugs permanent. A volume a user added at the VMI level, for example with `virtctl addvolume` without `--persist` from an older virtctl, has the same shape as kubevirt-csi's old attachments, because virt-api builds both the same way. There is nothing reliable to tell them apart by, so both are copied. Leaving the user's volume out would not keep it ephemeral: the first reconcile would unplug it from the running VM. After the upgrade it is part of `VM.spec` and attaches again on a restart, and `virtctl removevolume` drops it. Ephemeral hotplug stops working on VMs. With the old gate off, virt-api refuses VMI-level `addvolume` and `removevolume` on a VMI that a VM owns (`vmiVolumePatch` in v1.9.0). A volume on such a VMI is added and removed through the VM from then on. A VMI without a VM is not affected. Only running VMIs are copied. virt-controller reconciles no other VMI, and a stopped VM builds its next VMI from `VM.spec`, so copying from a stopped one would bring back volumes the restart was going to drop. Migrations run only on the isp-full, isp-full-generic and isp-hosted variants. KubeVirt comes from the platform only on isp-full and isp-full-generic, so those are covered. On the default variant, where packages are installed by hand, nothing runs the migration. This optional check lists the VMIs and volumes the migration would copy. It uses the same rule. On the default variant, run it before upgrading and recreate each tenant node it lists: ```bash kubectl get vm,vmi --all-namespaces --output json \| jq --raw-output ' ([.items[] \| select(.kind == "VirtualMachine") \| {key: "\(.metadata.namespace)/\(.metadata.name)", value: [.spec.template.spec.volumes[]?.name]}] \| from_entries) as $vm \| .items[] \| select(.kind == "VirtualMachineInstance") \| ([.metadata.ownerReferences[]? \| select(.controller == true and .kind == "VirtualMachine" and (.apiVersion \| startswith("kubevirt.io/"))) \| .name][0]) as $owner \| "\(.metadata.namespace)/\($owner)" as $key \| select(.status.phase == "Running" and $owner != null and $vm[$key] != null) \| .spec.volumes[]? \| select(.dataVolume.hotpluggable == true or .persistentVolumeClaim.hotpluggable == true) \| .name as $n \| select($vm[$key] \| index($n) \| not) \| "\($key) \($n)"' ``` Empty output means nothing is affected. I checked the query against the migration's test fixtures only. Neither the query nor the migration has run on a live cluster. #4287 also adds a migration 59. If it merges first, this one moves to 60: the file, the version it stamps, `migrations.targetVersion`, the bats file (its `run_migration` argument and expected `STAMP`) and `hack/testdata/migration-59-vmi-hotplug`. A VM with a pending hotplug request at upgrade time may also get stuck. virt-controller still processes `VM.status.volumeRequests` before the declarative handler, and it applies them through the VMI subresource, which virt-api rejects once the old gate is off. The sync then fails before the VM spec is updated, so nothing clears the request, and every later sync of that VM fails the same way. A retry through the VM subresource edits the spec and clears it, which covers kubevirt-csi's own requests. A one-shot request stays stuck until the VM restarts. I read this from the v1.9.0 code and haven't reproduced it. Waiting until this lists nothing before upgrading narrows the window, but a request made during the rollout can still hit it: ```bash kubectl get vm --all-namespaces --output json \| jq --raw-output '.items[] \| select((.status.volumeRequests // []) \| length > 0) \| .metadata.namespace + "/" + .metadata.name' ``` The two halves live in different HelmReleases, so there is a short window during the rollout where the driver and the gate disagree. In that window an unpublish fails and is retried until both are updated. A tenant cluster whose `kubernetes` HelmRelease is suspended or failing keeps its old driver for longer. Drivers from before July 2025 attach through the VMI subresource, so on such a tenant every attach is refused until its release reconciles. Such a driver can also attach a volume after the migration has run and before the gate flips. The migration doesn't see that volume, and the first declarative reconcile unplugs it from the running VM. The migration lists the VMs once and patches each one with a `resourceVersion` test, so a VM that changes in between fails the hook Job. The Job has `backoffLimit: 3`, and each retry lists again and skips the VMs already patched. Re-reading only the conflicting VM would avoid the retry; I left that out. The new gate is listed in the CR even though it is on by default. With the old gate gone it is the only hotplug path, and the chart refuses a render that disables it. I found no other user of `HotplugVolumes` in the platform or in the downstream repositories. The VMI-level fallback may have been failing already. The kcsi Role has no rule for the VMI-level `removevolume` (#4452), so that call likely got 403 with or without the gate. I haven't checked that on a cluster. `hack/migration-59-vmi-hotplug-volumes.bats` runs the migration against a fake kubectl: a VM that needs the copy, VMs and VMIs that must be left alone (a VMI that is not Running among them), a second run, a cluster without KubeVirt and the fail-closed paths. The helm-unittests for the KubeVirt CR pin the whole gate list, accept `HotplugVolumes` in `disabledFeatureGates` and refuse `DeclarativeHotplugVolumes` there. The CSI test pins the flag at false, and the `make update` idempotency test still passes. None of that exercises hotplug itself. The real test is e2e with volume attach and detach on a tenant cluster, including a teardown. No CI lane upgrades from the previous release yet, so an upgrade with a live VMI-only volume is untested; #4463 tracks that test. ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(kubevirt)!: KubeVirt now runs with DeclarativeHotplugVolumes instead of the deprecated HotplugVolumes feature gate, and kubevirt-csi no longer uses the VMI-level hot-unplug. Volume hotplug requests are applied through VM.spec. The upgrade copies volumes that older kubevirt-csi versions attached only to the VMI into VM.spec, so they stay attached; the check in the PR description is optional. This also applies to ephemeral volumes added to running user VMs at the VMI level, such as with virtctl addvolume without --persist: they become part of VM.spec and attach again after a VM restart. virt-api now refuses VMI-level addvolume and removevolume on a VMI owned by a VM, so such volumes are added and removed through the VM. A tenant cluster whose kubernetes release is suspended or failing keeps its old kubevirt-csi, and a driver from before July 2025 cannot attach volumes there until that release reconciles, and a volume it attaches during the upgrade itself can be unplugged again. A values override that puts DeclarativeHotplugVolumes into disabledFeatureGates now fails the kubevirt render. Clusters on the default variant do not run platform migrations and should run that check before upgrading. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * Declarative hotplug volumes are enabled by default for KubeVirt. * Running virtual machines keep eligible hotplugged volumes and disks when their configuration is migrated. * Bug Fixes * The migration avoids adding duplicate volumes or disks and does not mark the migration complete if required operations fail. * The Kubernetes CSI driver no longer uses the VMI hotplug fallback. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-06 17:06:03 +03:00
IvanHunters	f05718b2de	feat(redis)!: refuse external access without TLS at admission (#4639 ) <!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does An admission policy now refuses a Redis release that would expose a plaintext endpoint outside the cluster. Redis never turns TLS on by itself, so `external: true` without `tls.enabled: true` is exactly that case. The error names the release and a remedy that works. The rule has three parts, because TLS is fixed when a Redis instance is created: the RedisFailover schema refuses a change, while the Redis API accepts one and the HelmRelease then fails. A spec's `tls.enabled` is therefore a request, not proof of what the instance runs. - On create, `external: true` needs `tls.enabled: true`. - On update, turning `external` on is refused whatever `tls.enabled` says. - While `external` stays on, `tls.enabled` cannot change. Before the first reconcile nothing else stops TLS being dropped from a release admitted as external with TLS. Withdrawing `external` in the same edit lifts the freeze, and the callers trusted by `cozystack-tenant-host-policy` are exempt, so an instance whose TLS was flipped earlier can still be unblocked. Edits to a release that was already external pass, so releases exposed in plaintext before this policy keep working. The guarantee covers the Redis app. A generic publisher such as `tcp-balancer` can still forward to a plaintext Redis, by design. This diverges from the Security section of the `external-database-exposure` proposal, which refuses only an explicit `tls.enabled: false` and covers every engine. That rule was written before redis had TLS and matches only postgres. Postgres is left out here on purpose: CloudNativePG keeps `ssl=on` and its default pg_hba accepts plaintext from any client that asks, whatever the chart's `tls.enabled` says, so an admission rule on that field closes nothing. That gap is #4619. The proposal update and a decision record are cozystack/community#91. The policy follows `cozystack-tenant-host-policy`: a ValidatingAdmissionPolicy in `cozystack-basics`, enforced inside `cozystack-api` for the aggregated kind. Its decisions are tested in Go by compiling the shipped template with the apiserver's own CEL compiler and running Redis objects through it, including the bypass sequences found in review. A structural chart test can't show what the policy admits. The redis README, the `external` field description and the threat model describe the new rule. Related: #4622 (valkey `external` is always plaintext), #4624 (the redis `tls.enabled` description implies a live switch). ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: I walked the trigger map against the diff and nothing in it fires, but I left the boxes empty because cozystack/community is reached anyway: the `external-database-exposure` proposal still states the old admission rule. Its update and a decision record are cozystack/community#91. ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(redis)!: a Redis release can be exposed outside the cluster only when it is created with `tls.enabled: true`. Creating one with `external: true` and no TLS, turning `external` on for an existing instance, or changing `tls.enabled` while `external` is on is refused at admission. Instances already external without TLS keep working and stay editable. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * Redis admission checks now require TLS when external access is enabled at instance creation. They block enabling external access on an existing instance and changing TLS while external access remains enabled. Disabling external access in the same update allows TLS changes. * Existing instances that already had external access without TLS before the policy was introduced remain editable. * Documentation * Updated Redis configuration and security guidance to explain these restrictions and TLS migration behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-06 16:39:23 +03:00
Mattia Eleuteri	21fa1fcdb1	feat(postgres): add per-user createRole and bypassRls attributes Without these, a tenant that needs to manage its own roles has no option but the postgres superuser, which also hands over filesystem access and command execution on the instance. Role attributes are not inherited through role membership, so the generated <db>_admin group role cannot carry them: they have to be set on the login user itself. users.<name>.createRole and users.<name>.bypassRls set CREATEROLE and BYPASSRLS on a managed login user. Both are reconciled on every init-script run, so setting one back to false emits the NO... form and revokes the attribute. createRole is refused on PostgreSQL 15 and older. Before PostgreSQL 16, CREATEROLE allowed granting membership in any non-superuser role, so a user holding it could grant itself pg_execute_server_program and reach command execution on the instance, which is exactly what not exposing the superuser prevents. PostgreSQL 16 restricted CREATEROLE to roles the grantee administers. bypassRls carries no such gate: BYPASSRLS only waives row-level security inside the tenant's own databases, whose owner already reads every row, so it adds no reach. The two attributes are needed together, not separately. PostgreSQL only lets a role create other roles carrying BYPASSRLS if it holds the attribute itself, so provisioning a privileged service role fails with "Only roles with the BYPASSRLS attribute may create roles with the BYPASSRLS attribute" when only createRole is granted. createRole also sets createrole_self_grant as a role-level default. PostgreSQL 16 grants a CREATEROLE holder only ADMIN OPTION on the roles it creates, governed by that parameter, whose default is empty. ADMIN OPTION allows granting the role onward but not SET ROLE to it, and creating an object owned by another role requires SET. The attribute therefore looks functional -- CREATE ROLE succeeds -- while the next statement a tenant writes, CREATE SCHEMA ... AUTHORIZATION <new role>, fails with "must be able to SET ROLE". The default is reset when the attribute is withdrawn. The init job runs as the superuser, and before this change it trusted two things a CREATEROLE holder controls for the roles it creates: role comments and role names. It picked the users and roles to drop by their comment, and a PostgreSQL 16 CREATEROLE holder may comment every role it created, then hand it to the chart for deletion. It adopted any role that already existed under a declared name, so a role created under a name an administrator declares later stayed under its creator's control. Neither can be caught from the catalog: creating a role through an intermediate role and then dropping that role leaves the squatted role with no member at all, while the objects, settings and default privileges planted in it stay. The chart therefore records the roles it manages as members of "cozystack:managed", a role only the superuser can grant, and the delete passes act only on those. Its first run marks every role the chart had commented before, when no role could yet hold CREATEROLE through it. Once any user has held createRole, recorded as a setting of the postgres database, the chart refuses to adopt a role it did not mark and fails with the role's name, rather than hand a squatted role REPLICATION or the ownership of a database. Verified against live PostgreSQL instances. On 17 a migration checklist replayed as the login user, with no superuser: role creation with attributes, CREATE SCHEMA ... AUTHORIZATION with the requested owner, extensions in a named schema, per-role search_path and timeouts, RLS including FORCE ROW LEVEL SECURITY, and SECURITY DEFINER functions all succeed. On 17 the forged-comment and squatting paths are refused or ignored, the upgrade from the previous chart marks the roles it already managed, and a fresh install still adopts the role CNPG bootstraps for users.app. On 13, 15 and 16 the init script runs and re-runs cleanly. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io>	2026-10-06 16:26:59 +03:00
Aleksei Sviridkin	9734a16692	feat(postgres): add a postgis image flavor (#4587 ) <!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Adds a `flavor: postgresql \| postgis` enum to the `postgres` app. `postgis` runs the cluster on the CloudNativePG PostGIS operand image, so tenants can enable `postgis`, `postgis_raster`, `postgis_topology`, `postgis_sfcgal`, `postgis_tiger_geocoder` and `address_standardizer` through the existing `databases.<name>.extensions`. The default stays `postgresql` and renders exactly the image it rendered before this change. This picks up #2672 by @matthieu-robin, which the stale bot closed after a rebase request, with these changes: - The postgis tags in #2672 (`17.7-3.5`, `16.11-3.5`, ...) do not exist in `ghcr.io/cloudnative-pg/postgis`; the registry answers 404, so the flavor would have ended in `ImagePullBackOff`. The map now uses tags that exist, `<minor>-<postgis>-standard-trixie`, on the same PostgreSQL minor as `files/versions.yaml`. - v18 is supported: PostGIS images for 18 now exist upstream. - `standard` rather than `system`: the chart archives through the barman-cloud plugin since the backup rework, so the operand no longer needs the barman binaries of the deprecated `system` variant. - `hack/update-versions.sh` derives the postgis map from the minors it picks, and `hack/check-postgres-postgis-versions.bats` fails if the two maps drift to different minors or away from `standard-trixie`. - The chart refuses a flavor change on a live `Cluster`. The default images are built on Debian bullseye (glibc 2.31) and the postgis ones on trixie (glibc 2.41): CNPG would roll the swap out as a plain image update, under an initialised data directory, which silently invalidates every index on collatable text, and the postgis-to-postgresql direction also drops the libraries behind every PostGIS object. Only the image name is compared, so minor bumps and registry mirrors keep passing. - `pgrouting` is not claimed: it is absent from the `18.1-3.6.1` build the chart pins and only appears in later postgis builds. A general mechanism (CNPG ImageVolume extensions) was considered and left out: it needs PostgreSQL 18 and Kubernetes 1.35 or the `ImageVolume` gate, so it cannot serve v13-v17 nor most management clusters today, and PostGIS is the only extension CloudNativePG ships as a full operand image. No other extension request exists in this repository. Testing: - `helm unittest packages/apps/postgres`: 145/145, including 6 new cases in `tests/flavor_test.yaml` (default image, postgis on the Cluster and on the init Job, the guard in both directions with a mocked live Cluster, and a mirrored digest-pinned image taking a new minor). Removing the guard makes exactly the two guard cases fail. - `hack/cozytest.sh hack/check-postgres-postgis-versions.bats` passes, and fails on a shifted minor or a `system` tag. - `make -C packages/apps/postgres generate` leaves no diff. - The rendered `ghcr.io/cloudnative-pg/postgis:18.1-3.6.1-standard-trixie` was started locally: `CREATE EXTENSION` for citext, postgis, postgis_raster and postgis_topology succeeds and `ST_Buffer` answers. A three-instance cluster on the `system` sibling of that image has been running PostGIS in production since 2026-09-29. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: cozystack/terraform-provider-cozystack#63 (issue: the provider models every field by hand, and the default handling needs a decision there) - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(postgres): add a `flavor` field accepting `postgresql` (default) or `postgis`. `postgis` bootstraps the cluster from the CloudNativePG PostGIS image, so PostGIS extensions can be enabled per database through `databases.<name>.extensions`. The flavor is fixed at creation: changing it on an existing cluster is refused, and so is restoring a backup into a cluster of the other flavor, whether through a RestoreJob or `bootstrap.enabled`; move data across flavors with a logical dump. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - New Features - Added a `flavor` option for PostgreSQL clusters. The default `postgresql` flavor remains available, while `postgis` selects an image with PostGIS extensions. - PostGIS images are available for PostgreSQL versions 13–18. - Cluster flavor is fixed at creation. To switch flavors, create a new cluster and migrate data with a logical dump. - Bug Fixes - Restores are refused when the backup and target cluster use different flavors. - Documentation - Updated PostgreSQL configuration guidance to describe flavor options and limitations. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-06 16:17:22 +03:00
Mattia Eleuteri	39bdc9c404	fix(vm-instance): make the netplan network-data opt-in Keying the network-data on instanceProfile reached every VM that never set the field, because the profile defaults to ubuntu whatever image the disk holds. The sysconfig, NetworkManager and eni renderers turn each match stanza into an interface literally named after the netplan ID, so a non-netplan guest left on the default profile would lose the DHCP it gets today. A VM with neither cloudInit nor sshKeys would also gain a cloud-init disk on upgrade, and KubeVirt's NoCloud instance-id is new to that guest, so its next boot would regenerate host keys as a first boot. The configuration also shadows a static NIC set up through netplan: netplan emits one networkd file per ID, networkd applies only the first file in lexical order that matches a link, and 10-netplan-cozy-en sorts before the file for enp2s0. The external-VLAN recipe claimed otherwise. An explicit netplanDhcp value, off by default, leaves every existing VM rendering exactly as before and lets the documentation say when not to turn it on. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io>	2026-10-06 16:11:33 +03:00
Alexey Artamonov	e63b9e439c	fix(monitoring): count a central request as the receiver admitted it The limiter refuses for good any request it counts above burst, and burst was the receiver's 4 MiB cap. A request grows on its way to the limiter: the sampler adds a tracestate to every span it keeps, and the limiter counts a request as the collector re-encodes it, which writes the empty value of every attribute the wire may omit, so a valid request of spans with empty-valued attributes is counted 1.4x. A request the receiver admitted near its cap was therefore refused on every retry. The limiter now runs right after memory_limiter, so it counts the request before sampling. The receiver caps a request at 3 MiB and burst stays at 5 MiB, which holds 1.4x the cap; a larger burst would also be what each pod start lets through above the rate. The rate is counted before sampling, so the store keeps only the sampled share of it. The texts say that a request above the cap gets a 400, which senders do not retry, and that each collector rollout a tenant causes lets that burst through again on every pod. The replica bound that keeps that allowance small now applies only to a central tenant's collector, whose render fails above ten replicas: a collector writing to the tenant's own store has no rate, and the replica counts it shipped with stay valid. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
Alexey Artamonov	65f0caae2b	fix(monitoring): keep a central tenant's write login out of its Grafana A tenant's Grafana pod may reach the shared traces store's vmauth, and the Secret a tenant names in oidc.customConfig.secretRef was mounted into it whole. The apps API does not prune keys the Monitoring schema leaves out, so a tenant could also set grafana.image: an image of its choosing, with the collector's Secret mounted, writes past the rate limit the collector enforces. Grafana's image is the platform's, like the collector's, and for a central tenant only the auth.ini key Grafana reads is mounted from the named Secret. Other tenants have no collector login to keep out and keep the whole Secret, which an auth.ini reading a sibling key with $__file{} relies on. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
Alexey Artamonov	cc51cac63d	fix(monitoring): give a central tenant's Grafana a login that cannot write The collector holds a central tenant to its write rate, but the one login it writes with was also the login of the tenant's Grafana datasource, and it could both read and write. vmauth cleans the request path before it picks a route, so any user the tenant admits to its Grafana, a Viewer included, could POST through the datasource proxy to ../insert and write into the shared store past the limiter. Each central tenant now has two VMUsers on its one account: the collector's, which may only write, and Grafana's, which may only read, each with its own chart-owned Secret and password. A namespace name holds no dot, so the reader's username never matches another tenant's. On upgrade the datasource moves to the new login in the same release that creates it, so reads may fail until vmauth loads the new user. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
Alexey Artamonov	3497855e15	fix(platform): read the tracing rate only as plain decimal digits Sprig's int64 reads a string with a leading zero as octal and turns a parse error into 0, so "0100" would reach the collectors as 64, and "08" or digits beyond int64 would leave the default in place without a word. A float beyond int64 converts differently on each CPU architecture, and the collector adds its replica count to the rate before splitting it. A string rate must now be plain decimal digits, and any rate above 1 TiB/s fails the render; the collector also falls back to the default on digits it cannot hold rather than start with a rate of zero, which the processor refuses. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
Alexey Artamonov	fc0db49187	fix(monitoring): bound what a central tenant writes above its rate Each collector pod starts with a full token bucket, and burst was at least 64 MiB, the largest request vtinsert accepts. A tenant controls when its collector pods start, through any Monitoring change or by deleting them, and how many there are, so it could write tens of mebibytes above its rate per pod start and gigabytes per rollout of many replicas, which the two-day bound does not survive. The limiter only sees requests the receiver admits, so in central mode the receiver now caps them at 4 MiB, decompressed for HTTP, and burst drops to that floor; a pod start lets at most 4 MiB through above the rate. The replica count is capped at 10. The texts now say what the default rate guarantees: one tenant writes at most a default store's disk cap over two days counted uncompressed, rather than that it cannot fill it, and they name the per-pod-start allowance and that a sender pinned to one replica gets that replica's share. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
Alexey Artamonov	177b207a27	feat(monitoring): hold each shared-central tenant to a write rate The design has the per-tenant collector rate-limit its tenant, so that a noisy or hostile tenant cannot exhaust a shared backend. The disk cap alone does not do that: VictoriaTraces always keeps the last two days, so a tenant writing faster than the cap allows over two days fills the disk for every tenant on the store. The platform sets the rate with monitoring.tracingCentralTenantBytesPerSecond, and it reaches every tenant in _cluster through the cozystack-values Secret. A change there is a change in the values each monitoring HelmRelease reads, so Flux upgrades them all; a rate read with lookup at render time would never reach a tenant whose release is already in sync. The platform writes the key only when it is set, since every release in every tenant reads that Secret. Without it the collector holds the tenant to the rate of a default 10Gi store. Each central tenant's collector splits the rate across its replicas and refuses data above it before the batch processor, so the sender sees the refusal. An OTLP/HTTP sender gets a 429 and retries; the limiter answers gRPC with RESOURCE_EXHAUSTED and no RetryInfo, which the OTLP specification makes final, so central tenants should export over HTTP. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
Alexey Artamonov	474377e8ca	build(monitoring): build the traces collector with a rate limiter A shared-central traces store needs each tenant held to a write rate, or one tenant can fill the disk every tenant shares. No upstream OpenTelemetry distribution or processor limits throughput. Elastic's EDOT Collector registers Elastic's ratelimitprocessor among its default components, but it ships under the Elastic License 2.0 and carries far more components than the chart uses. The collector is therefore built here with the OpenTelemetry Collector Builder from the upstream components the chart already uses, at the same v0.136.0, plus that ratelimitprocessor (Apache-2.0). Its v0.8.2 is the release built against v0.135.0, the last before v0.137.0. The image is versioned by the collector release with a cozystack respin counter, runs as the upstream image's user, and is built with the other package images. The builder stage is alpine, which has no git, so a proxy error during its go install would have had neither the direct fallback nor a retry; the stage installs git. The module-fetch contract saw only go build, so it now checks go install too, and takes only git as cover for a versioned go install, which resolves outside the stage's module. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>	2026-10-06 09:39:04 +03:00
IvanHunters	0639ce39aa	fix(kamaji): drop any subchart nested under the vendored kamaji chart (#4459 ) ## What this PR does A subchart nested under the vendored kamaji chart would deploy no matter what the values say, so this PR removes that directory in `make update` and adds a test that fails if anything renders from it. The vendored chart declares a `kamaji-etcd` dependency gated on `kamaji-etcd.deploy`, and the subchart itself is not vendored. The issue proposed pinning `kamaji-etcd.deploy: false` in the package values. I tested that, and it would not have helped: through this package the condition is never evaluated at all. Helm skips the condition because the umbrella chart declares no dependencies. `processDependencyEnabled` returns immediately when the chart it starts from has no `dependencies` in its `Chart.yaml`, and `cozy-kamaji` has none. The vendored `kamaji` chart is loaded from `charts/` without being declared, so Helm never recurses into it and never reads its `kamaji-etcd.deploy` condition. The same early return is in Helm v3 and v4. I put `kamaji-etcd-0.15.0.tgz` into `charts/kamaji/charts/` and rendered the package three ways: - default values: StatefulSet, Job, DataStore and four Certificates from `kamaji-etcd` - `kamaji.kamaji-etcd.deploy: false` in the package values: the same etcd - `deploy: false` edited into the vendored `charts/kamaji/values.yaml`: the same etcd Rendering `charts/kamaji` on its own with `kamaji-etcd.deploy=false` renders no etcd, which confirms the umbrella is what skips the condition. So any chart that ends up under `charts/kamaji/charts/` deploys whatever the values say, and no values change can close the hazard. This PR changes two things: - `make update` removes `charts/kamaji/charts` after extracting the upstream chart. Upstream does not ship that directory today, so the recipe still reproduces the current tree exactly. I checked that by running it on a copy of the package. I also ran it against a fake upstream tarball that carried the subchart: the directory was extracted and then removed. - The package gets its first helm-unittest suite. It asserts that no template of a subchart nested under `charts/kamaji/charts/` renders a document, and a `test` target lets `hack/helm-unit-tests.sh` pick it up in `make unit-tests`. With the subchart tgz dropped in, the suite fails. With the `rm` line removed from the recipe, the fake-upstream update leaves the tgz behind and the suite fails too. `/tests` is anchored in `.helmignore`, so the packaged chart's file list is unchanged. I left the `kamaji-etcd` block in the vendored values alone. It is not quite dead: `certmanager_issuer.yaml` reads it to decide whether to keep the self-signed issuer. That only matters when `certManager.issuerRef.name` is set, and this package does not set it. `helm template` output is byte-identical to main. Closes #4397 ### Screenshots Not a UI change. ### Downstream repositories The change touches only the kamaji package's update recipe, its test suite and its `.helmignore`. The closest trigger is package `Makefile`s → the website's `development.md`, but that page describes `make update` and `make test` in general terms that still hold, and the rendered chart is unchanged. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kamaji): `make update` now drops any subchart nested under the vendored kamaji chart, and a unit test fails if one appears, because such a subchart (for example the bundled kamaji-etcd) would deploy regardless of `kamaji-etcd.deploy`. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * Bug Fixes * Helm chart updates now avoid including nested charts that could deploy unexpectedly. * Helm chart packages exclude test files, keeping them out of distributed chart artifacts. * Tests * Added a chart check to verify nested charts render no resources. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-05 15:14:48 +03:00
IvanHunters	6a98acb61d	fix(api): leave failed golden images out of the image options (#4568 ) <!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The VM image picker no longer offers golden images whose import failed. A golden keeps its PVC in `cozy-public` after the import fails, so a picker built on the PVC list offered an image whose clone never completes. The `image` option source in cozystack-api now reads the `vm-default-images-` DataVolumes next to the PVCs and drops names whose DataVolume is in phase `Failed`. I filter rather than mark them, because such an option can never be used and `OptionItem` has no disabled state. A golden that is still importing stays in the list, since CDI holds a clone until the source is populated, and so does a PVC with no DataVolume. When the DataVolumes cannot be read, the PVCs go out unfiltered. The Option list drops a source whose provider errors, and an empty picker would be worse than the old behaviour. cozystack-api gets a read grant on `cdi.kubevirt.io` DataVolumes for this. The console did not use that source at all. `SourceField` listed the PVCs itself, as the tenant, so the fix above would not reach the picker users see. It now reads the same Option list the other dynamic dropdowns use. With that, nothing in the console lists `cozy-public` PVCs, so I removed the read grant on them from the `cozy:tenant:dashboard` Role. That is a visible change: a tenant loses `kubectl get pvc --namespace cozy-public`, which was only ever granted for this picker in #2843. One limitation: in CDI v1.66.1, which cozystack ships, a golden import reaches `Failed` only on a checksum mismatch, and a broken URL keeps it in `ImportInProgress`. So this filter covers less than the issue assumed. I filed #4564 for that. Closes #4553 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> I have no screenshot. The picker looks the same; only the list behind it changes. `apps/console/src/components/SourceField.test.tsx` covers it: the picker shows the images the option source serves and not the raw PVC names. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(api): the VM image picker no longer offers golden images whose import failed. The console now reads images from the image option source, and tenants no longer get read access to PersistentVolumeClaims in cozy-public. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit New Features * The image picker now displays available images from the tenant’s image options. * Bug Fixes * Images whose imports have failed are excluded from the picker. Images remain available when import status is still in progress, has succeeded, or cannot be retrieved. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-05 15:14:25 +03:00
IvanHunters	f9f7562e56	fix(kamaji): resolve Gateway API availability once at startup (#4501 ) ## What this PR does Carries the fix proposed upstream in clastix/kamaji#1327 (not yet merged there; issue clastix/kamaji#1326) as a build patch on the 26.9.4-edge tag that cozystack builds Kamaji from. In Kamaji 26.9.4-edge, every TenantControlPlane reconcile checks whether the Gateway API is available. The discovery call ignores the reconcile context, so it is bounded only by the client's 32s default timeout, which is longer than the 30s reconcile budget. The watches, however, read availability only once, at startup. With the patch, Kamaji resolves availability once when the manager starts. The watches, the indexer, the reconciled resources and the admission webhook's Gateway API check all use that one value. A discovery error at startup now stops the manager, so it is no longer mistaken for "Gateway API not installed". A Gateway API installed after Kamaji starts needs a Kamaji restart. On this platform that case does not come up: the kamaji package depends on cert-manager, and cert-manager depends on gateway-api-crds. The webhook's separate TLSRoute check still runs discovery on each request, as before, and is listed as a follow-up in clastix/kamaji#1327. Kamaji's chart sets no `timeoutSeconds` on the webhook, so the apiserver's 10s default applies, and a discovery call that can take up to the client's 32s can make that admission time out. The patch is the upstream commit applied unchanged except for the dropped docs hunk, which is not part of the image. It applies cleanly to 26.9.4-edge with no offsets. It modifies different files than the existing `fix-kubelet-config-compat.diff`, so the two apply in either order. The upstream tests stay in the patch so that it matches the upstream commit; the image build does not run them. On the patched tree, `go build ./...` passes, and `go test -race` passes for `./controllers/...`, `./internal/webhook/...`, `./internal/utilities/...`, `./internal/kubeadm/...` and `./cmd/...`. I also checked that the new tests can fail: they go red when the availability check is inverted and when the old per-reconcile lookup is put back. If clastix/kamaji#1327 changes before it merges, this patch should be updated to match. It is dropped once Kamaji ships the change. Refs #4489. This does not close #4489, because this change does not explain the long stall reported there. ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kamaji): Kamaji now checks for the Gateway API once at startup instead of on every TenantControlPlane reconcile, so a slow discovery endpoint can no longer use up the reconcile timeout. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * Bug Fixes * Gateway API availability is now checked at startup and consistently applied to Gateway-related setup and configuration validation. * Gateway configuration is rejected if the Gateway API was unavailable when Kamaji started. If you install the required CRDs afterward, restart Kamaji before configuring Gateway resources. * Discovery errors now stop startup rather than allowing Kamaji to continue with an uncertain Gateway API status. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-05 14:43:51 +03:00
IvanHunters	bb2946990d	fix(vm-instance)!: refuse guest memory below KubeVirt's 1M floor (#4569 ) <!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The vm-instance resource guard now refuses guest memory below 1M, the same floor KubeVirt admission enforces. Before, a value such as `500m` or `1Ki` rendered and then failed when the VirtualMachine was applied. This is a breaking change. A memory above zero and below 1M rendered before and is now refused at render time. KubeVirt v1.9.0 would not admit such a VM either: `validateMemoryRequestsNegativeOrNull` refuses a non-zero request below 1M (`pkg/virt-api/webhooks/validating-webhook/admitters/vmi-create-admitter.go:1135`). The VirtualMachine admitter runs the same check on its template. The cpu, sockets and memory descriptions now list the values the guard refuses, so the schema states the same rule as the error. The error cannot quote an unquoted number as it was written, because YAML has decoded it before the chart sees it. A large one also printed in exponent form, so `4294967296` came out as `4.294967296e+09`. A string is still quoted exactly as written. A number is now printed in plain digits and marked "read as", which is the closest honest wording I found. Closes #4413 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(vm-instance)!: refuse `resources.memory` below 1M at render time, since KubeVirt does not admit a VM with less. A value above zero and below 1M rendered before and is now refused. The cpu, sockets and memory descriptions list the refused values, and the guard error marks an unquoted number as read rather than written. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * Bug Fixes * VM resource values are now checked when rendering: CPU cores and sockets must be positive whole numbers no greater than 4294967295. Unit-suffixed, fractional, zero, negative, and quoted values with leading zeros are rejected. * Memory values below `1M` are rejected, with clearer messages showing the value as entered or interpreted. * Documentation * Updated resource guidance to describe these limits and validation behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-05 14:43:15 +03:00
Aleksei Sviridkin	2af8b0c229	ci(build): fall back to direct, retry and cache Go module downloads (#4660 ) <!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does One transient error from proxy.golang.org is enough to fail an image build or the `Unit & controller tests` job today. The error is an HTTP/2 stream reset (`stream error ... INTERNAL_ERROR`), and nothing retries or falls back after it. This PR adds three things: a direct fallback, a bounded retry and a module cache. Image builds now fall back to fetching modules from their origin when the proxy fails. They get `GOPROXY=https://proxy.golang.org\|direct` as a build arg from `hack/common-envs.mk`, and every Go builder stage declares `ARG GOPROXY`. With a pipe Go goes direct on any proxy error. With Go's default comma it does that only on a 404 or 410. The value comes from `IMAGE_GOPROXY` and not from the host's `GOPROXY`, because buildkit can't reach a mirror on localhost or a `file://` proxy. Every workflow job that runs Go on the runner sets the same value as env. Explicit module downloads in Dockerfiles are retried three times with a growing pause. The loop is inline, in one form that the contract test pins. I tried a shared script from a named build context first, but buildx refuses any `--build-context` for the four Dockerfiles still pinned to the `dockerfile:1.2` frontend. The images also build from different contexts, so no single file can be copied in. Raising those pins changes how those images build and needs a real build to check, so I left it for a follow-up. Two alpine stages, token-proxy and shadowbox's `ss-server`, left the download to `go build`. The alpine golang images have no git, so the direct fallback can't fetch for them, and they now get the same download step. The runner jobs use `hack/go-mod-download.sh` with the same bound. The unit test job now starts from a module cache instead of downloading everything. It restores `GOMODCACHE` with `actions/cache`, keyed on every `go.sum` in the tree, and then prefetches all tracked modules through the retry, so a cache miss still works. A cache saved by a PR run is visible only to that PR, but one saved on the default branch can be restored by any PR ([docs](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache)). That is why the build cache warmer on main writes the same key before its image build. This way the first run of a PR, which is the one that flakes, starts warm. The warmer restores only an exact key, so old module versions don't pile up, and its Go steps can't fail the image build. Both runner labels are Linux, so the key matches. Both also check out under `/home/ubuntu/_work`, so `go env GOMODCACHE` should give the same cache path, but I have not seen it on the warmer's runner yet. The release jobs in `tags.yaml` and `promote-rc.yaml` get the GOPROXY env only, no cache. `hack/go-module-fetch-contract.bats` covers all three. Each test is red on main, and I checked each guard with a mutation. Closes #4591 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ci(build): Go module downloads in image builds and CI jobs now fall back to direct fetching and retry on transient proxy.golang.org errors. The unit test job reuses a module cache written from main. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - Reliability - Go module downloads now fall back to direct access when the public proxy encounters errors. - Builds and automated checks retry failed module downloads, helping them recover from temporary network issues. - Performance - CI caches Go modules to reduce repeated downloads and speed up builds and tests. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-05 14:14:47 +03:00
IvanHunters	1621c471f2	ci(dashboard): run eslint on the console and clear its errors (#4692 ) ## What this PR does The console now passes its own eslint config, and CI runs it. On main `pnpm lint` exits 1 with 54 problems, 51 of them errors, and `ui-test.yaml` never ran it. On this branch it exits 0 with no errors, and a Lint step sits between Typecheck and Test. How each rule was handled: - `no-explicit-any` (38 errors): replaced with real types. These commits build a vite bundle byte-identical to main's. - `react-refresh/only-export-components` (6 errors): the hooks, context objects, `resolveSource` and the RJSF registry moved out of component files into their own modules. `useTenantContext` stays in `tenant-context.tsx`, so its importers and the tests that mock that module are untouched. - `no-unused-vars` (1 error): dropped the unused `ad` binding in `OverviewTab`. - `react-hooks/set-state-in-effect` (6 errors): relaxed per line with a reason, not in the config. The last rule is the only relaxation. Each of these effects re-syncs local state to something that changes outside the component. The rule wants the state derived during render, which changes when the value is committed or what the first render shows, so it would be a behaviour change. Per file: - `components/ResourceQuotasField.tsx` re-syncs the inputs after the form value is reset from outside. - `components/SourceWidget.tsx` re-syncs the selected option when the value loads asynchronously in edit forms. - `components/command-palette/command-palette.tsx` resets query and level whenever the palette closes. - `components/command-palette/use-keyboard-nav.ts` moves the highlight back to the top when the item list changes. - `hooks/use-is-mac.ts` would return `true` on the first render on macOS with a lazy initial state. Today it returns `false`. - `lib/tenant-provider.tsx` falls back to an existing tenant once the tenant list has loaded. The eslint config itself is unchanged, and `no-explicit-any` stays on everywhere. `pnpm typecheck` and `pnpm test` (442 tests) pass at every commit. After the file moves the bundle has the same chunk set, and only the entry chunk differs, by reordered and renamed minified functions. Three `react-hooks/exhaustive-deps` warnings from main are left as they are, because fixing them changes when the effects run. One is in `SourceWidget.tsx`, where the effect reads `options` but depends only on `value`. The other two come from `ads = adList?.items \|\| []` in `use-command-items.tsx`, which gives two `useMemo` hooks a new dependency on every render. Things I did not do here: - The step has no `--max-warnings`, so a new warning will not fail CI. Once the three warnings are fixed, `--max-warnings 0` can go in. - `SourceField.tsx` and `SourceWidget.tsx` each declare the same small `SourceOptionSchema` interface. I kept them local since the two widgets share nothing else. - The job name stays `Typecheck and test`, so a required check with that name keeps matching. - `pnpm -r lint` only reaches `apps/console`. `packages/{k8s-client,ui,types}` have no `lint` script, same as on main. Closes #3911 Closes #4566 ### Screenshots No visual change: types, module layout and CI only. ### Downstream repositories The diff touches only the console workspace and `ui-test.yaml`. The console has no downstream copy since `cozystack-ui` was archived, and no trigger map entry matches. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note ci(dashboard): the console's eslint now runs in the UI test job, and the console passes it with no errors. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * Improvements * Tenant selection now uses available tenant namespaces, restores a saved selection when possible, and falls back to the root tenant or the first available tenant. * Tenant options are sorted, and the selected tenant is kept in sync with the available choices. * Chores * Automated checks now include linting alongside type checks and unit tests. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-05 14:11:31 +03:00
IvanHunters	94abf6940b	feat(kubernetes): let kubevirt-csi detach without the HotplugVolumes gate (#4451 ) ## What this PR does This lets kubevirt-csi detach volumes without the deprecated KubeVirt `HotplugVolumes` gate. The default stays as it is, so nothing changes at runtime yet. Part of #4032. The driver got a switch upstream for exactly this, and I bumped to it. kubevirt/csi-driver#382 added `--enable-vmi-hotplug-fallback`, merged as cb4770e4e475. With it set to false, the driver no longer calls the VMI-level `removevolume` on a VMI owned by a VM. virt-api rejects that call once `HotplugVolumes` is off. Our wrapper now passes the flag to upstream and no longer has its own hot-unplug path. `unpublishHotplugVolume` was a backport of kubevirt/csi-driver#184, and the previous pin already contained that change. Upstream detaches from the VMI when the VM is gone, then waits for the VMI status to drop the volume before it returns. So the wrapper's re-check never found the volume attached. Adding the ownership check there would have been dead code. Closes #4450 The chart leaves the flag at its default of true and does not render it. The driver image in the chart is stamped only at release preparation, so between releases main runs the previous release's binary, which does not know the flag. Go's `flag` package exits on an unknown flag. Rendering `=true` would crash-loop the CSI controller for no effect. The right value depends on the KubeVirt CR, not on the tenant, so the comment next to `HotplugVolumes` in `kubevirt-cr.yaml` points at the controller Deployment. The follow-up PR drops the gate and renders the flag as false. Nothing changes on the node side. Main already pins f5de10d39556, and cb4770e4e475 is that commit plus kubevirt/csi-driver#382. A Go unit test runs an unpublish through the wrapper during a teardown, with the flag on, off with a VM-owned VMI, and off with an orphaned VMI. It checks that the flag reaches upstream and that only upstream makes the VMI-level call. Upstream's own tests cover the ownership rule. A second test holds that an NFS volume only releases the node from its CiliumNetworkPolicy and never reaches the upstream hot-unplug. A helm-unittest pins that the chart renders no such argument. I haven't built the image or run e2e. Volume attach and detach on a tenant cluster in e2e is the real check. ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(kubernetes): kubevirt-csi supports --enable-vmi-hotplug-fallback, so it can detach volumes without the KubeVirt HotplugVolumes feature gate. The flag stays true for now and behaviour is unchanged. ```	2026-10-05 10:46:12 +03:00
Aleksei Sviridkin	f32e0693c1	ci(dashboard): run the console's eslint in the UI test job The console defines a lint script next to typecheck and test, but no workflow ran it, so new findings landed unseen next to the old ones. The backlog is cleared down to three react-hooks/exhaustive-deps warnings. eslint exits zero on warnings alone, so the lint script caps them at that count: the step gates from its first run, and a new warning fails it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:09 +03:00
Aleksei Sviridkin	4d3796e41d	refactor(dashboard): mark the deliberate setState calls inside effects Each of these effects re-syncs local state to something that changes outside the component. The rule's suggested rewrite derives the state during render instead, which changes when the new value is committed and what the first render shows, so each site carries a scoped disable with its reason rather than a rewrite. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:08 +03:00
Aleksei Sviridkin	be9b148e6c	refactor(dashboard): move non-component exports out of component files Vite's fast refresh can only hot-swap a module that exports nothing but components; a file that also exports a hook, a helper or a registry object forces a full page reload on every edit to it. Hooks, context objects and the RJSF registry now live in their own modules, and the components stay where they were. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:08 +03:00
Aleksei Sviridkin	f1281e6176	refactor(dashboard): drop the unused ad binding in OverviewTab The tab never reads it; the prop stays in the interface because the detail page passes the same props to every tab. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:07 +03:00
Aleksei Sviridkin	8298c43e05	refactor(dashboard): type command palette instances as K8sResource client.list was called without a type argument, so its items came back as unknown and were cast to any to read metadata.name. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:07 +03:00
Aleksei Sviridkin	05ff14dbc1	refactor(dashboard): type the RJSF templates and source fields instead of any RJSF's generic defaults are any, and the templates restated them, so every schema read went untyped. The templates now default to unknown and FormContextType, and the source widgets read option schemas through a small local interface. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:07 +03:00
Aleksei Sviridkin	def917a5d5	refactor(dashboard): type the backup form pages instead of any The console's own rules forbid any, and eslint flags every one of these. The create pages only read a handful of reference fields, so a local interface per page covers them; the rest of the spec still passes through from RJSF untouched. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:43:06 +03:00
Aleksei Sviridkin	800bb0369d	feat(redis)!: refuse external access without TLS at admission A Redis release with external: true and TLS off publishes a plaintext endpoint outside the cluster, and nothing stopped a tenant from asking for one. The chart never infers TLS from external, because enabling it moves Redis to a TLS-only listener, so external: true without an explicit tls.enabled: true is exactly that case. A ValidatingAdmissionPolicy on the Redis kind now refuses it with a message naming the release and a remedy that works. TLS is fixed when a Redis instance is created: the operator's RedisFailover schema refuses a change, but the Redis API accepts it. A spec's tls.enabled is therefore a request, not the instance's state, and cannot prove on UPDATE that an instance has TLS; trusting it would let a tenant flip it, expose the instance, and remove it again. So external access can be enabled only at creation, where tls.enabled does decide the instance, and any UPDATE turning it on is refused. The RedisFailover schema does not exist before the first reconcile, so a release admitted as external with TLS could have TLS dropped before the instance is created. tls.enabled is therefore frozen while external access stays on. Withdrawing external access in the same edit lifts the freeze, which leaves a tenant a way to unblock an instance whose TLS was flipped earlier, and the callers trusted by the tenant host policy are exempt so an administrator can repair one in place. Other edits to an instance that is already external pass, so releases exposed in plaintext before the policy keep working. Postgres is left out on purpose. CNPG pins ssl=on, and its default pg_hba accepts plaintext from any client that asks, whatever the chart's tls.enabled says; that field only decides whether the external hostname goes into the certificate. A policy keyed on it would close nothing. The decisions are tested by compiling the shipped template with the apiserver's own CEL compiler, since a structural chart test cannot show what the policy admits. BREAKING CHANGE: creating a Redis with external: true and without tls.enabled: true, which applied before, is now refused, including a GitOps re-apply of such a manifest into a fresh cluster. Turning external on for an existing Redis is refused as well, so exposing an instance means creating a new one with TLS, and tls.enabled can no longer be changed on an instance with external access. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-03 01:42:57 +03:00
Aleksei Sviridkin	89a1fe52d9	build(multus): say why the image build runs no tests of the patched package The other patched Go images now test the packages their patches touch in the builder stage. The one package the multus patch touches, pkg/multus, creates network namespaces in its tests, which a build container is not allowed to do, so that image keeps building without a test step. The Dockerfile records the reason, so the absence reads as a decision and not as an omission. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:01:02 +03:00
Aleksei Sviridkin	f51be95430	build(objectstorage-controller): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Each package is tested from its own directory rather than as a ./... path of the root module, because internal/ is a module of its own and the root module cannot name ./internal/runtime. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:01:02 +03:00
Aleksei Sviridkin	070d235d1c	build(kube-ovn): run the tests of the packages the patches touch The image build ran one test by name, the one shipped next to the migration patch, and go build never compiles a _test.go, so the rest of the tests in the packages the patches change never ran. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. It replaces the hand-picked go test line: the shipped test lives in pkg/controller, which a patch touches, so the loop runs it with the rest of that package. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:01:01 +03:00
Aleksei Sviridkin	59c3174b44	build(kamaji): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:01:00 +03:00
Aleksei Sviridkin	0b00b69267	build(capi-providers-cpprovider): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:01:00 +03:00
Aleksei Sviridkin	1ffc8580a4	build(keycloak-operator): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:01:00 +03:00
Aleksei Sviridkin	e2757ba87b	build(redis-operator): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Two of the files the patches edit are upstream integration tests behind the integration build tag, which go test compiles none of, so each directory is also vetted with that tag. Doing it per derived directory rather than for the one directory those tests live in today keeps a later patch to a tagged test elsewhere covered. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:00:59 +03:00
Aleksei Sviridkin	0a606815ab	build(kilo): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:00:59 +03:00
Aleksei Sviridkin	a3a7d66bdf	build(linstor): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:00:58 +03:00
Aleksei Sviridkin	a73b91e232	build(bucket): run the tests of the packages the patches touch The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:00:58 +03:00
Aleksei Sviridkin	293a780938	build(opensearch-operator): run the tests of the packages the patches touch The image build ran go test on two package trees listed by hand, and go build never compiles a _test.go. A patch that reaches a directory nobody listed brings tests that are proven once, by hand, when the patch is cut, and nothing notices them failing after the next version bump. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. It replaces the hand-picked go test line, keeping its -race flag: two lines testing the same packages is one too many. The hand-picked line also covered pkg/reconcilers/util, which no patch touches. controllers/ is left out. The stall-tracing patch touches it, and its suite starts an envtest control plane from etcd and kube-apiserver binaries the builder does not have, so testing it fails the build. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 22:00:57 +03:00
Aleksei Sviridkin	bf03c84711	chore(deps): move to the Kubernetes 1.37 libraries without the apimachinery fork (#4473 ) ## What this PR does This moves the root module's `k8s.io/` from v0.35 to v0.37.1 and controller-runtime from v0.23 to v0.25.1, and drops the `k8s.io/apimachinery` fork on the way. Supersedes #3001, #4467 and #4472; those were a stack of the same commits. The four commits are meant to be read one by one, and each builds and passes the tests on its own: 1. Plain watches stop getting the initial-events-end bookmark.* The apiserver defaults `sendInitialEvents=true` onto any watch with an empty or `0` resourceVersion and leaves bookmarks off, and our aggregated storages took that flag alone as a WatchList request. From `k8s.io/apiserver` v0.36 the watch handler compiled into cozystack-api panics on that bookmark, so `kubectl get -w` on any cozystack-api resource would die after the bump. The marker now needs both flags, the same condition the watch cache uses. There are tests per storage and one through the real handler chain. 2. The apimachinery fork goes. Every apps kind shares the `Application` Go type, and stock `Scheme.ConvertToVersion` picks the first kind registered for it. The fork patched that; the apps group now encodes through a serializer that keeps the kind the object carries. A handler-chain test covers create, get, list, update, watch, apply and table for two kinds and fails without it. 3. MutatingAdmissionPolicy stays out of admission. From v0.36 the plugin is on by default and refuses every write until it has synced `mutatingadmissionpolicies`, which cozystack-api has no RBAC to list and which clusters before 1.36 do not serve. Without this, every create, update and delete would fail after the bump. It is also re-applied in `Complete`, since `--disable-admission-plugins` replaces the list. 4. The bump itself, with the regenerated clients, openapi and strategy CRDs. Commits 2 and 4 are where the review effort goes; 1 and 3 are small. Known changes in behaviour: - Without the fork, the row objects of a Table requested with `includeObject=Object` carry the first registered app kind. kubectl asks for that only with `--sort-by`, and it sorts on fields, not on the kind. Nothing at this layer can fix it, because the handler converts through `APIGroupInfo.Scheme`, a concrete `runtime.Scheme`. Admission webhooks and ValidatingAdmissionPolicy still see the right kind. - The alpha PodSpec field `workloadRef` became `schedulingGroup` upstream. Once the new strategy CRDs are applied, the apiserver prunes `workloadRef` from stored strategies it serves, and their next write erases it. It only ever worked behind an alpha feature gate. The branch is rebased onto the Renovate rollup (#4447), with `go mod tidy` and codegen rerun on top of it. ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note chore(deps): cozystack-api and the controllers build against the Kubernetes 1.37 libraries without the apimachinery fork. Plain watches no longer receive the WatchList initial-events-end bookmark, and objects in Table rows requested with includeObject=Object may carry another app's kind. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit New Features * Application API responses now preserve the requested resource kind across operations, including lists and watches. * Backup strategy resources support additional Kubernetes pod settings, including probe protocols, volume ownership options, and scheduling groups. * Backup strategy schemas include updated Kubernetes pod fields and descriptions. * Bug Fixes * Initial-event watch bookmarks are sent only when requested; plain watches continue without an ending bookmark. * Mutating Admission Policy is disabled by default, including when admission plugin options are customized. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-02 21:45:51 +03:00
Aleksei Sviridkin	9809b22280	fix(vm-disk): report DataVolume readiness through WorkloadsReady (#4380 ) ## What this PR does Nothing on a vm-disk application tells whether CDI ever finished populating the disk. Its WorkloadMonitor has no pods to count, and `Ready` follows the Helm release. So an import or clone that never completes can leave a green application over a disk nobody can use. cozystack/cozystack#2908 is one way to get there: a clone into a StorageClass that cannot place a replica where the source data sits. The expensive part there was the silence, not the stuck clone. The disk state was in the DataVolume phase all along, and nothing read it. With this change the WorkloadMonitor in cozystack-controller reads it. The monitor lists the DataVolumes its selector matches and reports not operational while any of them is in flight, Failed, Unknown or has no phase. An upload counts as in flight until it completes. That surfaces on the application as `WorkloadsReady=False` with reason `DataVolumeNotReady` and a message that names the DataVolume and its phase, for example `DataVolume vm-disk-x is ImportInProgress`. `Ready` still follows the HelmRelease, as for any other kind (`convertHelmReleaseToApplication` in `pkg/registry/apps/application/rest.go`), and for a vm-disk it no longer waits on the DataVolume at all. So the disk state has to be visible outside `Ready`, and this PR puts it where operators look by default: - `kubectl get` on any application kind gets a `WORKLOADS` column next to `READY`. It shows the `WorkloadsReady` status, or `<none>` when the application has no WorkloadMonitor. - The console badge turns amber and reads `DataVolumeNotReady` while `Ready` is True and `WorkloadsReady` is not True for that reason. A failing `Ready` still wins. The console shows only the disk case: other kinds need a separate decision, because a stopped VM legitimately has no pod and reports `WorkloadsReady=False` too. A Helm readiness gate on the DataVolume would be the obvious alternative, and it is not used on purpose. helm-controller runs that wait inside its reconcile, so an importing disk would hold one of the shard's worker slots for the whole install window, and a disk stuck for good would keep taking the slot back. cozystack/cozystack#4145 has the details. The monitor costs no slot, keeps watching after install, and covers existing disks without anyone writing to them. Without any gate, the install would still wait on the DataVolume. helm-controller's default poller falls back to a resource's `Ready` condition for kinds it has no rule for (`checkReadyCondition` in fluxcd/cli-utils `pkg/kstatus/status/status.go`), so a vm-disk install could hold a worker slot for the whole install timeout. The ApplicationDefinition therefore gets a `healthCheckExprs` entry for DataVolume with `current: "true"`, the same idiom postgres-rd uses for its ScaledObject. With it Helm never waits on the DataVolume. The entry is pinned by a helm-unittest in `packages/system/vm-disk-rd`, which gets a `test` target for it. Release settings reach a HelmRelease only when the application is created or updated, so an existing disk gets the entry on its next update. The reconciler that syncs definitions onto existing releases copies chartRef, valuesFrom and labels only. That affects any kind with release settings, so it is filed as cozystack/cozystack#4409 and not fixed here. `WorkloadsReady` covers every disk right away, since the monitor needs no write to the application. CDI copies the DataVolume labels onto the PVC it creates and makes the DataVolume its controller, so the monitor also publishes a `pvc-<name>` Workload for the disk. That Workload reads not operational while its DataVolume is not ready, even though the PVC is already Bound, so `kubectl get workloads` agrees with `WorkloadsReady`. While the DataVolumes cannot be read it keeps the verdict it has, like the monitor. The WorkloadMonitor CRD now requires at least one selector label. An empty selector would match every Pod, PVC, Service and DataVolume in the namespace. No chart renders one. When an application has several monitors that are not operational and only some of them name a cause, `WorkloadsReady` carries the named messages and reason only. A vm-disk has one monitor, so it does not hit this. ### Which phases count as ready Settled phases pass: `Succeeded`, `PendingPopulation`, `WaitForFirstConsumer`, `Paused`. A disk with `bindImmediately: false` on a `WaitForFirstConsumer` class rests in `PendingPopulation` or `WaitForFirstConsumer` until a VM consumes it. A multi-stage import rests in `Paused` until its next checkpoint, and the vm-disk chart renders no checkpoint source. These report ready while holding no data, because that is where such a disk is supposed to sit. `UploadReady` is not one of them. CDI enters it when the upload server pod is running and ready, and leaves it for `Succeeded` only when that pod succeeds (`updateStatusPhase` in `pkg/controller/datavolume/upload-controller.go` at v1.66.1), so the one phase covers a disk nobody has uploaded to and a half-uploaded one. A `source: upload` disk therefore reports `WorkloadsReady=False` from creation until `virtctl image-upload` completes. A phase the controller does not know also reads as ready, so a CDI release that adds or renames a phase cannot turn working disks red. The cost goes the other way: a renamed in-flight phase silently reads as settled. The list wants a re-read whenever the CDI version moves. It matches the named phases of the CDI v1.66.1 API, the version vendored in `packages/system/kubevirt-cdi-operator`. ### Clusters without CDI CDI comes with the IaaS bundle, while cozystack-controller runs everywhere, and CDI can be installed while the controller is running. A watch on a kind whose CRD is absent never syncs, and then the whole controller does not start. So the watch is not set up at startup. A runnable adds the DataVolume watch once the kind is served. It waits for the first reconcile, because only from then on does a new watch start at once, and then checks discovery right away and every 30s after that. It installs the DataVolume reader only after the DataVolume informer has synced within a timeout, because a reconcile context has no deadline and a List on an informer that never syncs would block every WorkloadMonitor reconcile. The watch source is started once: a controller-runtime `Kind` source adds an event handler to the informer and never removes it, so on a timeout the next check waits on the source it already has instead of starting another one. RBAC needs no change: the controller's ClusterRole already has read on everything. When the DataVolumes cannot be read, because the reader is not installed yet or the List fails, the monitor still publishes its replica counts and keeps its last DataVolume verdict, and a List error is returned so the request is retried and recorded as a `DataVolumesUnavailable` Warning event on the monitor. The status of a monitor without DataVolumes is not affected by a failing List. A disk whose verdict is already recorded does not read as ready in the window after a controller restart. A monitor with no recorded verdict yet, which is every monitor right after the upgrade that brings this change and any disk created right after a restart, reads ready until the DataVolume watch has synced. Because the watch starts on the first reconcile rather than on the next discovery tick, that window is the time to sync the DataVolume informer plus at most one 5s requeue, not a 30s poll interval. The window is not zero, since a monitor reconciled in it is published before its DataVolumes can be read. From then on, a reconcile that runs before the watch has synced is requeued after a few seconds, so the DataVolume events that queue it are not lost. A pass that changes nothing in the monitor status does not write it, so that requeue sends no status writes to the API server. Once the reader is installed, every monitor that holds a DataVolume verdict is queued once more. So a verdict whose DataVolume was deleted while the controller was down is recomputed then, not at the periodic resync. ### Tests The table test in `workloadmonitor_controller_test.go` covers the named phases of CDI v1.66.1, plus Failed, Unknown, an empty phase and a missing status. A second table checks the message and reason for an importing, a failed, a phaseless and a populated disk, and a third test lists two stuck DataVolumes in both orders and expects one message. It also covers a DataVolume outside the selector, the kind not being served, a List error that still publishes the replica counts, and a missing reader that keeps the last verdict. The watch tests check that the watch starts on the first reconcile and not before it. A source that never syncs leaves no reader and is retried on the same source, with the reconciles in between still requeued. A List failure records the Warning event, the sync window records none, and an unchanged pass writes no status. A manager that stops during the wait gets the cancellation back, and a reconcile that runs while the watch syncs is requeued without an error. A table test covers the PVC Workload: importing, populated, a PVC with no DataVolume controller or a controller of another kind or group, and the unread cases. A helm-unittest in `packages/system/cozystack-controller` pins the selector's `minProperties`. `rest_table_test.go` pins the `WORKLOADS` column, `rest_workloads_test.go` the condition message and reason, and `status.test.ts` the console badge, including a halted VM that stays green. In `hack/e2e-chainsaw/vminstance` the existing VMDisk scenarios now wait for the DataVolume to be Ready, then for `Ready` and `WorkloadsReady` on the VMDisk itself. That replaces the wait on the HelmRelease. Those asserts would pass even if the monitor ignored DataVolumes, so a new scenario imports from an `.invalid` URL. It first waits for the importer to have failed, which CDI reports on the DataVolume as `Running=False` with reason `Error`, and then for `Ready=True` with `WorkloadsReady=False` and reason `DataVolumeNotReady` on the VMDisk. Without the first wait the VMDisk assert passed in about a second, on a DataVolume that had no phase yet. The e2e runs on a fresh install only, so nothing here exercises an existing release across an upgrade. ### What it does not do #2908 lists possible fix directions, and none of them is here. The cross-class clone still never completes, it just stops being invisible. Keep the issue open. While the DataVolume watch syncs, every monitor is still requeued. Only the ones whose selector reaches a DataVolume need it, but the controller cannot tell which those are before it can read DataVolumes. Closes #4449 ### Screenshots There is no screenshot: I had no running console to take one from. The badge change is pinned by the vitest cases in `packages/system/dashboard/images/console/apps/console/src/lib/status.test.ts`: a VMDisk whose `WorkloadsReady` has reason `DataVolumeNotReady` goes amber, and a halted VMInstance with the generic reason stays green. ### Downstream repositories I walked the trigger map against the diff. cozystack/terraform-provider-cozystack is affected: its `wait_for_ready` reads only `Ready`, so on a `cozystack_vmdisk` it returns before the disk is populated once this lands. That needs a decision on the provider side, so it is an issue there. The release-time bot regenerates `packages/apps/vm-disk/README.md` into the website's virtualization pages, and `vm-disk` is in the `VMS` list that generator reads, so the website needs no follow-up. The ApplicationDefinition keeps its kind, plural, prefix and path. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/59 - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(vm-disk): report whether CDI finished populating a disk in the WorkloadsReady condition, which names the stuck DataVolume and its phase. kubectl get shows WorkloadsReady in a new WORKLOADS column for every application kind, and the console badge turns amber while a disk is not populated, so a disk whose import or clone never completes no longer looks healthy. Ready of a new or updated vm-disk no longer waits for the DataVolume, so wait on WorkloadsReady to know the disk is populated. ```	2026-10-02 19:47:02 +03:00
IvanHunters	11910e0b2c	fix(kubernetes-nodes): refuse a resources.cpu that cannot be a core count (#4386 ) ## What this PR does A worker pool's CPU value could land in the VM as something KubeVirt cannot accept. `resources.cpu` is typed as a quantity, but on the kubevirt substrate the template wrote it unchanged into `domain.cpu.cores`, which is `uint32` in KubeVirt and an integer in the KubevirtMachineTemplate CRD. With `resources.cpu: 500m` the chart rendered `cores: 500m`. That string was truthy, so it also counted as explicit sizing and dropped the pool's `instanceType` from the VM, and then the API server refused the template anyway. The render now decodes the value as YAML, the same way `cores: <raw>` is decoded, and stops unless the result is a whole number from 1 to 4294967295. `500m`, `"1.5"` and `"1e10"` now fail at render time with an error naming the field, before the value reaches the VM template or the instancetype decision. `"0"` and `"-2"` were already refused on `main` and still are. Only inputs that could not produce a working pool are refused. Every value that rendered a pool before renders the same bytes now, so the content-hashed template keeps its name and nothing rolls. That includes `2`, `"2.0"`, `"+2"` and `"1e1"`, and also two odd cases I kept on purpose because pools may already carry them. An unquoted `0` is falsy, so the pool renders without `domain.cpu` and is sized by its instancetype, as on `main`. A leading-zero value such as `"010"` is still emitted as written and keeps the reading it gets on `main` today, where YAML treats it as octal. I checked both against a render of `main`, and the tests pin the template names that `main` produces for them. This is where kubernetes-nodes and vm-instance now differ. Since #4405, vm-instance refuses a leading zero and an unquoted `0`, and this chart does not. Stored worker pools can carry these values, because migration 56 copies `resources` verbatim, so refusing them here without a migration would fail those pools' releases on their next upgrade. #4554 tracks the migration and the refusal after it. The proxmox substrate is not checked. It converts the quantity to millicores and rounds up to whole cores, so `500m` is a valid size there and becomes one core. The field description now says what the value means on each substrate. Memory is left alone. It goes to `domain.memory.guest`, which is a `resource.Quantity`, so a quantity string is valid there. The issue suggested coercing to an integer, as vm-instance did before #4405. I didn't, because `int64` turns `500m` into `0` without an error. The new helm-unittest suite covers each input form, both preserved cases, the upper bound, the instancetype case and the proxmox path. Two neighbouring descriptions still say only "CPU available.": `nodeGroups[].resources.cpu` in `packages/extra/computeplane` and in the parent `kubernetes` chart. computeplane passes `resources` straight into this chart, so the new check applies to its pools as well. I left both alone to keep this change to one chart. Closes #4150 ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kubernetes-nodes): on the kubevirt substrate, a worker pool whose resources.cpu cannot be a whole vCPU core count (for example 500m, 1.5 or a value above 4294967295) now fails at render time with a clear error, instead of rendering a VM template that is refused later or dropping the pool's instanceType. Values that rendered a working pool before render unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * KubeVirt CPU settings now accept whole-core values from 1 to 4,294,967,295. Fractional, millicore, negative, and larger values are rejected; an unquoted `0` is treated as unset. * Proxmox rounds fractional and millicore CPU values up to whole cores. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-02 15:57:11 +03:00
IvanHunters	339be8d42d	fix(vm-default-images,kubernetes-worker-image,vm-disk): wait for golden images to import and refuse cloning a failed one (#4392 ) ## What this PR does Both golden image catalogs, `vm-default-images` and `kubernetes-worker-image`, went Ready before any image had imported. Each one renders a DataVolume per image with immediate bind, so every import starts at install, but neither PackageSource had `healthCheckExprs`. The HelmRelease went Ready as soon as helm applied the DataVolumes, and an image that failed or stalled stayed green. This PR makes both releases wait for the imports. Each component's `install:` block now has a DataVolume health check. `current` holds only on `Succeeded`, and `failed` holds on `Failed`. Any other phase, `Unknown` included, matches neither, and flux keeps waiting. A DataVolume with no `status` key makes the CEL evaluation error out. Flux reports that as Unknown and keeps waiting too. The other settled phases are safe to treat as not ready for these charts. `PendingPopulation` and `WaitForFirstConsumer` are where a volume waits for a consumer, and immediate bind removes that wait. `UploadReady` and `Paused` belong to upload and multi-stage sources, and both charts render only `source.http`. There is no per-component timeout, so I did not set one. The window is the operator-wide `--helmrelease-install-timeout`, 10m by default. The operator sets `RetryOnFailure` on both install and upgrade, so a download that outlasts the window is retried as an upgrade until the import finishes. Nothing uninstalls the DataVolumes in between, and the import keeps running. One image that never imports keeps its whole catalog not Ready until the image imports or its entry is removed. There is no time limit, and I kept it that way on purpose. A group/kind health check cannot tell one image from another, so it cannot be made per image. A per-component timeout would be an API change for one opt-in package. Both charts' `values.yaml` document this. Nothing on `main` lists either package in `dependsOn`, and the images that did import stay usable in the meantime. Tenant worker pools look up each golden directly and never wait on the catalog's status. The price is a helm-controller slot. The operator labels the generated HelmRelease with `cozystack.io/package`, plus `cozystack.io/privileged` when privileged. The flux-shard-operator acts only on HelmReleases that already carry `sharding.fluxcd.io/key`. So both catalogs go to the system helm-controller, which picks up releases without that key and runs with `--concurrent=5`. A DataVolume in `Failed` ends each attempt at once and costs next to nothing. An import that stalls without reaching `Failed` is the expensive case, for example an importer that keeps retrying an unreachable URL. Each attempt waits out the full window, and the next one starts 30s later. So one of those five slots stays busy about 95% of the time for as long as the image is stuck. #4145 describes the same cost on the shard controllers. The status change alone does not protect a vm-disk, because vm-disk clones from a golden without checking it. A golden still importing is fine, since CDI holds the clone until the source is populated. A golden whose import has Failed is never populated, so the clone would wait forever. vm-disk now refuses to render a new disk on a Failed golden and names the image. This matches the check the worker pools already have. A disk that already exists is exempt, so a golden that fails later does not break that disk's upgrades. The check fires only when CDI sets Failed, and at CDI v1.66.1 an import reaches Failed only on a checksum mismatch. The goldens carry no checksum, so a broken URL keeps the importer retrying and the new disk still waits, with only the catalog's not-Ready status pointing at the image. #4564 tracks that gap. A golden that does not exist at all is a separate gap, tracked in #4548. Tests pin the expressions for both catalogs. A helm-unittest case checks them on the rendered PackageSource. A Go test builds the HelmRelease spec from the shipped file and checks that they arrive with the poller wait strategy. The repo has no harness that evaluates flux CEL, so the phase behaviour above is argued from flux's evaluator and not exercised by a test. The vm-disk check has helm-unittest cases for a Failed, a Succeeded, and a still-importing golden, for a render with no cluster, and for an existing disk. Closes #4169 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(vm-default-images, kubernetes-worker-image): the golden image catalogs now report Ready only after their images have finished importing, and a failed or stalled import keeps the catalog not Ready. A vm-disk created from a golden image whose DataVolume is in phase Failed is now refused with an error naming the image, instead of waiting forever on a clone that cannot complete. ```	2026-10-02 15:44:28 +03:00
Aleksei Sviridkin	10a6cf2a0b	test(tests): make absence-only helm-unittest cases fail when they check nothing (#4664 ) ## What this PR does Twenty-one helm-unittest cases asserted only that something is absent from a document, and could pass without ever looking at the document they name. This PR makes each of them fail when that happens. 1. Thirteen cases picked their document by position: mongodb (4), nats certmanager (3), opensearch (3), qdrant certmanager (2), seaweedfs cluster_scoped_names (1). When a template gains or reorders a document, the index points at some other object that also lacks the field, and the case stays green. These now use `documentSelector` on `kind` or `metadata.name`, like the earlier harbor conversion. 2. Seven cases had no selector at all, on templates wrapped in a top-level `if`: kubernetes cilium_gateway_alpn, mongodb restore, linstor-scheduler admission, seaweedfs s3_traffic_distribution and three in vertical-pod-autoscaler. If a values change turns the condition off, the render is empty and the absence check passes on nothing. A `documentSelector` turns an empty render into `document not found`. 3. The redis null-namespace case checked that the in-cluster name is present, but never that the external name is absent. It now asserts that neither `my-redis.` nor `my-redis.example.com` (the suite-level host that `_namespace: null` must override) is in the SAN list. The `kubernetes cluster` entry in the #3634 catalog no longer applies, since `packages/apps/kubernetes/tests/cluster_test.yaml` has been removed from main. I did not add the `my-redis.redis.` check that #4621 mentions, because the per-engine SAN it refers to has not landed. Each case was checked by mutation. I broke the template in a way the old case could not see, then ran the chart's `make test` with the new test file and with the one from main. For group 1 I removed the guard and put an extra document in front of the target. For group 2 I turned off the top-level condition. For group 3 I let the empty-host branch emit the bare SAN. Every time the main version stayed green and the new one went red, and the fixed cases were the only extra failures. All templates are back to their committed bytes. Follow-ups, not in this PR: - The same absence-only shape has appeared in more cases since #3634 was filed: mongodb `tls_test.yaml` (8 cases), mariadb `tls_test.yaml` (6), redis `tls_test.yaml` (2 more), kubernetes `kamaji_managed_args_test.yaml` (2), and one each in kafka `dashboard-resourcemap_test.yaml`, cert-manager-issuers `solver_test.yaml` and seaweedfs `volume_resize_hook_test.yaml`. They need a separate issue. The four platform `bundles_proxy_protocol` cases are #3864, handled in #3929. Closes #3634 Closes #4621 ### Screenshots Not applicable, test-only change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit Tests * Updated chart validation to identify rendered resources by their type or name rather than relying on document order, improving consistency across Kubernetes applications. * Expanded Redis TLS checks to verify that certificates omit generic and external hostnames when no namespace data is provided. * Kept existing expected behavior checks for gateway, database, messaging, search, storage, and autoscaling resources. <!-- end of auto-generated comment: release notes by coderabbit.ai -->	2026-10-02 14:08:07 +03:00
Aleksei Sviridkin	9f72f0bda9	feat(api): refuse a WorkloadMonitor with an empty selector An empty selector matches every Pod, PVC, Service and DataVolume in the namespace, so one such monitor goes non-operational on any import there. No chart renders one, but the schema marked the field required without requiring any label, and the API accepted it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 14:02:37 +03:00
Aleksei Sviridkin	6170c1615f	feat(dashboard): turn the status badge amber while a disk is not populated The console badge followed Ready alone, and Ready follows the Helm release. A vm-disk install does not wait on its DataVolume, so a disk CDI never populated was painted green. The badge now shows the WorkloadsReady condition while Ready is True and WorkloadsReady is not True with reason DataVolumeNotReady. Other causes of WorkloadsReady=False keep the badge as it was: a stopped VM has no pod and reports not operational on purpose, and turning it amber for every kind needs its own decision. A failing Ready still wins, because it is the more specific cause. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 14:02:34 +03:00
Aleksei Sviridkin	99f3d3b2b7	feat(controller): tag a DataVolume cause with a condition reason WorkloadsReady=False covers different situations: a stopped VM has no pod and its monitor reports it not operational on purpose, while a disk whose DataVolume never populates is broken. A consumer that wants to react to one and not the other had only the message text to go by. The WorkloadMonitor status gains a Reason next to Message, set to DataVolumeNotReady by the DataVolume check, and the Application API carries it into the WorkloadsReady condition in place of the generic reason whenever a monitor that is not operational names one. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 14:02:33 +03:00
Aleksei Sviridkin	48ece79ffc	feat(controller): name the DataVolume that keeps a disk not ready For a vm-disk, WorkloadsReady is the one condition that reports whether CDI populated the disk, and its generic "One or more workloads are not operational" gives an operator nothing to act on: a disk has no workloads, and the stuck DataVolume and its phase are exactly what they need to know. The WorkloadMonitor status gains a message that the controller fills with each selected DataVolume that is not ready and its phase, and the Application API uses the messages of the monitors that are not operational in WorkloadsReady instead of the generic text whenever one of them names a cause. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 14:02:32 +03:00
Aleksei Sviridkin	f4b1f08adb	fix(vm-disk): keep the Helm install from waiting on the DataVolume Helm's default poller falls back to a resource's Ready condition for kinds it has no rule for, so a vm-disk install could wait on the DataVolume's Ready and hold a helm-controller worker for the whole install timeout while an import runs or never completes. Whether CDI populated the disk is reported by the WorkloadMonitor, so the release now treats the DataVolume as current and Ready of a vm-disk no longer depends on the import. Release settings reach a HelmRelease when its application is created or updated; the reconciler that syncs definitions onto existing releases copies the chart reference, valuesFrom and labels only, so an existing disk picks this up on its next update. The vm-disk-rd package had no unit test target; it gets one, and a test that pins the entry. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la>	2026-10-02 14:01:13 +03:00

1 2 3 4 5 ...