Replaces the now-obsolete "Package admission rejects tenant-* in
gateway.attachedNamespaces" e2e (the VAP it pinned was dropped in
the inheritance refactor) and adds two new e2e cases that cover
the end-to-end inheritance contract:
* "Package admission accepts gateway.attachedNamespaces with
tenant-* entries" — regression guard against a future refactor
re-introducing the gate; the hijack surface is now handled by
Layers 4/5/7 (Tenant.spec.host VAP, namespace label immutability
VAP, HTTPRoute hostname VAP), not by entries in attachedNamespaces.
* "child tenant without explicit gateway inherits _namespace.gateway
from a Gateway-owning parent" — chart-level lockstep: a parent
tenant with gateway=true produces both the namespace label and the
cozystack-values Secret pointing at itself, and a child Tenant
under that parent with the gateway field unset receives the
parent's name in both places. Plus a negative assertion that the
child does NOT get its own gateway HelmRelease.
* "child tenant's HTTPRoute attaches to parent's Gateway via
inheritance label" — full cross-namespace attach: parent owns
the Gateway, child inherits, an HTTPRoute in the child namespace
pointing at parent's Gateway reaches Accepted=True once Cilium
evaluates the label-based allowedRoutes selector. Route hostname
uses the child's derived apex so Layer 7 admits it.
The Gateway-Programmed wait is intentionally omitted from the
last test — Programmed depends on the LB allocator that ships with
the e2e cluster, while Accepted is a pure spec evaluation on the
parent Gateway's allowedRoutes selector and is the property the
inheritance refactor is meant to deliver.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## Summary
Bump the `cozyvalues-gen` binary pinned in
`.github/workflows/pre-commit.yml` from `v1.4.0` to `v1.5.0`.
`v1.5.0` (cozystack/cozyvalues-gen#24) adds support for the `@immutable`
annotation. Without this bump, pre-commit re-runs `make generate` with
the old `v1.4.0` binary on every PR; the old binary silently ignores `##
@immutable` directives and regenerates `values.schema.json` / embedded
`openAPISchema` without the corresponding `x-kubernetes-validations`
entries — which then shows up as committed-vs-regenerated drift on any
PR that tries to use the new annotation.
## No-op for main today
No chart on main currently uses `@immutable` (zero hits for `##
@immutable` under `packages/apps/*/values.yaml`), so this bump
regenerates byte-identical output. The change is purely lifting the
floor for downstream PRs that will start using the annotation.
## Unblocks
- cozystack/cozystack#2639 — apply `@immutable` to `storageClass` across
every stateful Cozystack app (currently fails pre-commit on the v1.4.0
pin).
## Release info
- v1.5.0 release notes:
https://github.com/cozystack/cozyvalues-gen/releases/tag/v1.5.0
- Tag signed by F20B7CCBF00A5CAE8F9F8B8C7988329FDF395282 (Aleksey
Sviridkin)
- Goreleaser uploaded `cozyvalues-gen-linux-amd64.tar.gz` (the artefact
this workflow downloads) along with the rest of the OS/arch matrix.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated development tooling dependencies for build process
improvements.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2730?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace the 'Inheritance from parent Gateway' known-limitation entry
(which described inheritance as deferred) with:
* A top-of-README 'Inheritance: when to opt in for a separate
Gateway' section explaining when a tenant owns its Gateway vs
inherits, and how the namespace.cozystack.io/gateway label is
written by apps/tenant chart and patched onto cozy-* by the
controller.
* DNS-01 cert-mode docs spell out SAN expansion for child apexes:
the wildcard Certificate now carries <child-apex> + *.<child-apex>
per inheriting tenant, with a per-child *.<child-apex> listener
on the parent Gateway. Documents the DNS-provider-permission
requirement for deeply-nested children (ACME challenge writes
TXT under each apex zone, so zone delegation or wide-scope
provider creds are needed for grandchild apexes).
* New known-limitation entries: 64-listener cap on the parent
Gateway under inheritance (with the high-fanout escape via
tenant.spec.gateway=true), and Cilium sharing-key port-collision
(multiple per-tenant Gateways still each take their own LB IP
until ListenerSet ships in Cilium).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Removes two gates that prevented child tenants from attaching to a
parent's publishing Gateway:
1. Render-time fail() in templates/gateway-hostname-policy.yaml that
refused to render the chart when _cluster.gateway-attached-
namespaces contained any tenant-* entry.
2. cozystack-gateway-attached-namespaces-policy VAP that enforced
the same rule at Package CR admission time.
The original rationale — 'a tenant in attachedNamespaces can hijack
the publishing tenant's hostnames' — assumed a static-name
allowedRoutes whitelist where adding tenant-* literally widened the
attach surface for arbitrary hostnames. Under the inheritance
refactor (Stage 1/3/5) the attach surface is governed by the
namespace.cozystack.io/gateway label selector, not by entries in
AttachedNamespaces; and hostname hijack is independently blocked by
three other layers:
* Layer 4 — cozystack-tenant-host-policy VAP on Tenant.spec.host
* Layer 5 — cozystack-namespace-host-label-policy VAP on the
namespace label
* Layer 7 — cozystack-route-hostname-policy VAP on HTTPRoute and
TLSRoute hostnames
Those three defend against the hijack vector regardless of what
sits in attachedNamespaces, and they apply uniformly to every
tenant (inheriting or owning a Gateway). The dropped gates added
no extra defence, while their presence broke the inheritance flow
kvaps reasonably asks for in the May 25 review.
Doc count for gateway-hostname-policy.yaml drops from 8 to 6 (one
VAP + one Binding removed); helm-unittest fixtures updated to
match, with a regression-guard test asserting the removed VAP name
no longer appears in any rendered document.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
In DNS-01 mode, the parent's wildcard Certificate now covers every
inheriting child tenant's apex (`<child-apex>` + `*.<child-apex>`
as additional SANs) and the parent Gateway grows one wildcard HTTPS
listener per child apex referencing the same Certificate. Without
the expansion, a child route hostname like `harbor.alice.example.org`
fails to match the parent's listeners — `*.example.org` is a single-
label wildcard and `alice.example.org` is two labels deep — and the
TLS handshake fails the SNI check because the parent cert never
listed the child apex.
The controller discovers inheriting children by listing namespaces
labelled `namespace.cozystack.io/gateway = <tgw.Namespace>` (the
existing inheritance contract from Stage 1/3) and reading their
`namespace.cozystack.io/host` label, which apps/tenant chart writes
to the child apex. Children with no host label are skipped
(defensive); a child host equal to the parent apex is deduplicated
(cert-manager rejects duplicate dnsNames). HTTP-01 mode is
unchanged — its per-listener / per-cert flow already handles child
hostnames without any wildcard.
Listener naming: `https-child-<first-label>-<8-hex>`, where the hex
suffix is sha256(child-apex)[:8] so two children whose first DNS
label collides produce distinct listener names.
Cap: Gateway API hard-caps spec.listeners at 64 entries. The parent
uses 3 slots (http + https + https-apex) leaving ~61 wildcard
children per parent. Operators whose subtree fans out beyond that
must run the high-fanout branch on its own Gateway via
tenant.spec.gateway=true.
Tests (TDD red→green):
* TestReconcile_DNS01WildcardCertCoversInheritingChildApexes pins
the SAN list contains both `<child-apex>` and `*.<child-apex>`.
* TestReconcile_DNS01WildcardCertDeduplicatesChildApexEqualToParent
pins the dedupe path (child host == parent apex).
* TestReconcile_HTTP01WildcardCertNeverRendered pins the inverse:
HTTP-01 mode never renders a wildcard Certificate.
* TestReconcile_DNS01GatewayHasListenerPerChildApex pins the
listener-side expansion (one `*.<child-apex>` listener per
inheriting tenant, all referencing the parent's wildcard cert).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Gateway's HTTPS / TLS-passthrough listeners now match attached
namespaces by the namespace.cozystack.io/gateway label keyed on the
TenantGateway's own namespace, instead of a static
kubernetes.io/metadata.name In [list] whitelist. This restores
inheritance parity with the legacy ingress flow: child tenants of a
Gateway-owning ancestor publish through the ancestor's Gateway by
default — without their own LB IP / Certificate, and without
operator intervention to add each child by name to platform values.
Two writers populate the label:
* apps/tenant chart namespace.yaml — each tenant namespace gets a
label pointing at the nearest ancestor that owns a Gateway (self
if owning, inherited otherwise). The tenant Helm chart is the
source of truth for tenant-tree labels; the controller never
touches them (annotation-gated GC).
* cozystack-controller (ensureNamespaceLabels) — patches the label
onto every namespace in tgw.Spec.AttachedNamespaces, so cozy-*
system namespaces (cert-manager, monitoring, harbor, …) reach the
publishing Gateway alongside the tenant tree. Tracks its own
writes via the cozystack.io/gateway-attached-by annotation; on a
reconcile where an entry is removed from AttachedNamespaces, the
label is garbage-collected (annotation present + namespace no
longer expected → strip both label and annotation).
Also adds the gateway-attach label to tenant-root itself via
cozystack-basics/templates/tenant-root.yaml when
_cluster.gateway-enabled is true, so the Gateway in tenant-root's
own namespace passes its own selector for the controller-owned
http→https redirect HTTPRoute.
Tests (TDD, red→green):
* TestReconcile_HTTPSListenerUsesGatewayLabelSelector pins the new
MatchLabels shape on HTTPS listeners.
* TestReconcile_LabelsAttachedNamespaces pins the patch path: every
namespace in spec.AttachedNamespaces (plus tgw.Namespace) carries
namespace.cozystack.io/gateway=<owner> after reconcile.
* TestReconcile_LabelGCRemovesDroppedAttachedNamespaces pins the GC
path: removing an entry between reconciles strips the label.
* TestReconcile_LabelGCDoesNotStripHelmOwnedLabels pins the safety
contract: labels lacking the controller annotation (Helm-owned
tenant labels) are left alone.
* tenant-root_test.yaml adds the positive case for the conditional
label render under _cluster.gateway-enabled=true.
The HTTP listener (port 80) keeps the narrower static-name selector
— it allows only the tenant namespace plus cozy-cert-manager (for
HTTP-01 ACME challenges) and must NOT accept routes from app
namespaces or child tenants, since unattended routes on the HTTP
listener serve plaintext and leak credentials.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Mirror the existing _namespace.ingress inheritance for Gateway API
attachment. A child tenant under a Gateway-owning ancestor publishes
its routes through that ancestor's Gateway by default, without burning
a dedicated LB IP / Certificate per child.
Resolution rule (tenant.gatewayEffective):
* gateway=true explicit → own Gateway (LB IP + cert burned)
* gateway=false explicit → parked (no inherit, no own)
* gateway unset → inherit $parentNamespace.gateway
Removes the auto-default for derived-apex tenants — the previous shape
gave every derived-apex child its own Gateway / IP / cert, which scales
linearly with tenant depth and produces ~N×K LB IPs on a tree of N
top-level × K derived-apex tenants. Inheritance collapses that to one
LB IP per Gateway-owning ancestor.
Custom-apex tenants (host explicitly set to a non-derived value) now
fall through to inheritance instead of auto-firing a separate Gateway.
The previous auto-fire conflated 'custom apex' with 'want public
exposure' and would have silently failed TLS termination anyway (the
ancestor's cert does not cover an unrelated apex). The new rule:
explicit gateway=true is the only way to ask for a separate Gateway.
The namespace.cozystack.io/gateway label and _namespace.gateway in the
rendered cozystack-values Secret now both carry the owner-tenant name
— could be self (when this tenant owns one) or any ancestor up the
chain. Apps in the tenant render HTTPRoutes whose parentRefs.namespace
points at that owner.
helm-unittest pinned for the HelmRelease render side. The
cross-namespace attach end of the flow (ReferenceGrants, listener /
SAN expansion on the owner Gateway) lands separately in the
TenantGateway controller.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
v1.5.0 adds the @immutable annotation support
(cozystack/cozyvalues-gen#24). Without the bump, pre-commit re-runs
make generate with the v1.4.0 binary, silently strips any
x-kubernetes-validations entries that downstream PRs add to
values.schema.json / embedded openAPISchema, and reports drift against
the committed artefacts.
This is a no-op for the current main: no chart currently uses
@immutable, so regeneration with v1.5.0 produces identical output to
v1.4.0. Unblocks cozystack/cozystack#2639 (apply @immutable to
storageClass across stateful apps).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds Gateway API equivalents for the two ingress endpoints in the
monitoring chart — Grafana (web UI) and Alerta (alert console) — gated
on the same _cluster.gateway-enabled flag the keycloak / dashboard /
cozystack-api templates already use.
* Grafana's ingress is configured inline via Grafana CRD's spec.ingress
block; that block is now wrapped with {{ if ne gateway-enabled true }}
and a standalone HTTPRoute attaching to the tenant Gateway is rendered
in the alternate branch.
* Alerta uses a standalone Ingress object; the same Ingress↔HTTPRoute
swap is wired through an if/else.
Both routes attach to Gateway cozystack in the publishing namespace
(default tenant-root), matching the convention established by the rest
of the system charts.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Gateway API source-flag toggle (gatewayAPI=true) already enabled
gateway-httproute discovery. cozystack ships TLSRoute attachments for
SSL-passthrough services (kubernetes control plane API, kubevirt CDI
uploadproxy, kubevirt VM exportproxy) — without gateway-tlsroute in the
external-dns sources list those hostnames are never picked up for DNS
record management.
Adds gateway-tlsroute under the same conditional and updates the value
description / schema / README to reflect that the toggle now covers both
route kinds.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Security model section split into three groups by who they defend
against, anchoring the framing in the apps.cozystack.io/* tenant API
surface (tenants do not write Gateway API resources directly, so most
of the seven layers are not protecting against tenant-user input):
- tenant-user-input gates: Layer 4 (cozystack-tenant-host-policy)
via cozystack-api admission. Tenant.spec.host is the user-supplied
field that surfaces as a security boundary.
- defense-in-depth: Layers 1, 2, 5, 6, 7. Cover bugs in
cozystack-controller / Flux, supply-chain compromise of an app
chart, confused-deputy admin mistakes. Fail-closed.
- admin-against-themselves: Layer 3 (cozystack-gateway-attached-
namespaces-policy). Catches kubectl edit on the platform Package.
Layer 7 wording fix: drop the implication that a tenant user with
HTTPRoute RBAC could exploit the cross-apex hostname surface — that
RBAC is not granted in Cozystack. Reframed as defense-in-depth
against an app chart bug or supply-chain compromise.
External IP allocation section reframed as admin-side concern:
allocator (MetalLB / Cilium LB-IPAM / robotlb / externalIPs) is
configured at the platform layer, tenant API stays
mechanism-agnostic. Per-Service IP uniqueness is the allocator's
responsibility — same shape as for any LoadBalancer Service.
Inheritance Known limitations entry: spell out both upstream gaps
(Gateway API 64-listener cap, Cilium sharing-key inactive on
443/TCP via cilium#42756) so readers understand why per-tenant
Gateway is the only scalable shape today.
publishing.gateway.attachedNamespaces -> gateway.attachedNamespaces
fix: the values block is at top-level, not under publishing.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two related fixes for the gap where the tenant chart's gateway HelmRelease materialises but child apps (harbor, bucket) silently fall back to ingress-nginx:
- packages/apps/tenant/templates/namespace.yaml: switch the
_namespace.gateway resolution from raw .Values.gateway to
tenant.gatewayEffective. This is the same helper consulted by
gateway.yaml (HelmRelease render), so per-tenant Gateway
materialisation and child-app HTTPRoute / Ingress selection now
stay in lockstep — including the auto-default for derived-apex
tenants.
- packages/system/cozystack-basics/templates/cozystack-values-secret.yaml:
add a conditional gateway field to the tenant-root cozystack-values
Secret. cozystack-basics is the only writer of _namespace.* for
tenant-root (apps/tenant chart skips tenant-root via the existing
ne 'tenant-root' gate). Without this line, tenant-root apps read
_namespace.gateway as empty and route to ingress even when the
platform's gateway.enabled flag is on. Conditional on
_cluster.gateway-enabled='true' so non-Gateway clusters keep the
empty value.
Test fixture in cozystack-basics/tests/cozystack-values-secret_test.yaml
pins the conditional and the regression-guard that the other four
_namespace keys (etcd, ingress, monitoring, seaweedfs) stay
unconditional.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Stops `ControllerPublishVolume` from silently reporting success when the
volume is in VM spec but the hotplug attach never actually completed.
### Root cause
Upstream's fast path (`EnsureVolumeAvailableVM`) checks only
`vm.Spec.Template.Spec.Volumes`. If a prior Publish wrote that entry but
timed out waiting for VMI Ready (e.g. the infra PVC is stuck
`ClaimPending` because LINSTOR cannot autoplace), the next Publish
short-circuits to success. external-attacher sets `Attached=true`, but
QEMU was never sent `device_add`. The tenant kubelet's NodeStageVolume
then fails with:
```
rpc error: code = Unknown desc = couldn't find device by serial id
```
Observed in production. Sequence from kubevirt-csi-driver controller
logs:
```
16:53:13 /ControllerPublishVolume called
16:55:13 volume failed to be ready in time (2m), context deadline exceeded
16:55:14 /ControllerPublishVolume called (retry)
16:55:14 Volume X already attached to VM Y - skipping hot-plug
16:55:14 /ControllerPublishVolume returned with response: {}
```
VMI volumeStatus during that window: `phase=Pending message="PVC is in
phase ClaimPending"`. VolumeAttachment then goes `Attached=true`; tenant
kubelet surfaces "couldn't find device by serial id".
### Fix
After upstream Publish returns success for a hotplug volume, the wrapper
re-reads VMI status and checks `Phase == VolumeReady` for this
DataVolume. If it is not Ready (or not present), the RPC returns
`codes.Unavailable` with the upstream reason, so external-attacher keeps
retrying instead of declaring the volume attached. The underlying
provisioning failure surfaces at the CSI layer instead of becoming a
confusing kubelet-side "device not found by serial id".
This is a wrapper-side workaround for a defect in upstream
`kubevirt.io/csi-driver`; the structural fix belongs upstream (tighten
the fast path to require VMI Ready) and will be filed separately. The
wrapper guard is safe to keep as defense-in-depth even after the
upstream fix lands.
Relates to #2634.
### Release note
```release-note
fix(csi): kubevirt-csi-driver Publish now verifies VMI Ready before reporting attached, surfacing stuck-PVC failures at the CSI layer instead of as "couldn't find device by serial id" on the tenant kubelet
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved hotplug volume publishing: non‑NFS volumes are now verified
on the VM before being marked published; if the volume is missing or not
ready, publishing returns unavailable, improving reliability for
hotplugged storage.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2659?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes routing in `WrappedControllerService` so RWX **Block** volumes
(KubeVirt live-migration disks) are no longer treated as NFS during
unpublish and expand.
### Root cause
`ControllerUnpublishVolume` selected the NFS cleanup branch based on
`hasRWXAccessMode(pvc)` alone. RWX Block matched and went down the NFS
path — which just deletes a `CiliumNetworkPolicy`. For a block volume
that CNP does not exist, so `removeCNPOwnerReference` returns
successfully without doing anything, and the function reports success
without ever calling upstream's hotplug detach. The DataVolume entry
stays in `VM.spec.template.spec.volumes` permanently. When the same PVC
later attaches to a different VM, linstor-csi's anti-split-brain check
sees pods from two different VMs referencing the same RWX block PVC and
blocks the attach with:
```
RWX block volume <ns>/<pvc> is being used by pods from different VMs (<vm-a> and <vm-b>);
this is not supported - RWX block volumes with allow-two-primaries are only for live migration of a single VM
```
`ControllerPublishVolume` was unaffected because it routes on the
explicit `nfsVolumeKey` from `VolumeContext`.
`ControllerUnpublishVolume` does not receive `VolumeContext` and has to
infer from the PVC, and the previous inference was incomplete.
### Fix
`isNFSVolume(pvc)` now requires `ReadWriteMany` access mode **and**
`VolumeMode == Filesystem` (always set by our NFS `CreateVolume` path).
RWX Block correctly delegates to upstream `ControllerUnpublishVolume`,
which removes the entry from VM spec via `RemoveVolumeFromVM`. The same
predicate replaces the previous access-mode check in
`ControllerExpandVolume`, which was also clearing
`NodeExpansionRequired` for RWX Block — block volumes still need in-VM
disk expansion.
Fixes#2634.
### Release note
```release-note
fix(csi): route RWX Block volumes in kubevirt-csi-driver to upstream hotplug detach so VM spec entries are cleaned up on unpublish
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* More accurate detection of network-attached (NFS) vs block volumes,
preventing incorrect cleanup of block volumes.
* NFS volumes continue to receive special cleanup; non-NFS volumes now
follow upstream cleanup paths.
* **Refactor**
* Node-side expansion logic refined to base decisions on improved NFS
classification (RWX + filesystem), reducing expansion errors.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2658?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
On release builds IMAGE_TAG = github.ref_name = vX.Y.Z, and Makefiles
that pass v$(COZYSTACK_VERSION) as the versioned arg evaluate to the
same vX.Y.Z on the tag commit. The macro then emitted
`--tag :vX.Y.Z --tag :vX.Y.Z --tag :latest`. docker buildx dedupes
internally so there is no functional impact, but the build logs carried
the redundant flag for every affected package.
Replace the empty-string guard with filter-out so the versioned tag is
also skipped when it would duplicate IMAGE_TAG.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The include pointed at ../../../scripts/common-envs.mk, but that
directory does not exist — only hack/common-envs.mk does. flux-plunger
is not part of the root `make build`, so the broken path went unnoticed
until the new image-tags macro made the recipe depend on it.
Without the include, `make -C packages/system/flux-plunger image` fails
immediately with "No such file or directory".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The image-talos recipe chains skopeo copy with `;` in a single shell.
Without `set -e`, the recipe's exit status is the last command's, so a
failed copy in the middle of the chain can be masked by a later
successful command — or worse, by a false-branch `if`/`fi` (which exits
0 and would silently hide a push failure on PR builds where both
PUBLISH_* flags are 0).
Prepend `set -e;` to abort the recipe on the first failure. Also quote
"$SRC" defensively.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Cluster-member workloads (OVN raft, LINSTOR, cert-manager) fail when
replicas start at different times due to per-node image-pull stagger.
This test pre-pulls the images that those charts reference onto every
node before the platform HelmReleases install, so all replicas can
start with images already cached.
Mechanics:
- Source images directly from the rendered charts (kubeovn, linstor,
cert-manager) via `helm template` + a `yq` filter that walks every
PodSpec-shaped object and emits each container's image. Scopes the
output to images the kubelet actually pulls — skips configmap fields
and CRD examples that happen to contain an `image:` key. Each render
is staged through a tmp file so a helm-template failure trips `set -e`
cleanly without needing `pipefail` (cozytest.sh runs @test bodies
under /bin/sh, which is dash on Ubuntu CI), and without capturing
rendered YAML into a shell variable that `set -x` would expand and
balloon the CI trace.
- The pre-pull DaemonSet is the hard part: distroless images
(cert-manager and friends) ship no shell and no /bin/sleep, so the
obvious `command: ["sleep", "infinity"]` fails with exec ENOENT.
Stage a statically-linked busybox:musl into a shared emptyDir from
an initContainer; every prepull container then execs /shared/sleep
regardless of what the image itself ships. The glibc busybox tag is
unusable here — it needs /lib64/ld-linux-x86-64.so.2 which distroless
lacks. Loop is written in POSIX shell (no bash arrays) for the same
/bin/sh-is-dash reason.
Verified locally on kind v1.33.1 single-node: every chart-referenced
image is held Ready under the DaemonSet before the platform HelmRelease
applies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Install Cozystack failed because grafana-operator pod looped in
CrashLoopBackOff under concurrent helm-install load (Flux v2.8
helm-controller dispatching every system HR in parallel + kstatus
walking all applied resources). The chain that fell over:
grafana-operator pod liveness probe :8081/healthz fails 3x in 30s
-> monitoring-rd dependency 'cozy-grafana-operator/grafana-operator'
-> tenant-rd dependency 'cozy-system/monitoring-rd'
-> cozystack-basics dependency 'cozy-system/tenant-rd'
(cozystack-basics creates the cozy-public ns)
-> kubevirt-cdi RoleBinding cdi-clone-dv: namespace
"cozy-public" not found
-> vm-disk-rd, dashboard, flux-plunger all wait, install times out
Pod log shows the manager starts at T+0, controllers spin up workers,
folder/dashboard/datasource sync completes by T+10s — then SIGTERM at
T+27s. Liveness probe is the upstream chart default (delay=0s,
timeout=1s, period=10s, failureThreshold=3) which gives ~30s grace,
not enough on a fresh cluster where controller-runtime's HTTP probe
server is still binding while kubelet starts polling immediately.
Same shape as the kubevirt-cdi-operator commit already in this branch
(37b06e21). Mirror the fix:
- Edit deployment.yaml on the vendored chart to add a startupProbe
(30 x 5s = 150s grace, /healthz on :8081). Once it passes once,
liveness/readiness keep the existing strict config.
- Stash the change as patches/startupProbe.diff and re-apply it from
the Makefile update target after the curl|tar upstream fetch, so
the next chart bump does not silently regress.
The patch uses package-relative paths (a/charts/...) so `patch -p1`
from the package dir resolves correctly — repo-relative paths would
need patch's fuzzy fallback which is unreliable across distros.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 40949c80b1)
cdi-operator's controller-runtime starts the health probe HTTP server
(:8444) only after the leader election lease is acquired. On a fresh
cluster the lease can take 15–20 s, exceeding the liveness probe's
5s + 3×10s grace under load. Kubelet kills the container before the
probe can succeed; the pod loops in CrashLoopBackOff, CDI never
reconciles its CR, datavolumes.cdi.kubevirt.io stays unavailable, and
every downstream test that creates a DataVolume (tenant kubernetes
worker VMs, vm-instance) fails.
Add a startupProbe that hits /healthz with 30×5s grace (150 s total).
Once it succeeds once, liveness/readiness take over with their existing
strict config. No effect on healthy clusters; only relaxes the
ramp-up window.
Implemented as a patches/*.diff applied from the Makefile update
target, mirroring the fluxcd-operator pattern. Survives the next
upstream regen; if upstream restructures the area around the
readinessProbe/resources boundary the patch will fail loudly so the
maintainer notices.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 8f5d45e0eb)
The helm-unittest assertions added in the previous commit cover the
structural shape of the sed lines (they must exist in
preKubeadmCommands). They do not catch the failure mode this PR exists
to fix: a future containerd release renames the registry plugin
section, the sed silently stops matching, and per-registry mirroring
under /etc/containerd/certs.d quietly breaks.
Add a bash test that:
1. Drift-checks the script's copy of the two sed lines against the
verbatim strings embedded in templates/cluster.yaml, so the test
cannot diverge from the template without failing.
2. Runs both seds against minimal fixtures of containerd 1.x and 2.x
default configs (tests/fixtures/containerd-{1,2}x.toml) and asserts
that config_path is rewritten exactly once in each. If containerd
3.x renames the section again, the fixture stops matching, the test
fails, and CI surfaces the regression instead of letting it ship
silently.
Wired into the package's `make test` target, which is already invoked
in CI alongside helm-unittest.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add helm-unittest assertions for both sed lines in preKubeadmCommands so
that a future regression to a single-regex form, which silently fails
against containerd 2.x where the plugin section was renamed and quoted
with single quotes by default, is caught at template render.
Add a Helm comment above the two seds explaining the rename from
io.containerd.grpc.v1.cri (containerd 1.x) to io.containerd.cri.v1.images
(containerd 2.x), so future readers understand why both lines exist and
that the wrong-version one is intentionally a no-op on each image
version.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
This adds support for containerd 2.x CRI config_path configuration, as the CRI plugin section was renamed from io.containerd.grpc.v1.cri to io.containerd.cri.v1.images. It also supports both single and double quotes around plugin names.
Assisted-By: Gemini <noreply@google.com>
Signed-off-by: Elaugaste <elaugaste@gmail.com>
## What this PR does
Fixes the OpenSSF Scorecard workflow added in #2720.
The initial workflow used floating major-version tags (`@v2`, `@v4`,
`@v3`). `ossf/scorecard-action` does not publish a floating `v2` tag —
only specific patch releases — so the first scheduled run failed:
> Unable to resolve action `ossf/scorecard-action@v2`, unable to find
version `v2`
This PR pins every action by its full commit SHA with a trailing comment
showing the human-readable version. This also satisfies the
**Pinned-Dependencies** Scorecard check itself.
- `actions/checkout@de0fac2` (v6.0.2)
- `ossf/scorecard-action@4eaacf0` (v2.4.3)
- `actions/upload-artifact@043fb46` (v7.0.1)
- `github/codeql-action/upload-sarif@03e4368` (v3)
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated CI/CD infrastructure dependency versions to maintain security
and stability.
**Note:** This release contains no user-facing changes.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2721?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The initial Scorecard workflow used floating major-version tags
(@v2, @v4, @v3). ossf/scorecard-action does not publish a floating
v2 tag — only specific patch releases — so the first scheduled run
failed with `Unable to resolve action ossf/scorecard-action@v2`.
Pin every action by its full commit SHA with a trailing comment
showing the human-readable version. This also satisfies the
`Pinned-Dependencies` Scorecard check itself.
- actions/checkout@de0fac2 (v6.0.2)
- ossf/scorecard-action@4eaacf0 (v2.4.3)
- actions/upload-artifact@043fb46 (v7.0.1)
- github/codeql-action/upload-sarif@03e4368 (v3)
Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io>
## What this PR does
Adds a GitHub Actions workflow that runs the [OpenSSF
Scorecard](https://github.com/ossf/scorecard) on this repository.
### Schedule
- Weekly on Mondays at 06:00 UTC
- On branch protection rule changes
- On pushes to `main`
- Manual dispatch
### Where results go
- **Public**
[`scorecard.dev`](https://scorecard.dev/viewer/?uri=github.com/cozystack/cozystack)
— transparency for adopters and reviewers (CNCF DD).
- **Code-scanning dashboard** (Security tab) — for in-team triage of
individual findings.
- SARIF artifact retained for 5 days for local inspection.
### Token permissions
Default `read-all`, with the minimum required scopes elevated only
inside the analysis job:
- `security-events: write` — upload to code scanning
- `id-token: write` — publish to `scorecard.dev` via OIDC, no long-lived
secret
- `contents: read`, `actions: read`
### Why no README badge yet
This PR enables the workflow but **does not add a badge to the README**.
Once the first scans have completed and quick-win improvements (token
permissions across other workflows, dependency pinning in workflows,
etc.) have landed, a follow-up PR will add the badge.
### Context
- Tracked under the project's CNCF Incubation readiness work and the
in-progress security self-assessment.
- Companion to the OpenSSF Best Practices passing badge
(https://www.bestpractices.dev/en/projects/10177) — Scorecard adds
machine-verifiable, continuously-updated evidence; Best Practices
captures self-attested controls.
### Release note
\`\`\`release-note
NONE
\`\`\`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added automated security scanning to the development process to
continuously monitor codebase health and security posture.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2720?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Runs the OpenSSF Scorecard action on:
- weekly schedule (Mon 06:00 UTC)
- branch protection rule changes
- pushes to main
- manual dispatch
Publishes results to:
- the public OpenSSF Scorecard API (scorecard.dev) for transparency
- the GitHub code-scanning dashboard (Security tab) for triage
No README badge is added yet — the badge will be introduced in a
follow-up PR once the first scans have run and any quick-win
improvements (token permissions, dependency pinning) have landed.
Aligns with the project's CNCF Incubation readiness work and the
security self-assessment in progress.
Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io>
## What this PR does
Updates @lllamnyp's responsibility in `MAINTAINERS.md` from
`Etcd-operator Lead` to `Core Maintainer`.
The etcd-operator is hosted in the separate
[aenix-io/etcd-operator](https://github.com/aenix-io/etcd-operator)
repository — its leadership scope is tracked there. In
`cozystack/cozystack`, @lllamnyp's role is a project-level Core
Maintainer, which this change reflects.
### Release note
```release-note
NONE
```
## What this PR does
Expands the **Stepping Down / Emeritus Process** section in
`CONTRIBUTOR_LADDER.md`:
- Adds an explicit PR-based mechanism for moving a maintainer's row from
`Active` to `Emeritus` in `MAINTAINERS.md`.
- Adds a five-business-day technical offboarding checklist: GitHub team
membership, org role, direct collaborator entries, `CODEOWNERS`, Actions
secret rotation, internal channels.
- Defines a path for an Emeritus maintainer to return to active status.
This codifies the offboarding flow actually exercised in #2717 and
removes ambiguity for future maintainer lifecycle events. Useful
evidence for the CNCF Incubation Due Diligence review (Governance
section, maintainer-lifecycle criteria).
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated contributor documentation with details on the Emeritus
transition process, including offboarding procedures and reactivation
guidelines.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2718?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The etcd-operator lives in the aenix-io/etcd-operator repository, not
cozystack/cozystack. Listing it as a responsibility in this project's
MAINTAINERS.md was a leftover from the earlier sub-project layout.
Replace with the project-level "Core Maintainer" role, which reflects
the actual scope of @lllamnyp's work on this repository.
Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io>
Expands the Stepping Down / Emeritus Process section with:
- An explicit PR-based mechanism for recording an Emeritus transition.
- A five-business-day technical offboarding checklist (team membership,
org role, direct collaborator entries, CODEOWNERS, secret rotation,
internal channels).
- A defined path for an Emeritus maintainer to return to active status.
Reflects the actual offboarding flow exercised in #2717 and removes
ambiguity for future maintainer lifecycle events.
Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io>
## What this PR does
- Add `Active Maintainers` and `Emeritus Maintainers` sub-sections.
- Update @gecube affiliation to Independent (was Ænix).
- Update @lllamnyp affiliation to Ænix (was 3commas).
- Mark @nbykov0 as Independent (was Ænix).
- Move @aobort to Emeritus Maintainers.
- Move @klinch0 to Emeritus Maintainers.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated project maintainers information in the maintainers list.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2717?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Complete the strategy.backups.cozystack.io/Job driver as the generic,
app-agnostic counterpart to the Altinity driver: render the strategy
PodTemplateSpec, run it as a one-shot batch/v1.Job, poll its terminal
condition, and emit a Backup. Wire it into the existing BackupClass
dispatch via reconcileJob/reconcileJobRestore rather than a standalone
reconciler, matching the post-BackupClass backups API (the old draft
read BackupJob.Spec.StrategyRef/StorageRef, which no longer exist).
Template context follows the in-tree convention (.Application, .Release,
.Mode, .Parameters, .Backup); the JobSpec.Template doc and regenerated
CRD are updated to match. Restore re-renders the same template with
.Mode=restore; cross-namespace restore is unsupported, mirroring
Altinity.
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
add implementation of backup-strategy controller for etcd
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Cluster-scoped Etcd backup strategy CRD (S3-only) plus controller
support for BackupJob (snapshot → Backup) and destructive in-place
RestoreJob flows; snapshot metadata persisted and destination validation
enforced.
* **Documentation**
* Complete etcd backup/restore README and admin/tenant scenario guides.
* **Examples & Tests**
* End-to-end demo scripts (helpers, stepper, cleanup, runner), Bats e2e,
and extensive unit tests.
* **Chores**
* RBAC updates for etcd access, chart version bumped, demo env files
added to .gitignore.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2641?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
PostgreSQL server parameters like `max_connections` are natural
integers,
but the chart's schema declared them as `{type: string}` only. When
users
supply bare integers (e.g. `max_connections: 100`) — either directly in
Application CR specs or via the UI — Helm rejects the values at schema
validation time before template rendering, causing HelmRelease failures.
The Helm template already handles both types via `{{ $value | toString |
quote }}`,
so the fix is to teach the schema to accept both.
### Approach
The schema files (`values.schema.json` and the embedded `openAPISchema`
in `postgres-rd/cozyrds/postgres.yaml`) are generated from `values.yaml`
annotations via `cozyvalues-gen`. Hand-editing them does not survive
`make generate` — same regeneration trap that caught #2466 (issue
#2597).
`cozyvalues-gen` v1.4.0 (cozystack/cozyvalues-gen#25) adds an
`intOrString` type alias for map values that emits
`additionalProperties.anyOf: [integer, string]` plus the Kubernetes
`x-kubernetes-int-or-string: true` marker.
### Changes
- `packages/apps/postgres/values.yaml`: annotation flipped from
`@field {map[string]string}` to `@field {map[string]intOrString}`,
description updated to note integer values are accepted.
- Regenerated artifacts: `values.schema.json`, embedded `openAPISchema`
in `packages/system/postgres-rd/cozyrds/postgres.yaml`,
`packages/apps/postgres/README.md`, and
`api/apps/v1alpha1/postgresql/types.go`
(Parameters is now `map[string]intstr.IntOrString`).
- `packages/apps/postgres/templates/db.yaml`: `$dangerousParams` list
extended with `archive_cleanup_command` and `recovery_end_command`
(same RCE surface as `archive_command` / `restore_command`).
### Verification
- `helm template` with `max_connections: 200` (int) and
`max_connections: "200"`
(string) produce identical render output.
- Dangerous-param block list rejects `archive_cleanup_command` and
`recovery_end_command` with the expected `SECURITY:` error.
- `make test` in `packages/apps/postgres/`: 13/13 pass.
- `make generate` is idempotent — schema files don't drift on regen.
### Release note
```release-note
fix(postgres): accept integer values for postgresql.parameters to prevent HelmRelease failures when parameters are supplied as unquoted integers; also block archive_cleanup_command and recovery_end_command (RCE surface).
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* PostgreSQL parameters now accept integer or string values; integers
are automatically coerced to strings.
* **Security**
* Enhanced parameter filtering blocks additional dangerous settings:
archive_cleanup_command and recovery_end_command.
* **Documentation**
* Updated docs and charts to describe int-or-string parameter support
and revised blocked-parameter guidance.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2687?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review feedback from gemini-code-assist on
packages/apps/kubernetes/images/kubevirt-csi-driver/controller.go:562:
classify a PVC as NFS only when its VolumeMode is Filesystem (or nil,
which defaults to Filesystem), instead of "anything not Block". Future
VolumeMode values will no longer be silently routed to the NFS path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
WrappedControllerService.ControllerUnpublishVolume decided between NFS
cleanup and upstream hotplug detach using only hasRWXAccessMode(pvc).
RWX Block volumes (KubeVirt live-migration disks) match that check and
were routed to the NFS path, which only deletes a CiliumNetworkPolicy.
For a block volume that CNP does not exist, so removeCNPOwnerReference
returns successfully without doing anything, and the function reports
success without ever calling upstream's hotplug detach. The DataVolume
entry stays in VM.spec.template.spec.volumes; when the same PVC later
attaches to a different VM, linstor-csi's anti-split-brain check sees
pods from two different VMs referencing the same RWX block PVC and
rejects the attach permanently.
Introduce isNFSVolume(pvc) requiring both ReadWriteMany access mode and
VolumeMode=Filesystem (always true for PVCs provisioned via our NFS
CreateVolume path). RWX Block now delegates to upstream, which removes
the entry from VM spec via RemoveVolumeFromVM. The same predicate
replaces the access-mode check in ControllerExpandVolume - RWX Block
volumes still need in-VM disk expansion.
Fixes#2634
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The fallback branches in publishHotplugVolume silently returned the
upstream success response when the node ID failed to parse or the VMI
get call returned an error. external-attacher then set Attached=true
without VolumeReady proof, which recreates the false-positive attach
this wrapper exists to prevent.
Fail loudly instead:
- parseErr -> codes.Internal: external-attacher is expected to provide
a well-formed "namespace/name" node ID per the CSI contract; if it
does not, surface the contract violation rather than mask it.
- getErr -> codes.Unavailable: this is the same code returned for the
not-Ready and not-present cases just below, so the retry behavior on
the attacher side stays consistent. By the time we reach this branch
the upstream Publish call already succeeded, which means it just
looked the VMI up; a transient get error here is a race or API hiccup
and a retry is safe.
Also add a one-line comment on GetVirtualMachine: in the upstream
kubevirt.io/csi-driver client this method returns a
VirtualMachineInstance despite its name, so accessing
vmi.Status.VolumeStatus on the result is type-correct.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Upstream ControllerPublishVolume has a fast path that returns success
based only on the volume's presence in VM.spec.template.spec.volumes
(via EnsureVolumeAvailableVM). If a prior Publish wrote that entry but
the volume never became Ready in VMI status - e.g. the infra PVC is
stuck ClaimPending because LINSTOR cannot autoplace - the retry hits
the fast path and reports success. external-attacher marks the
VolumeAttachment Attached=true, but QEMU was never sent device_add, so
the tenant kubelet fails NodeStageVolume with:
rpc error: code = Unknown desc = couldn't find device by serial id
Wrap upstream Publish for hotplug volumes (non-NFS) with a post-call
check that the volume reached VolumeReady in VMI.Status.VolumeStatus.
When it has not, the RPC returns codes.Unavailable carrying the actual
reason (e.g. "PVC is in phase ClaimPending"), so external-attacher
keeps retrying and the underlying provisioning failure is visible at
the CSI layer rather than as a confusing "device not found by serial
id" on the tenant kubelet.
This is a wrapper-side workaround for an upstream defect in
kubevirt.io/csi-driver; the structural fix belongs upstream (tighten
the fast path to require VMI Ready) and will be filed separately.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## Problem
PR #2454 renamed `nodeGroups[*].ephemeralStorage` →
`nodeGroups[*].diskSize` with
a hard `{{ fail }}` guard. Any cluster whose HelmRelease still carries
the legacy
field cannot be reconciled by Flux at all — unrelated control-plane
changes and
MachineHealthCheck remediations are also blocked.
## Solution
Add platform migration 41 that runs as a pre-upgrade hook before any
chart
resources are applied. The migration walks every
`kuberneteses.apps.cozystack.io`
Application CR cluster-wide and renames `nodeGroups[*].ephemeralStorage`
to
`nodeGroups[*].diskSize`, preserving the user's value.
- Idempotent: a second run is a no-op (field is already absent)
- Best-effort: a failed patch is logged; the version stamp stays at 41
so the
migration retries on the next platform upgrade
- Bumps `migrations.targetVersion` 41 → 42
The chart-side guard remains in place with an updated error message that
directs
operators to the migration job logs if the field somehow reappears
post-upgrade.
## Testing
Verified on dev cluster: migration correctly renames `ephemeralStorage`
values
in existing Application CRs. All 123 helm unit tests pass.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Documentation**
* Platform migration 41 now automatically handles the `ephemeralStorage`
to `diskSize` field rename for Kubernetes node groups
* Migration is transparent during upgrade with no manual configuration
changes required
* Error messages updated with clearer troubleshooting guidance
* **Tests**
* Updated test assertions for migration error messages
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2688?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Companion regen for the type change in the previous commit. The
deepcopy generator emits a fresh import for intstr and updates the
inner make() call to match the new map value type. Caught by the
codegen-drift check on CI.
Generated by `make generate` at repo root (hack/update-codegen.sh).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
v1.4.0 adds the `intOrString` type alias used by the postgres package's
`postgresql.parameters` schema in this PR. CI must install v1.4.0 so the
"Verify generated code is up to date" check regenerates the same schema
shape (anyOf + x-kubernetes-int-or-string: true) rather than reverting
to {type: string}.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Ivan's original schema edits were correct in intent but lived in the
generated artifacts (values.schema.json and the embedded openAPISchema
in postgres-rd/cozyrds/postgres.yaml). `make generate` reverts them
because values.yaml's `@field {map[string]string}` annotation is the
source of truth -- same regeneration trap that caught #2466 (issue #2597).
cozyvalues-gen v1.4.0 (cozystack/cozyvalues-gen#25, merged 2026-05-21)
adds an `intOrString` type alias for map values that emits
`additionalProperties.anyOf: [integer, string]` plus
`x-kubernetes-int-or-string: true`. Move the change to source by
flipping the annotation, then regenerate:
- packages/apps/postgres/values.yaml: `@field {map[string]intOrString}`
with updated description noting integer values are accepted.
- packages/apps/postgres/values.schema.json: regenerated -- now carries
anyOf + x-kubernetes-int-or-string: true.
- packages/system/postgres-rd/cozyrds/postgres.yaml: embedded
openAPISchema regenerated to match.
- packages/apps/postgres/README.md: regenerated type column.
- api/apps/v1alpha1/postgresql/types.go: Parameters is now
`map[string]intstr.IntOrString`.
The result survives future regenerations. The template's
`{{ \$value | toString | quote }}` already coerces both inputs to
quoted strings, verified end-to-end with helm template against both
`max_connections: 100` (int) and `max_connections: "100"` (string) --
identical render output. Dangerous-param block list still rejects
archive_cleanup_command and recovery_end_command additions from the
prior commit.
Co-Authored-By: IvanHunters <xorokhotnikov@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin
