fix(vm-disk, kubernetes-nodes)!: refuse golden images whose import keeps failing (#4802)

## What this PR does

New clones can wait forever on a golden image whose import keeps
failing. CDI v1.66.1 retries ordinary HTTP failures without setting
phase `Failed`.

The guards now also refuse goldens with at least three importer restarts
unless they reached `Succeeded`. kubernetes-nodes and the picker require
`Running=False`; vm-disk refuses during running attempts too. Existing
vm-disks and already-cloned pools remain exempt. CDI never lowers the
restart count, so a healthy import with three early restarts stays
refusable until `Succeeded`, including while a recreated importer is
still starting.

An empty worker-pool StorageClass now resolves to the virtualization
default, then the GA Kubernetes default. Beta is a last fallback for the
class written explicitly onto a worker disk, not a CDI fallback. The
strict same-class policy stays.

This can refuse a previously accepted builtin pool on upgrade. The
affected pool has an empty StorageClass, no cloned disk yet, and a
golden on the Kubernetes default while another class is the
virtualization default. StorageClass is immutable: point
`osImage.builtin` at a catalog entry on the virtualization default or
switch to `osImage.factory`. Already-cloned pools keep their disk's
class.

A VMDisk's own HTTP import, golden-image health checks and imports that
hang without restarting remain outside this change.

Closes #4564

Refs #4059

### Screenshots

No UI changes.

### Downstream repositories

The platform values change only updates a comment. The application
schema change updates the StorageClass description without changing
fields, defaults or enums.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(vm-disk, kubernetes-nodes)!: refuse new clones of golden images whose import has not succeeded after at least three importer restarts. kubernetes-nodes and the image picker require the importer to be not running; vm-disk refuses during running attempts too. kubernetes-nodes now resolves an empty storageClass to CDI's virtualization default before the Kubernetes default. A builtin pool with an empty storageClass, no cloned disk yet, and a golden on a different class can now be refused. Point it at a catalog entry on the virtualization default or switch to osImage.factory. Already-cloned pools are unaffected.
```
This commit is contained in:
Aleksei Sviridkin authored and GitHub committed 2026-11-17 18:05:01 +03:00
commit 87f160d473
25 files changed
+809 -87

No files matched your search

+1 -1
View File
@@ -22,7 +22,7 @@ type ConfigSpec struct {
// +kubebuilder:default:=""
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="cluster is immutable"
Cluster string `json:"cluster"`
// StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.
// StorageClass for the worker node system disk. When empty, CDI picks the class: the one annotated `storageclass.kubevirt.io/is-default-virt-class`, else the cluster default. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: the chart refuses a pool whose class differs from its golden's, because across classes CDI either falls back to a host-assisted copy over the pod network (different CSI drivers or volume modes) or leaves a clone into the other class to the CSI driver. Leaving it empty makes the same comparison against the class CDI would pick. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.
// +kubebuilder:default:="replicated"
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="storageClass is immutable"
StorageClass string `json:"storageClass"`
@@ -13,10 +13,10 @@
# silently breaks every fresh install using osImage.builtin, and nothing catches it
# until a ~95-minute e2e run does.
#
# The StorageClass is the same trap from the other direction: CDI cannot
# CSI-clone across StorageClasses (it would silently fall back to a host-assisted
# copy over the pod network), so the render also rejects a group whose
# storageClass differs from its golden's. Bumping one file's default alone brings
# The StorageClass is the same trap from the other direction: the render also
# rejects a group whose storageClass differs from its golden's, because across
# classes CDI either falls back to a host-assisted copy over the pod network or
# leaves the clone to the CSI driver. Bumping one file's default alone brings
# that guard down on the default path.
#
# Both checks are cheap and exact, which is the point — they turn a slow, remote
@@ -70,8 +70,8 @@ CATALOG_VALUES="$REPO_ROOT/packages/system/kubernetes-worker-image/values.yaml"
echo "StorageClass mismatch on the default osImage.builtin path:" >&2
echo " packages/apps/kubernetes-nodes/values.yaml storageClass: $app_sc" >&2
echo " kubernetes-worker-image golden effective storageClass: $entry_sc" >&2
echo "CDI cannot CSI-clone across StorageClasses, so the tenant render" >&2
echo "rejects this outright. Keep the two defaults in step." >&2
echo "The tenant render refuses a pool whose storageClass differs from" >&2
echo "its golden's. Keep the two defaults in step." >&2
return 1
fi
}
@@ -91,6 +91,30 @@ spec:
(conditions[?type == 'WorkloadsReady']):
- status: "False"
reason: DataVolumeNotReady
# The golden image guard in the kubernetes-nodes chart and the image option
# list call an import failing once its importer has restarted three times and
# is not running, and the vm-disk guard once it has restarted three times,
# because CDI keeps an unreachable URL in progress instead of moving it to
# Failed. This pins that CDI does report both. The default kubelet back-off
# before the third restart adds up to 70s (10s, 20s, 40s), and the timeout
# leaves room for the importer's own runs on top of that. The count is
# compared as a string, because Chainsaw v0.2.15 mis-evaluates numeric
# literals; 'null' in the list makes an absent count, which CDI omits at
# zero, fail too.
- name: wait-importer-restarts
try:
- assert:
timeout: 4m
resource:
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: vm-disk-unreachable
status:
phase: ImportInProgress
(contains(['null', '0', '1', '2'], to_string(restartCount))): false
(conditions[?type == 'Running']):
- status: "False"
---
# VMDisk clone paths: source.image clones a golden image from cozy-public, and
# source.disk clones that disk in turn, so only the golden image downloads.
+3 -3
View File
@@ -23,9 +23,9 @@ The pool's Kubernetes minor (`version`) may lag the cluster's during an upgrade
### Common Parameters
| Name | Description | Type | Value |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------ |
| `storageClass` | StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`. | `string` | `replicated` |
| Name | Description | Type | Value |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------ |
| `storageClass` | StorageClass for the worker node system disk. When empty, CDI picks the class: the one annotated `storageclass.kubevirt.io/is-default-virt-class`, else the cluster default. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: the chart refuses a pool whose class differs from its golden's, because across classes CDI either falls back to a host-assisted copy over the pod network (different CSI drivers or volume modes) or leaves a clone into the other class to the CSI driver. Leaving it empty makes the same comparison against the class CDI would pick. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`. | `string` | `replicated` |
### Pool Sizing
@@ -441,18 +441,25 @@ still fails.
{{- end }}
{{- /*
Name of the cluster's default StorageClass, or the empty string when there is
none (and always under `helm template`, which has no cluster to read).
Name of the StorageClass CDI binds a kubevirt-content DataVolume on when it
names no class, or the empty string when there is none (and always under `helm
template`, which has no cluster to read). CDI takes a class annotated
storageclass.kubevirt.io/is-default-virt-class first and only then the
Kubernetes default (GetStorageClassByNameWithVirtFallback in CDI's
pkg/controller/common/util.go), and both the goldens and the worker disks are
kubevirt content.
An empty storageClass is a documented, schema-valid setting on both a worker pool
and a worker image catalog entry, and it means "the cluster default". Without
resolving it the golden-versus-pool StorageClass comparison simply skips whenever
either side is empty, which is the one corner where skipping is worst: CDI then
falls back to a host-assisted copy over the pod network, silently, and that copy
is the transfer the clone path exists to remove.
either side is empty, and lets a cross-class clone through unchecked.
Both the current annotation and its beta predecessor count, because clusters
provisioned years apart carry different ones and Kubernetes still honours both.
The beta predecessor of the Kubernetes annotation is read last, only when no
class carries the current one. CDI does not read it: with neither of the other
two defaults it creates no claim at all for a DataVolume that names no class
and no access modes (pvcFromStorage, ErrStorageClassNotFound). Read here, it
still gives a pool on such a cluster a class, written onto the worker disk
explicitly.
More than one class may carry the annotation at once. Kubernetes permits that --
it is the normal state midway through swapping a cluster's default -- and
@@ -463,16 +470,21 @@ that is actually in force. Timestamps are RFC3339 in UTC, so comparing them as
strings is comparing them chronologically.
*/ -}}
{{- define "kubernetes-nodes.defaultStorageClassName" -}}
{{- $classes := lookup "storage.k8s.io/v1" "StorageClass" "" "" -}}
{{- $classes := dig "items" (list) ((lookup "storage.k8s.io/v1" "StorageClass" "" "") | default dict) -}}
{{- $name := "" -}}
{{- $createdAt := "" -}}
{{- range (dig "items" (list) ($classes | default dict)) -}}
{{- $annotations := dig "metadata" "annotations" (dict) . -}}
{{- if or (eq (dig "storageclass.kubernetes.io/is-default-class" "" $annotations | toString) "true") (eq (dig "storageclass.beta.kubernetes.io/is-default-class" "" $annotations | toString) "true") -}}
{{- $at := dig "metadata" "creationTimestamp" "" . | toString -}}
{{- if or (not $name) (gt $at $createdAt) -}}
{{- $name = dig "metadata" "name" "" . | toString -}}
{{- $createdAt = $at -}}
{{- range list "storageclass.kubevirt.io/is-default-virt-class" "storageclass.kubernetes.io/is-default-class" "storageclass.beta.kubernetes.io/is-default-class" -}}
{{- $key := . -}}
{{- $createdAt := "" -}}
{{- if not $name -}}
{{- range $classes -}}
{{- $annotations := dig "metadata" "annotations" (dict) . -}}
{{- if eq (dig $key "" $annotations | toString) "true" -}}
{{- $at := dig "metadata" "creationTimestamp" "" . | toString -}}
{{- if or (not $name) (gt $at $createdAt) -}}
{{- $name = dig "metadata" "name" "" . | toString -}}
{{- $createdAt = $at -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}
@@ -269,7 +269,7 @@ spec:
{{- $httpURL := printf "%s/image/%s/%s/openstack-amd64.raw.xz" ($reqFactoryURL | trimSuffix "/") $reqSchematicID $reqVersion }}
{{- $goldenName := printf "talos-worker-%s-%s" $reqSchematicID $reqVersion }}
{{- /* A builtin pool whose golden is absent, failed to import, or on a
StorageClass the worker disk cannot CSI-clone from fails loudly here
StorageClass other than the worker disk's fails loudly here
rather than degrading silently. And because a CDI clone target must be
at least as large as its source, a worker diskSize below the golden's
storage would fail the clone at runtime — reject it at render when the
@@ -302,7 +302,7 @@ spec:
and no CR edit is needed to trigger the abort, only the next periodic
reconcile. The comment below rejects exactly that blast radius for a
mid-import golden; applying the same reasoning to a golden that is
absent or Failed is what it costs to be consistent.
absent or failing to import is what it costs to be consistent.
So the loud failure stays where it is the only signal — a pool still
waiting for its first disk — and a pool that is past that point is
@@ -334,7 +334,7 @@ spec:
from it too, and the two disagreeing is how a pool at storageClass ""
passed the comparison on the class its existing disks are bound to and
then bound its next disk on whatever the cluster default had become by
then -- taking the host-assisted copy the guard exists to refuse, with
then -- taking the cross-class clone the guard exists to refuse, with
the guard reporting no drift.
A pool that declares no class took the default in force when ITS disks
@@ -367,35 +367,51 @@ spec:
{{- fail (printf "nodeGroup %q: osImage.builtin requested golden image %q not found in cozy-public, neither as a DataVolume nor as a PVC — enable the cozystack.kubernetes-worker-image catalog via bundles.enabledPackages with a matching (schematicID, version) entry, or switch this pool to osImage.factory." .groupName $goldenName) }}
{{- end }}
{{- else }}
{{- /* Fail only on the golden's terminal failure phase, not on "not yet
{{- /* Fail only on a golden whose import is failing, not on "not yet
Succeeded". A golden mid-import is transient: CDI holds the worker
DataVolume until the source is populated, which is the correct
behaviour and recovers unattended. Hard-failing the render there would
abort the ENTIRE KubernetesNodes HelmRelease — every resource, not just
this disk — for every pool pinned to that golden, blocking unrelated
scale operations for the length of an import. Only Failed, which
never recovers without operator action, justifies that blast radius.
scale operations for the length of an import. Failed never recovers
without operator action, but CDI reaches it only on a fatal error such
as a checksum mismatch: an unreachable URL restarts the importer with
back-off while the phase stays in progress. So an importer that
restarted at least three times and is not running now is refused too;
that refusal lifts on its own once an attempt succeeds.
Falling back to the HTTP import is deliberately not an option either: it
would make disk-system's source depend on cluster state, so the source —
and with it the content-hashed KubevirtMachineTemplate name — would flip
once the golden went Ready and roll every worker in the pool. */}}
{{- $goldenPhase := dig "status" "phase" "" $goldenDV | toString }}
{{- $goldenRestarts := dig "status" "restartCount" 0 $goldenDV | int }}
{{- $goldenRunning := "" }}
{{- range (dig "status" "conditions" (list) $goldenDV) }}
{{- if eq (dig "type" "" . | toString) "Running" }}{{- $goldenRunning = dig "status" "" . | toString }}{{- end }}
{{- end }}
{{- if and (eq $goldenPhase "Failed") (not $poolCloned) }}
{{- fail (printf "nodeGroup %q: golden DataVolume %q in cozy-public failed to import (phase Failed) — cloning it would block this pool's workers indefinitely. Check the importer pod in cozy-public, then delete the DataVolume to retry the import." .groupName $goldenName) }}
{{- else if and (ne $goldenPhase "Succeeded") (ge $goldenRestarts 3) (eq $goldenRunning "False") (not $poolCloned) }}
{{- fail (printf "nodeGroup %q: golden DataVolume %q in cozy-public keeps failing to import (%d importer restarts) — cloning it would block this pool's workers while it does. The refusal lifts by itself while a new import attempt is running, and for good once one succeeds. The importer is retried with a growing back-off, and a recreated importer pod counts as not running until its container starts, so after the source is fixed the refusal lasts at least until the current back-off delay ends and the next importer starts. Check the importer pod logs in cozy-public." .groupName $goldenName $goldenRestarts) }}
{{- end }}
{{- /* CDI takes the storage-layer CSI-clone path only when source and target
share a StorageClass; across classes it silently falls back to a
{{- /* Keep a pool on its golden's StorageClass. CDI falls back to a
host-assisted copy over the pod network — exactly the per-worker
network transfer osImage.builtin exists to avoid (#3231). Reject the
case we can prove. An empty class on either side means "the cluster
default", which is resolvable against a live cluster and is resolved
here: leaving it unresolved is what let the one corner that CANNOT be
a clone — a pool at storageClass "" whose golden is on an explicit
class — pass unchecked. With no cluster to read (helm template) both
stay empty and the comparison is skipped, as before. This rules out
the guaranteed-host-assisted case, not every one: a matching
StorageClass still leaves the strategy to the CDI StorageProfile,
which the worker DataVolume inherits. */}}
network transfer osImage.builtin exists to avoid (#3231) — when the
two claims have different CSI drivers or volume modes
(validateAdvancedClonePVC in CDI's pkg/controller/clone/planner.go).
Two classes of one driver do not fall back: CDI asks that driver for
a snapshot or CSI clone into the other class, and whether that works
depends on the driver and on both classes' parameters. Sharing a
class rules out both a driver mismatch and a cross-class clone. An
empty class on either side means the class CDI binds on, which is
resolvable against a live cluster and is resolved here: leaving it
unresolved is what let a pool at storageClass "" whose golden is on
an explicit class pass unchecked. With no cluster to read (helm template) both stay empty
and the comparison is skipped, as before. A matching StorageClass
does not rule out every host-assisted copy: the strategy still comes
from the CDI StorageProfile the worker DataVolume inherits, and CDI
also falls back on differing volume modes, when no VolumeSnapshotClass
fits, or when a larger target cannot be expanded. */}}
{{- $workerSC := $cloneWorkerSC }}
{{- $goldenSC := dig "spec" "storage" "storageClassName" "" $goldenDV | toString }}
{{- /* A golden that declares no class is not on "the cluster default" as it
@@ -427,9 +443,9 @@ spec:
made, and reports it as the empty string. */}}
{{- $defaultNote := "" }}
{{- if $resolvedFromDefault }}
{{- $defaultNote = printf " An empty storageClass was resolved to the cluster default StorageClass %q." $defaultSC }}
{{- $defaultNote = printf " An empty storageClass was resolved to %q, the class CDI binds a disk that names none on." $defaultSC }}
{{- end }}
{{- fail (printf "nodeGroup %q: storageClass %q differs from golden image %q storageClass %q — CDI cannot CSI-clone across StorageClasses and would fall back to a host-assisted copy over the pod network.%s Neither side can be moved in place: this golden's class is fixed for its lifetime, because a DataVolume spec cannot be patched, and the pool's storageClass is immutable on an existing pool. So add a NEW kubernetes-worker-image catalog entry with a different (schematicID, version) on storageClass %q (kubernetesWorkerImage.images in the cozystack-platform Package values) and point this pool's osImage.builtin at it; or, once no pool clones this golden, delete it and let the catalog recreate it on that class. Switching this pool to osImage.factory also resolves it." .groupName $workerSC $goldenName $goldenSC $defaultNote $workerSC) }}
{{- fail (printf "nodeGroup %q: storageClass %q differs from golden image %q storageClass %q — across StorageClasses CDI either falls back to a host-assisted copy over the pod network (different CSI drivers or volume modes) or leaves a clone into the other class to the CSI driver.%s Neither side can be moved in place: this golden's class is fixed for its lifetime, because a DataVolume spec cannot be patched, and the pool's storageClass is immutable on an existing pool. So add a NEW kubernetes-worker-image catalog entry with a different (schematicID, version) on storageClass %q (kubernetesWorkerImage.images in the cozystack-platform Package values) and point this pool's osImage.builtin at it; or, once no pool clones this golden, delete it and let the catalog recreate it on that class. Switching this pool to osImage.factory also resolves it." .groupName $workerSC $goldenName $goldenSC $defaultNote $workerSC) }}
{{- end }}
{{- $goldenStorage := dig "spec" "storage" "resources" "requests" "storage" "" $goldenDV }}
{{- /* Same authority argument as the class one line up: a golden whose spec
@@ -529,14 +545,15 @@ spec:
rejects RWX volumes that are not on a DRBD-backed StorageClass,
which is why the pool's storageClass defaults to a replicated/DRBD
class rather than to the cluster default. An empty storageClass
omits the field so the cluster default applies.
omits the field, and CDI binds the disk on its virtualization
default or, without one, the Kubernetes default.
A builtin pool instead renders the class the guard above resolved for
it. Omitting the field there would let the disk bind on whatever the
cluster default is at bind time, and CDI takes the storage-layer path
only when source and target share a class -- so a default that moved
after the pool's earlier disks were bound turns the next clone into a
host-assisted copy over the pod network, the transfer the guard exists
cluster default is at bind time -- so a default that moved after the
pool's earlier disks were bound turns the next clone into a
cross-class one, which CDI either runs as a host-assisted copy over
the pod network or leaves to the CSI driver, the clone the guard exists
to refuse, while the guard reports no drift because it compared the
class those earlier disks sit on. With the class empty and no cluster
to resolve against the field is still omitted, so the HTTP path does
@@ -0,0 +1,81 @@
suite: osImage.builtin default-StorageClass selection with only the beta annotation
# CDI does not read the beta annotation, so the pool's class comes from the
# chart alone, written onto the worker disk explicitly.
templates:
- templates/nodegroup.yaml
release:
name: kubernetes-nodes-myk8s-md0
namespace: tenant-test
set:
cluster: myk8s
_cluster:
cluster-domain: cozy.local
version: "v1.35"
minReplicas: 1
maxReplicas: 3
instanceType: u1.medium
diskSize: 20Gi
storageClass: ""
roles: [ingress-nginx]
osImage:
builtin: {}
kubernetesProvider:
scheme:
"instancetype.kubevirt.io/v1beta1/VirtualMachineClusterInstancetype":
gvr: {group: instancetype.kubevirt.io, version: v1beta1, resource: virtualmachineclusterinstancetypes}
namespaced: false
"cluster.x-k8s.io/v1beta1/MachineSet":
gvr: {group: cluster.x-k8s.io, version: v1beta1, resource: machinesets}
namespaced: true
"infrastructure.cluster.x-k8s.io/v1alpha1/KubevirtMachineTemplate":
gvr: {group: infrastructure.cluster.x-k8s.io, version: v1alpha1, resource: kubevirtmachinetemplates}
namespaced: true
"cdi.kubevirt.io/v1beta1/DataVolume":
gvr: {group: cdi.kubevirt.io, version: v1beta1, resource: datavolumes}
namespaced: true
"storage.k8s.io/v1/StorageClass":
gvr: {group: storage.k8s.io, version: v1, resource: storageclasses}
namespaced: false
objects:
- kind: VirtualMachineClusterInstancetype
apiVersion: instancetype.kubevirt.io/v1beta1
metadata: {name: u1.medium}
- kind: MachineSet
apiVersion: cluster.x-k8s.io/v1beta1
metadata: {name: ignored-ms, namespace: tenant-test}
spec: {template: {spec: {infrastructureRef: {kind: Other, name: ignored}}}}
- kind: KubevirtMachineTemplate
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
metadata: {name: ignored-kmt, namespace: tenant-test}
# The outgoing default, created first.
- kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: beta-only
creationTimestamp: "2026-01-01T00:00:00Z"
annotations:
storageclass.beta.kubernetes.io/is-default-class: "true"
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: talos-worker-ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515-v1.13.6
namespace: cozy-public
spec:
storage:
resources: {requests: {storage: 6Gi}}
storageClassName: beta-only
status: {phase: Succeeded}
tests:
- it: resolves an empty pool storageClass to a class with only the beta annotation
asserts:
- notFailedTemplate: {}
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.storage.storageClassName
value: beta-only
documentSelector: {path: kind, value: KubevirtMachineTemplate}
@@ -1,6 +1,6 @@
suite: osImage.builtin golden guards on a pool that has already cloned
# The absence and Failed guards abort the ENTIRE KubernetesNodes HelmRelease, and
# The absence and failed-import guards abort the ENTIRE KubernetesNodes HelmRelease, and
# for a pool whose workers all cloned long ago that price buys nothing: replica
# changes, kubelet retuning and MachineHealthCheck timeouts have no stake in the
# golden, and no CR edit is needed to trigger the abort — the next periodic
@@ -152,6 +152,30 @@ kubernetesProvider:
resources: {requests: {storage: 6Gi}}
storageClassName: replicated
status: {phase: Failed}
# md0's own disk against a golden whose re-import is now stuck: the pool
# already holds its copy, so it needs nothing from the source.
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: disk-system-kubernetes-myk8s-md0-pqrst
namespace: tenant-test
annotations:
kubernetes-worker-image.cozystack.io/golden: talos-worker-stucklater-v1.0.0
kubernetes-worker-image.cozystack.io/pool: kubernetes-myk8s-md0
status: {phase: Succeeded}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: talos-worker-stucklater-v1.0.0, namespace: cozy-public}
spec:
storage:
resources: {requests: {storage: 6Gi}}
storageClassName: replicated
status:
phase: ImportInProgress
# A string: helm-unittest's lookup cannot deep-copy an int, and | int takes either.
restartCount: "7"
conditions:
- {type: Running, status: "False", reason: Error}
tests:
- it: renders when the golden is gone but this pool already cloned it
@@ -217,3 +241,13 @@ tests:
asserts:
- failedTemplate:
errorPattern: 'golden DataVolume "talos-worker-sibling-v1\.0\.0" in cozy-public failed to import'
- it: renders when the golden importer keeps failing but this pool already cloned it
set:
osImage:
builtin: {schematicID: stucklater, version: v1.0.0}
asserts:
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
value: talos-worker-stucklater-v1.0.0
documentSelector: {path: kind, value: KubevirtMachineTemplate}
@@ -101,10 +101,9 @@ kubernetesProvider:
status: {phase: Succeeded}
tests:
# Guaranteed NOT to be a clone, and the case an unresolved empty class waved
# through: the pool is on an explicit class, the golden is on the cluster
# default, and the two differ. CDI would fall back to the host-assisted copy
# over the pod network — silently, which is the transfer this feature removes.
# A cross-class clone, and the case an unresolved empty class waved through:
# the pool is on an explicit class, the golden is on the cluster default, and
# the two differ.
- it: fails when the golden takes a cluster default class the pool does not use
set:
storageClass: local
@@ -145,7 +144,7 @@ tests:
# Reading the spec alone would call this golden "on the cluster default", which
# is `replicated` here, and clone it for a `replicated` pool. Its PVC is on
# `local`, so that clone would silently be a host-assisted copy.
# `local`, so that clone would cross classes.
- it: takes a blank-spec golden's class from its PVC, not from today's default
set:
storageClass: replicated
@@ -189,7 +189,7 @@ tests:
documentSelector: {path: kind, value: KubevirtMachineTemplate}
# And the disk binds on the class the guard just compared. Leaving the field
# out here passed the guard and then bound the next disk on `fast-nvme`, one
# class away from the golden, which is the host-assisted copy the guard is
# class away from the golden, which is the cross-class clone the guard is
# there to refuse.
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.storage.storageClassName
@@ -243,7 +243,7 @@ tests:
version: v1.13.6
asserts:
- failedTemplate:
errorPattern: 'An empty storageClass was resolved to the cluster default StorageClass "fast-nvme"'
errorPattern: 'An empty storageClass was resolved to "fast-nvme", the class CDI binds a disk that names none on\.'
# With both sides explicit nothing was resolved from the default, and the
# sentence is absent rather than reporting an empty lookup. The two sentences
@@ -257,7 +257,7 @@ tests:
version: v1.13.6
asserts:
- failedTemplate:
errorPattern: 'host-assisted copy over the pod network\. Neither side can be moved in place'
errorPattern: 'to the CSI driver\. Neither side can be moved in place'
# The size floor reads the PVC when the DataVolume spec omits a size, so a
# golden it could size no longer slips past unmeasured.
@@ -119,6 +119,54 @@ kubernetesProvider:
storageClassName: replicated
status:
phase: Failed
# Goldens whose importer has restarted. CDI leaves an unreachable URL in
# progress and restarts the importer instead of reaching Failed, so restarts
# plus an importer that is not running now is what marks one as failing.
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: talos-worker-stuck-v1.0.0, namespace: cozy-public}
spec: &goldenSpec
storage:
resources: {requests: {storage: 6Gi}}
storageClassName: replicated
status:
phase: ImportInProgress
# A string: helm-unittest's lookup cannot deep-copy an int, and | int takes either.
restartCount: "3"
conditions:
- {type: Running, status: "False", reason: Error}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: talos-worker-recovered-v1.0.0, namespace: cozy-public}
spec: *goldenSpec
status:
phase: ImportInProgress
restartCount: "5"
conditions:
- {type: Running, status: "True", reason: Pod is running}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: talos-worker-flaky-v1.0.0, namespace: cozy-public}
spec: *goldenSpec
status:
phase: ImportInProgress
restartCount: "2"
conditions:
- {type: Running, status: "False", reason: Error}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: talos-worker-done-v1.0.0, namespace: cozy-public}
spec: *goldenSpec
status:
phase: Succeeded
restartCount: "4"
conditions:
- {type: Running, status: "False", reason: Completed}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: talos-worker-norunning-v1.0.0, namespace: cozy-public}
spec: *goldenSpec
status: {phase: ImportInProgress, restartCount: "3"}
# A populated golden that declares no storage request at all. The size guard
# reads spec.storage.resources.requests.storage, so its presence gate is the
# branch that decides between comparing and skipping; without a fixture the
@@ -271,8 +319,56 @@ tests:
- failedTemplate:
errorPattern: 'golden DataVolume "talos-worker-failed-v1\.0\.0" in cozy-public failed to import \(phase Failed\)'
# Cross-StorageClass clone: CDI would silently fall back to a host-assisted copy
# over the pod network, the exact transfer osImage.builtin exists to avoid.
- it: osImage.builtin fails the render when the golden importer keeps failing
set:
osImage:
builtin: {schematicID: stuck, version: v1.0.0}
asserts:
- failedTemplate:
errorPattern: 'golden DataVolume "talos-worker-stuck-v1\.0\.0" in cozy-public keeps failing to import \(3 importer restarts\).*lifts by itself.*a recreated importer pod counts as not running until its container starts'
- it: osImage.builtin clones a golden whose importer restarted but is running again
set:
osImage:
builtin: {schematicID: recovered, version: v1.0.0}
asserts:
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
value: talos-worker-recovered-v1.0.0
documentSelector: {path: kind, value: KubevirtMachineTemplate}
- it: osImage.builtin clones a golden whose importer failed fewer than three times
set:
osImage:
builtin: {schematicID: flaky, version: v1.0.0}
asserts:
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
value: talos-worker-flaky-v1.0.0
documentSelector: {path: kind, value: KubevirtMachineTemplate}
- it: osImage.builtin clones a golden that Succeeded after its importer restarted
set:
osImage:
builtin: {schematicID: done, version: v1.0.0}
asserts:
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
value: talos-worker-done-v1.0.0
documentSelector: {path: kind, value: KubevirtMachineTemplate}
- it: osImage.builtin clones a golden whose importer restarted and reports no Running condition
set:
osImage:
builtin: {schematicID: norunning, version: v1.0.0}
asserts:
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
value: talos-worker-norunning-v1.0.0
documentSelector: {path: kind, value: KubevirtMachineTemplate}
# Cross-StorageClass clone: a host-assisted copy over the pod network, or a
# clone left to the CSI driver to support, so the pool is refused.
- it: osImage.builtin fails when the pool storageClass differs from the golden's
set:
storageClass: local
@@ -114,3 +114,22 @@ tests:
asserts:
- failedTemplate:
errorPattern: 'requested golden image "talos-worker-elsewhere-v1\.0\.0" not found'
# CDI reads only the GA annotation when it picks a class itself, so a newer
# class carrying only the beta one does not displace a GA default.
- it: prefers a GA default over a newer class with only the beta annotation
kubernetesProvider:
objects:
- kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: beta-only
creationTimestamp: "2026-09-01T00:00:00Z"
annotations:
storageclass.beta.kubernetes.io/is-default-class: "true"
asserts:
- notFailedTemplate: {}
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.storage.storageClassName
value: new
documentSelector: {path: kind, value: KubevirtMachineTemplate}
@@ -0,0 +1,183 @@
suite: osImage.builtin default-StorageClass selection with a virtualization default
# CDI resolves a DataVolume with no class and contentType kubevirt to the class
# annotated storageclass.kubevirt.io/is-default-virt-class before it looks at the
# Kubernetes default, so a cluster that carries both binds the golden and the
# worker disks on the virtualization default. The clone guard has to resolve an
# empty class the same way, or it compares the golden against a class it is not
# on.
templates:
- templates/nodegroup.yaml
release:
name: kubernetes-nodes-myk8s-md0
namespace: tenant-test
set:
cluster: myk8s
_cluster:
cluster-domain: cozy.local
version: "v1.35"
minReplicas: 1
maxReplicas: 3
instanceType: u1.medium
diskSize: 20Gi
storageClass: ""
roles: [ingress-nginx]
osImage:
builtin: {}
kubernetesProvider:
scheme:
"instancetype.kubevirt.io/v1beta1/VirtualMachineClusterInstancetype":
gvr: {group: instancetype.kubevirt.io, version: v1beta1, resource: virtualmachineclusterinstancetypes}
namespaced: false
"cluster.x-k8s.io/v1beta1/MachineSet":
gvr: {group: cluster.x-k8s.io, version: v1beta1, resource: machinesets}
namespaced: true
"infrastructure.cluster.x-k8s.io/v1alpha1/KubevirtMachineTemplate":
gvr: {group: infrastructure.cluster.x-k8s.io, version: v1alpha1, resource: kubevirtmachinetemplates}
namespaced: true
"cdi.kubevirt.io/v1beta1/DataVolume":
gvr: {group: cdi.kubevirt.io, version: v1beta1, resource: datavolumes}
namespaced: true
"storage.k8s.io/v1/StorageClass":
gvr: {group: storage.k8s.io, version: v1, resource: storageclasses}
namespaced: false
"v1/PersistentVolumeClaim":
gvr: {group: "", version: v1, resource: persistentvolumeclaims}
namespaced: true
objects:
- kind: VirtualMachineClusterInstancetype
apiVersion: instancetype.kubevirt.io/v1beta1
metadata: {name: u1.medium}
- kind: MachineSet
apiVersion: cluster.x-k8s.io/v1beta1
metadata: {name: ignored-ms, namespace: tenant-test}
spec: {template: {spec: {infrastructureRef: {kind: Other, name: ignored}}}}
- kind: KubevirtMachineTemplate
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
metadata: {name: ignored-kmt, namespace: tenant-test}
# The Kubernetes default, created after the virtualization default so that
# creation order alone would pick it.
- kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: kube-default
creationTimestamp: "2026-06-01T00:00:00Z"
annotations:
storageclass.kubernetes.io/is-default-class: "true"
- kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: virt-default
creationTimestamp: "2026-01-01T00:00:00Z"
annotations:
storageclass.kubevirt.io/is-default-virt-class: "true"
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: talos-worker-ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515-v1.13.6
namespace: cozy-public
spec:
storage:
resources: {requests: {storage: 6Gi}}
storageClassName: virt-default
status: {phase: Succeeded}
tests:
- it: resolves an empty pool storageClass to the virtualization default
asserts:
- notFailedTemplate: {}
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.storage.storageClassName
value: virt-default
documentSelector: {path: kind, value: KubevirtMachineTemplate}
- it: takes the most recently created of two virtualization defaults
kubernetesProvider:
objects:
- kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: virt-newer
creationTimestamp: "2026-03-01T00:00:00Z"
annotations:
storageclass.kubevirt.io/is-default-virt-class: "true"
asserts:
- failedTemplate:
errorPattern: 'storageClass "virt-newer" differs from golden image .* storageClass "virt-default".* An empty storageClass was resolved to "virt-newer"'
- it: ignores a virtualization default annotation that is not "true"
kubernetesProvider:
objects:
- kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: virt-off
creationTimestamp: "2026-05-01T00:00:00Z"
annotations:
storageclass.kubevirt.io/is-default-virt-class: "false"
asserts:
- notFailedTemplate: {}
# Before the virtualization default was read, a pool that has not cloned a
# disk yet resolved to the Kubernetes default here and rendered.
- it: refuses a pool whose golden is on the Kubernetes default instead
set:
osImage:
builtin: {schematicID: kube, version: v1.0.0}
kubernetesProvider:
objects:
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: talos-worker-kube-v1.0.0
namespace: cozy-public
spec:
storage:
resources: {requests: {storage: 6Gi}}
storageClassName: kube-default
status: {phase: Succeeded}
asserts:
- failedTemplate:
errorPattern: 'storageClass "virt-default" differs from golden image .* storageClass "kube-default".* An empty storageClass was resolved to "virt-default"'
# A pool that already cloned reads its class back from its own disk's claim
# and never reaches the default lookup.
- it: keeps rendering a pool that already cloned from a golden on the Kubernetes default
set:
osImage:
builtin: {schematicID: kube, version: v1.0.0}
kubernetesProvider:
objects:
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: talos-worker-kube-v1.0.0
namespace: cozy-public
spec:
storage:
resources: {requests: {storage: 6Gi}}
storageClassName: kube-default
status: {phase: Succeeded}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: disk-system-kubernetes-myk8s-md0-abcde
namespace: tenant-test
annotations:
kubernetes-worker-image.cozystack.io/golden: talos-worker-kube-v1.0.0
kubernetes-worker-image.cozystack.io/pool: kubernetes-myk8s-md0
status: {phase: Succeeded}
- kind: PersistentVolumeClaim
apiVersion: v1
metadata: {name: disk-system-kubernetes-myk8s-md0-abcde, namespace: tenant-test}
spec: {storageClassName: kube-default}
asserts:
- notFailedTemplate: {}
- equal:
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.storage.storageClassName
value: kube-default
documentSelector: {path: kind, value: KubevirtMachineTemplate}
@@ -14,7 +14,7 @@
]
},
"storageClass": {
"description": "StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.",
"description": "StorageClass for the worker node system disk. When empty, CDI picks the class: the one annotated `storageclass.kubevirt.io/is-default-virt-class`, else the cluster default. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: the chart refuses a pool whose class differs from its golden's, because across classes CDI either falls back to a host-assisted copy over the pod network (different CSI drivers or volume modes) or leaves a clone into the other class to the CSI driver. Leaving it empty makes the same comparison against the class CDI would pick. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.",
"type": "string",
"default": "replicated",
"x-kubernetes-validations": [
+1 -1
View File
@@ -22,7 +22,7 @@ cluster: ""
## @section Common Parameters
##
## @param {string} storageClass - StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.
## @param {string} storageClass - StorageClass for the worker node system disk. When empty, CDI picks the class: the one annotated `storageclass.kubevirt.io/is-default-virt-class`, else the cluster default. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: the chart refuses a pool whose class differs from its golden's, because across classes CDI either falls back to a host-assisted copy over the pod network (different CSI drivers or volume modes) or leaves a clone into the other class to the CSI driver. Leaving it empty makes the same comparison against the class CDI would pick. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.
## @x-cozystack-options {source: storageclass}
## @immutable
storageClass: replicated
+16 -3
View File
@@ -12,8 +12,17 @@
{{- if hasKey .Values.source "image" }}
{{- $desiredSource = dict "pvc" (dict "name" (printf "vm-default-images-%s" (required "A valid .Values.source.image.name entry required!" .Values.source.image.name)) "namespace" "cozy-public") }}
{{- /* Refuse a golden that is missing or whose import Failed: CDI never
populates it, so the clone would wait forever. A golden still importing
is left alone because CDI holds the clone until the source is populated.
populates it, so the clone would wait forever. Failed is reached only on
a fatal error such as a checksum mismatch; an unreachable URL instead
restarts the importer with back-off while the phase stays in progress,
so an importer that restarted at least three times counts as failing
too, running or not: CDI reports every attempt as running, including
one that is about to fail again, and this check runs only until the
disk exists, so a disk created during an attempt would pass and wait
unnoticed. A healthy import with three early restarts is refused until
it succeeds, and the release retries the render. Any other golden still
importing is left alone because CDI holds the clone until the source is
populated.
Missing means neither the DataVolume nor the PVC CDI clones exists, as
the DataVolume can precede its PVC, and counts only when the release
namespace is visible: helm template has no cluster and every lookup
@@ -24,8 +33,12 @@
{{- if not (or $existingDV (lookup "v1" "PersistentVolumeClaim" .Release.Namespace .Release.Name)) }}
{{- $golden := lookup "cdi.kubevirt.io/v1beta1" "DataVolume" "cozy-public" $desiredSource.pvc.name }}
{{- $goldenPVC := lookup "v1" "PersistentVolumeClaim" "cozy-public" $desiredSource.pvc.name }}
{{- if eq (dig "status" "phase" "" $golden | toString) "Failed" }}
{{- $goldenPhase := dig "status" "phase" "" $golden | toString }}
{{- $goldenRestarts := dig "status" "restartCount" 0 $golden | int }}
{{- if eq $goldenPhase "Failed" }}
{{- fail (printf "source.image %q: golden DataVolume %q in cozy-public failed to import (phase Failed), so a clone of it would never complete. Ask the platform administrator to check the importer pod in cozy-public, then delete that DataVolume to retry the import." .Values.source.image.name $desiredSource.pvc.name) }}
{{- else if and (ne $goldenPhase "Succeeded") (ge $goldenRestarts 3) }}
{{- fail (printf "source.image %q: golden DataVolume %q in cozy-public has not finished importing after %d importer restarts, so a clone of it would not complete while the import keeps failing. The refusal lifts by itself once the import succeeds. The importer is retried with a growing back-off, so after the source is fixed the refusal lasts for the current back-off delay plus the full image download. Ask the platform administrator to check the importer pod logs in cozy-public." .Values.source.image.name $desiredSource.pvc.name $goldenRestarts) }}
{{- else if and (not $golden) (not $goldenPVC) (lookup "v1" "Namespace" "" .Release.Namespace) }}
{{- fail (printf "source.image %q: golden image %q not found in cozy-public, so a clone of it would never start. Check the image name, or ask the platform administrator to add cozystack.vm-default-images to bundles.enabledPackages." .Values.source.image.name $desiredSource.pvc.name) }}
{{- end }}
@@ -1,8 +1,9 @@
suite: vm-disk refuses to clone a golden image that is missing or failed
# A golden that is missing or whose import Failed is never populated, and CDI
# holds a clone of it forever, so a new disk on it would sit pending with
# nothing pointing at the image. A golden still importing is safe to clone
# because CDI waits for it, and helm template has no cluster to look it up in.
suite: vm-disk refuses to clone a golden image that is missing, failed or stuck
# A golden that is missing, whose import Failed, or whose importer keeps
# crashing is not populated, and CDI holds a clone of it until it is, so a new
# disk on it would sit pending with nothing pointing at the image. A golden
# still importing is safe to clone because CDI waits for it, and helm template
# has no cluster to look it up in.
release:
name: test-vm-disk
namespace: tenant-test
@@ -185,3 +186,139 @@ tests:
- equal:
path: spec.source.pvc
value: {name: vm-default-images-ubuntu, namespace: cozy-public}
- it: refuses a new disk on a golden whose importer keeps failing
set:
source: {image: {name: ubuntu}}
kubernetesProvider:
scheme: *scheme
objects:
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status: &stuck
phase: ImportInProgress
# A string: helm-unittest's lookup cannot deep-copy an int, and | int takes either.
restartCount: "3"
conditions:
- {type: Bound, status: "True"}
- {type: Running, status: "False", reason: Error}
asserts:
- failedTemplate:
errorPattern: 'golden DataVolume "vm-default-images-ubuntu" in cozy-public has not finished importing after 3 importer restarts.*lifts by itself once the import succeeds'
# CDI reports Running=True for every importer attempt, including one that is
# about to fail again, and this check runs only until the disk exists.
- it: refuses a new disk on a golden that restarted and is running again
set:
source: {image: {name: ubuntu}}
kubernetesProvider:
scheme: *scheme
objects:
- kind: Namespace
apiVersion: v1
metadata: {name: tenant-test}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status:
phase: ImportInProgress
restartCount: "5"
conditions:
- {type: Running, status: "True", reason: Pod is running}
- kind: PersistentVolumeClaim
apiVersion: v1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
asserts:
- failedTemplate:
errorPattern: 'has not finished importing after 5 importer restarts'
- it: clones a golden whose importer failed fewer than three times
set:
source: {image: {name: ubuntu}}
kubernetesProvider:
scheme: *scheme
objects:
- kind: Namespace
apiVersion: v1
metadata: {name: tenant-test}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status:
phase: ImportInProgress
restartCount: "2"
conditions:
- {type: Running, status: "False", reason: Error}
- kind: PersistentVolumeClaim
apiVersion: v1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
asserts:
- notFailedTemplate: {}
- it: clones a golden that Succeeded after its importer restarted
set:
source: {image: {name: ubuntu}}
kubernetesProvider:
scheme: *scheme
objects:
- kind: Namespace
apiVersion: v1
metadata: {name: tenant-test}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status:
phase: Succeeded
restartCount: "4"
conditions:
- {type: Running, status: "False", reason: Completed}
- kind: PersistentVolumeClaim
apiVersion: v1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
asserts:
- notFailedTemplate: {}
- it: leaves an existing disk alone when its golden is stuck
set:
source: {image: {name: ubuntu}}
kubernetesProvider:
scheme: *scheme
objects:
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status: *stuck
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata:
name: test-vm-disk
namespace: tenant-test
annotations:
vm-disk.cozystack.io/source: '{"image":{"name":"ubuntu"}}'
vm-disk.cozystack.io/storage-class: replicated
spec:
source: {pvc: {name: vm-default-images-ubuntu, namespace: cozy-public}}
storage: {resources: {requests: {storage: 5Gi}}, storageClassName: replicated}
asserts:
- notFailedTemplate: {}
- it: refuses a new disk on a golden whose importer restarted and reports no Running condition
set:
source: {image: {name: ubuntu}}
kubernetesProvider:
scheme: *scheme
objects:
- kind: Namespace
apiVersion: v1
metadata: {name: tenant-test}
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status: {phase: ImportInProgress, restartCount: "3"}
- kind: PersistentVolumeClaim
apiVersion: v1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
asserts:
- failedTemplate:
errorPattern: 'has not finished importing after 3 importer restarts'
@@ -1115,6 +1115,48 @@ tests:
asserts:
- notFailedTemplate: {}
- it: leaves a disk alone whose PVC exists without its DataVolume when the golden is stuck
set:
source: {image: {name: ubuntu}}
storageClass: local
kubernetesProvider:
scheme: *scheme
objects:
- *namespace
- *diskPVC
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status: &stuck
phase: ImportInProgress
restartCount: "3"
conditions:
- {type: Running, status: "False", reason: Error}
asserts:
- notFailedTemplate: {}
# Fixing the class alone would still leave the clone waiting on the import.
- it: names the stuck import, not the class, for a disk that trips both
set:
source: {image: {name: ubuntu}}
storageClass: local
kubernetesProvider:
scheme: *scheme
objects:
- *namespace
- *goldenPVC
- *cdiCSIClone
- *replicatedClass
- *localClass
- *localProfile
- kind: DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
metadata: {name: vm-default-images-ubuntu, namespace: cozy-public}
status: *stuck
asserts:
- failedTemplate:
errorPattern: 'golden DataVolume "vm-default-images-ubuntu" in cozy-public has not finished importing after 3 importer restarts'
# A key under another prefix belongs to someone else and is not read.
- it: clones into a LINSTOR class that allows remote access next to a foreign key that denies it
set:
+4 -4
View File
@@ -765,10 +765,10 @@ backupStrategyController: {}
# cozystack.kubernetes-worker-image to bundles.enabledPackages -- the package is
# opt-in and this key alone does not enable it. The two knobs that decide
# whether the feature is usable at all are imageFactoryURL (an air-gapped or
# rate-limited cluster must repoint the single import) and storageClass (CDI
# cannot clone across StorageClasses, so a cluster whose worker pools are not on
# `replicated` must move the golden onto their class). Empty (the default)
# forwards nothing and keeps the child chart defaults.
# rate-limited cluster must repoint the single import) and storageClass (a
# worker pool refuses a golden on another StorageClass, so a cluster whose
# worker pools are not on `replicated` must move the golden onto their class).
# Empty (the default) forwards nothing and keeps the child chart defaults.
kubernetesWorkerImage: {}
# Pod scheduling configuration
scheduling:
@@ -48,7 +48,7 @@ rules:
- apiGroups: ["kubevirt.io"]
resources: ["kubevirts"]
verbs: ["get", "watch", "list"]
# The image picker drops golden images whose import Failed.
# The image picker drops golden images whose import failed or keeps failing.
- apiGroups: ["cdi.kubevirt.io"]
resources: ["datavolumes"]
verbs: ["get", "watch", "list"]
@@ -21,7 +21,7 @@ spec:
singular: kubernetesnodes
plural: kubernetesnodeses
openAPISchema: |-
{"title":"Chart Values","type":"object","properties":{"cluster":{"description":"Name of the parent Kubernetes cluster (the `Kubernetes` CR) in the same namespace this node pool attaches to. Required. The pool's CAPI objects are wired to the parent cluster `kubernetes-<cluster>`; the KubernetesNodes CR must be named `<cluster>-<pool>`. The pool's objects are named `kubernetes-<cluster>-<pool>`; when a MachineDeployment of that name already exists and belongs to another Helm release, the render fails instead of taking it over.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"cluster is immutable"}]},"storageClass":{"description":"StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.","type":"string","default":"replicated","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}},"minReplicas":{"description":"Minimum number of replicas in this pool. Used as the cluster-autoscaler floor.","type":"integer","default":0},"maxReplicas":{"description":"Maximum number of replicas in this pool. Used as the cluster-autoscaler ceiling.","type":"integer","default":10},"instanceType":{"description":"KubeVirt instance type that sizes each worker VM when `resources` is not set. Ignored when `resources.cpu` and `resources.memory` are both set, and therefore on `substrate: proxmox`, which requires them.","type":"string","default":"u1.medium","x-cozystack-options":{"source":"instancetype"}},"diskSize":{"description":"System disk size for the worker VM. Carries the Talos OS image, kubelet state, containerd image cache, and any local-path PVCs. How the OS image reaches the disk follows this pool's `osImage`: the default and `osImage.factory` stream the Image Factory raw artifact in over HTTP, while `osImage.builtin` makes the disk a storage-layer CDI clone of a shared golden image, with no HTTP fetch. A `builtin` pool must keep diskSize at or above the golden's storage size (6Gi in the shipped worker image catalog), because a CDI clone target cannot be smaller than its source -- the chart rejects a smaller value at render time rather than letting the clone stall. Sizing a `builtin` pool also has to allow for transient headroom the finished disk does not show: CDI populates a larger target by cloning the golden at the source size and then growing it, so a worker being provisioned occupies a temporary clone PVC at the golden's size on top of this diskSize, and both are multiplied by the replica count of a replicated StorageClass. The golden itself is a separate one-time cost for the whole cluster, not a per-worker one. So the peak is the golden, plus `diskSize + golden storage` for each worker cloning at that moment. A pool whose storage backend cannot spare that peak does not fail: the clone waits for space, the VM never boots, and the reason is visible only in the storage layer's own events. Has no effect on `substrate: proxmox`, where the worker disk is a clone of the VM template's disk and keeps its size.","default":"20Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"roles":{"description":"List of node roles. Each role `r` labels nodes with `node-role.kubernetes.io/<r>`. Use `[ingress-nginx]` for a pool that hosts the tenant ingress-nginx controller.","type":"array","default":[],"items":{"type":"string"}},"resources":{"description":"Explicit CPU and memory for each worker node, as an alternative to `instanceType` sizing. Optional: when omitted, the node is sized by `instanceType`. When both `cpu` and `memory` are set, they take precedence and `iLine truncated
{"title":"Chart Values","type":"object","properties":{"cluster":{"description":"Name of the parent Kubernetes cluster (the `Kubernetes` CR) in the same namespace this node pool attaches to. Required. The pool's CAPI objects are wired to the parent cluster `kubernetes-<cluster>`; the KubernetesNodes CR must be named `<cluster>-<pool>`. The pool's objects are named `kubernetes-<cluster>-<pool>`; when a MachineDeployment of that name already exists and belongs to another Helm release, the render fails instead of taking it over.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"cluster is immutable"}]},"storageClass":{"description":"StorageClass for the worker node system disk. When empty, CDI picks the class: the one annotated `storageclass.kubevirt.io/is-default-virt-class`, else the cluster default. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: the chart refuses a pool whose class differs from its golden's, because across classes CDI either falls back to a host-assisted copy over the pod network (different CSI drivers or volume modes) or leaves a clone into the other class to the CSI driver. Leaving it empty makes the same comparison against the class CDI would pick. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.","type":"string","default":"replicated","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}},"minReplicas":{"description":"Minimum number of replicas in this pool. Used as the cluster-autoscaler floor.","type":"integer","default":0},"maxReplicas":{"description":"Maximum number of replicas in this pool. Used as the cluster-autoscaler ceiling.","type":"integer","default":10},"instanceType":{"description":"KubeVirt instance type that sizes each worker VM when `resources` is not set. Ignored when `resources.cpu` and `resources.memory` are both set, and therefore on `substrate: proxmox`, which requires them.","type":"string","default":"u1.medium","x-cozystack-options":{"source":"instancetype"}},"diskSize":{"description":"System disk size for the worker VM. Carries the Talos OS image, kubelet state, containerd image cache, and any local-path PVCs. How the OS image reaches the disk follows this pool's `osImage`: the default and `osImage.factory` stream the Image Factory raw artifact in over HTTP, while `osImage.builtin` makes the disk a storage-layer CDI clone of a shared golden image, with no HTTP fetch. A `builtin` pool must keep diskSize at or above the golden's storage size (6Gi in the shipped worker image catalog), because a CDI clone target cannot be smaller than its source -- the chart rejects a smaller value at render time rather than letting the clone stall. Sizing a `builtin` pool also has to allow for transient headroom the finished disk does not show: CDI populates a larger target by cloning the golden at the source size and then growing it, so a worker being provisioned occupies a temporary clone PVC at the golden's size on top of this diskSize, and both are multiplied by the replica count of a replicated StorageClass. The golden itself is a separate one-time cost for the whole cluster, not a per-worker one. So the peak is the golden, plus `diskSize + golden storage` for each worker cloning at that moment. A pool whose storage backend cannot spare that peak does not fail: the clone waits for space, the VM never boots, and the reason is visible only in the storage layer's own events. Has no effect on `substrate: proxmox`, where the worker disk is a clone of the VM template's disk and keeps its size.","default":"20Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"roles":{"description":"List of node roles. Each role `r` labels nodes with `node-role.kubernetes.io/<r>`. Use `[ingress-nginx]` for a pool that hosts the tenant ingress-nginx controller.","type":"array","default":[],"items":{"type":"string"}},"resources":{"description":"Explicit CPU and memory for each worker node, as an alternative to `instanceType` sizing. Optional: when omitted, the node is sized by `instanceType`. When bLine truncated
release:
labels:
sharding.fluxcd.io/key: tenants
@@ -30,10 +30,12 @@
## `talos-worker-<schematicID>-<version>` from cozy-public. The tenant render fails
## rather than degrade silently when the referenced (schematicID, version) has no
## matching entry in this catalog, when its golden terminally failed to import
## (`phase: Failed`), or when the node group's storageClass differs from the
## (`phase: Failed`) or keeps failing to (its importer restarted at least 3 times
## and is not running), or when the node group's storageClass differs from the
## golden's — so keep the catalog in sync with the (schematicID, version) pairs and
## StorageClasses that tenant node groups reference. A golden that is merely
## mid-import is not an error: CDI holds the worker disk until the source populates.
## StorageClasses that tenant node groups reference. A golden still importing, with
## its importer running or restarted fewer than 3 times, is not an error: CDI holds
## the worker disk until the source populates.
## Goldens carry `helm.sh/resource-policy: keep`, so removing an `images[]` entry
## stops managing that golden rather than deleting it. Deleting one by hand while a
## worker pool still clones it fails that pool's render on its next values or chart
@@ -88,7 +90,7 @@
## @param {string} imageFactoryURL - Default base URL of the Talos Image Factory serving the raw worker OS image (`<url>/image/<schematicID>/<version>/openstack-amd64.raw.xz`). Overridable per image. No trailing slash.
imageFactoryURL: "https://factory.talos.dev"
## @param {string} storageClass - Default StorageClass for the golden image(s). Must match the StorageClass of the worker disks that clone it: CDI cannot CSI-clone across StorageClasses and would silently fall back to a host-assisted copy over the pod network, so the tenant render rejects a mismatch outright. Overridable per image.
## @param {string} storageClass - Default StorageClass for the golden image(s). Must match the StorageClass of the worker disks that clone it: across classes CDI either falls back to a host-assisted copy over the pod network or leaves the clone to the CSI driver, so the tenant render rejects a mismatch outright. Overridable per image.
storageClass: "replicated"
## @param {quantity} storage - Default storage size allocated for each golden image. Must be >= the imported raw image virtual size and <= the smallest worker diskSize that clones it. The Talos v1.13.x openstack raw is ~4.15 GiB, so 6Gi leaves headroom. This size is also the unit of the transient cost every clone pays: CDI populates a worker disk larger than the golden by cloning at the golden's size and then growing it, so one worker in flight occupies the golden plus a temporary clone PVC plus its own diskSize, times the replica count of a replicated StorageClass. Keeping this at the raw image's size rather than rounding it up is therefore worth doing -- it is paid once per golden and again per concurrent clone. Overridable per image, but only when the entry is added: a DataVolume spec cannot be patched in place, so raising this on an entry whose golden already exists is rejected on the next upgrade — a Talos version whose raw outgrows it needs a new `images[]` entry, which it gets anyway since the name is keyed on the version.
@@ -9,7 +9,7 @@
## images that did import stay usable meanwhile.
##
## @param {string} storageClass - Default StorageClass for all images. If empty, uses the cluster default StorageClass.
## @param {string} storageClass - Default StorageClass for all images. If empty, CDI picks the class: the one annotated `storageclass.kubevirt.io/is-default-virt-class`, else the cluster default.
## NOTE: The default set of images requires approximately 320Gi of storage (16 images × 20Gi each).
## Adjust the image list or per-image storage sizes to match your cluster capacity.
storageClass: "replicated"
+32 -4
View File
@@ -335,9 +335,10 @@ func permittedResourceNames(kv *unstructured.Unstructured) map[string]struct{} {
// imageProvider lists the default image PVCs in cozy-public and strips the
// vm-default-images- prefix to get the catalog name used by the vm-disk chart.
//
// A golden whose DataVolume is in phase Failed is left out: CDI never
// populates it, so a disk cloned from it never becomes usable. One still
// importing stays, because CDI holds a clone until the source is populated.
// A golden whose import is failing is left out (see goldenImportFailing):
// CDI does not populate it, so a disk cloned from it never becomes usable.
// One still importing stays, because CDI holds a clone until the source is
// populated.
func imageProvider(dyn dynamic.Interface) providerFunc {
return func(ctx context.Context, _ string) ([]corev1alpha1.OptionItem, error) {
list, err := dyn.Resource(gvrPVCs).Namespace(publicImagesNamespace).List(ctx, listOpts())
@@ -353,7 +354,7 @@ func imageProvider(dyn dynamic.Interface) providerFunc {
logProviderError("image", err)
} else {
for i := range dvs.Items {
if phase, _, _ := unstructured.NestedString(dvs.Items[i].Object, "status", "phase"); phase == "Failed" {
if goldenImportFailing(&dvs.Items[i]) {
failed[dvs.Items[i].GetName()] = true
}
}
@@ -371,6 +372,33 @@ func imageProvider(dyn dynamic.Interface) providerFunc {
}
}
// goldenImportFailing reports a golden DataVolume whose import is failing.
// CDI sets phase Failed only on a fatal error such as a checksum mismatch; an
// unreachable URL restarts the importer with back-off while the phase stays
// in progress, so an importer that restarted at least three times and is not
// running now counts as failing too. The kubernetes-nodes chart applies the
// same test before cloning a golden; vm-disk refuses on the restarts alone.
func goldenImportFailing(dv *unstructured.Unstructured) bool {
phase, _, _ := unstructured.NestedString(dv.Object, "status", "phase")
if phase == "Failed" {
return true
}
if phase == "Succeeded" {
return false
}
restarts, _, _ := unstructured.NestedInt64(dv.Object, "status", "restartCount")
if restarts < 3 {
return false
}
conditions, _, _ := unstructured.NestedSlice(dv.Object, "status", "conditions")
for _, c := range conditions {
if cm, ok := c.(map[string]any); ok && cm["type"] == "Running" {
return cm["status"] == "False"
}
}
return false
}
// storagePoolProvider derives selectable pool names from the seaweedfs
// HelmRelease(s) in the request namespace (volume.pools and
// volume.zones.<zone>.pools map keys). Best-effort: returns nil when no
@@ -344,6 +344,41 @@ func TestImageProviderSkipsGoldensWhoseImportFailed(t *testing.T) {
}
}
func TestImageProviderSkipsGoldensWhoseImporterKeepsFailing(t *testing.T) {
pvcGVK := gvrPVCs.GroupVersion().WithKind("PersistentVolumeClaim")
dvGVK := gvrDataVolumes.GroupVersion().WithKind("DataVolume")
dv := func(name, phase string, restarts int64, running string) *unstructured.Unstructured {
o := newObj(dvGVK, publicImagesNamespace, name, nil)
_ = unstructured.SetNestedField(o.Object, phase, "status", "phase")
_ = unstructured.SetNestedField(o.Object, restarts, "status", "restartCount")
if running != "" {
_ = unstructured.SetNestedSlice(o.Object, []any{
map[string]any{"type": "Bound", "status": "True"},
map[string]any{"type": "Running", "status": running},
}, "status", "conditions")
}
return o
}
golden := func(name string) *unstructured.Unstructured {
return newObj(pvcGVK, publicImagesNamespace, "vm-default-images-"+name, nil)
}
dyn := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), listKinds(),
golden("stuck"), golden("recovered"), golden("flaky"), golden("done"), golden("norunning"),
dv("vm-default-images-stuck", "ImportInProgress", 3, "False"),
dv("vm-default-images-recovered", "ImportInProgress", 5, "True"),
dv("vm-default-images-flaky", "ImportInProgress", 2, "False"),
dv("vm-default-images-done", "Succeeded", 4, "False"),
dv("vm-default-images-norunning", "ImportInProgress", 3, ""),
)
items, err := DefaultProviders(dyn)["image"](context.Background(), "")
if err != nil {
t.Fatalf("image provider: %v", err)
}
if got := values(items); len(got) != 4 || got[0] != "done" || got[1] != "flaky" || got[2] != "norunning" || got[3] != "recovered" {
t.Fatalf("image: got %v, want [done flaky norunning recovered] (stuck golden dropped)", got)
}
}
func TestImageProviderListsGoldensWhenDataVolumesCannotBeRead(t *testing.T) {
for name, listErr := range map[string]error{
"no CDI": apierrors.NewNotFound(gvrDataVolumes.GroupResource(), ""),