Files
cozystack/packages/system/kubernetes-worker-image/values.yaml
T
Aleksei Sviridkin a09eded9ec fix(kubernetes-worker-image): wait for goldens to import before Ready
The Talos worker image catalog renders its goldens the same way the VM
image collection does: DataVolumes with immediate bind whose import
starts at install. With no health check the release went Ready as soon
as helm applied them, so a golden that never imported left the catalog
green.

Gate it on the DataVolume phase with the same expressions. Worker
pools read each golden directly and do not wait on the catalog status,
so the gate changes what the status reports and delays no pool.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-29 22:15:59 +03:00

108 lines
7.8 KiB
YAML

##
## @section Talos worker image catalog
##
## Pre-populates a catalog of Talos worker OS disk images as DataVolumes in the
## cozy-public namespace, so a tenant Kubernetes worker node group can boot by CDI
## csi-clone (a storage-layer copy) of a shared golden image instead of each worker
## independently importing the raw image over HTTP from the Talos Image Factory.
## One import per (schematicID, version) is then shared by every worker of every
## tenant node group that clones it — removing the per-worker Factory dependency
## (and the kubernetes-* e2e node-join flake, cozystack/cozystack#3231) on the hot
## path. Each images[] entry is one catalog (schematicID, version) flavor.
##
## This package is OPT-IN: it is registered through the platform's optional package
## helper and is disabled by default. Enable it by adding
## `cozystack.kubernetes-worker-image` to `bundles.enabledPackages` in the Cozystack
## config; without it no golden is created and every worker imports over HTTP.
##
## The package reports Ready only once every golden in images[] has imported. One
## golden that cannot be fetched keeps the whole package not Ready, retrying with
## no time limit, until it imports or its entry is removed. A golden that reached
## phase Failed does not import again by itself: delete it and the next retry
## recreates it. Pools do not wait on that status: they read each golden
## directly, as described below.
##
## A tenant node group opts into a clone explicitly by setting `osImage.builtin`
## (optionally overriding schematicID/version) on its own `KubernetesNodes` resource
## -- one per pool since the Phase 2 split, NOT `nodeGroups` on the parent `Kubernetes`
## CR, which is still accepted and stored there but has no effect. The pool then
## clones the golden named deterministically
## `talos-worker-<schematicID>-<version>` from cozy-public. The tenant render fails
## rather than degrade silently when the referenced (schematicID, version) has no
## matching entry in this catalog, when its golden terminally failed to import
## (`phase: Failed`), or when the node group's storageClass differs from the
## golden's — so keep the catalog in sync with the (schematicID, version) pairs and
## StorageClasses that tenant node groups reference. A golden that is merely
## mid-import is not an error: CDI holds the worker disk until the source populates.
## Goldens carry `helm.sh/resource-policy: keep`, so removing an `images[]` entry
## stops managing that golden rather than deleting it. Deleting one by hand while a
## worker pool still clones it fails that pool's render on its next values or chart
## change, and leaves an autoscaler scale-up months later with nothing to clone, so
## check before deleting rather than after.
##
## Adding an entry comes before anything that makes a pool resolve it. A pool that
## pins no version follows `talos.version` from the kubernetes-nodes chart, so
## bumping that default -- which a Cozystack upgrade does -- repoints every such
## pool at a golden that does not exist yet, and the pool's render fails on the
## missing entry until this catalog carries it. That state clears itself once the
## entry lands, since an app HelmRelease retries with no remediation cap, but the
## pool cannot provision a worker in the meantime: add the entry in the same change
## as the bump, not after it.
##
## The authority is the pool declarations, NOT the running disks. A pool sits at
## zero workers whenever the autoscaler has nothing to place — `minReplicas: 0` is
## the default — and it still pins its golden, so a query over live DataVolumes
## answers "empty" for exactly the pool that breaks on its next scale-up. Read the
## declarations first:
##
## kubectl get kubernetesnodes -A -o json | jq -r '.items[]
## | select(.spec.osImage.builtin != null)
## | "\(.metadata.namespace)/\(.metadata.name)\tschematic=\(.spec.osImage.builtin.schematicID // "<talos.schematicID>")\tversion=\(.spec.osImage.builtin.version // "<talos.version>")"'
##
## A pool that leaves either field unset inherits it from its own `talos.*`, which
## is why the fallbacks print as placeholders rather than as values — resolve those
## against that pool's `talos` block before concluding it pins a different golden.
##
## The live disks are the second read, not the first. Every cloned worker disk
## annotates the golden it came from, which is what tells you whether a clone is in
## flight right now:
##
## kubectl get datavolume -A -o json | jq -r '.items[]
## | select(.metadata.annotations["kubernetes-worker-image.cozystack.io/golden"]
## == "talos-worker-<schematicID>-<version>")
## | "\(.metadata.namespace)/\(.metadata.name)"'
##
## Both empty is the go-ahead. The first non-empty on its own is enough to stop.
##
## Worker pools that do not set `osImage.builtin` import over HTTP and never touch
## this catalog.
##
## An entry is effectively immutable once its golden exists. The DataVolume is named
## from `(schematicID, version)` alone, while `storage`, `storageClass` and the
## source URL are rendered into a spec Kubernetes does not allow to be patched in
## place — so editing one of those on an existing entry produces a rejected patch on
## the next HelmRelease upgrade, and the only way out is deleting the golden, which
## runs straight into the check above. Treat a size, class or URL change as a NEW
## `(schematicID, version)` entry, move the pools onto it, then retire the old one.
## @param {string} imageFactoryURL - Default base URL of the Talos Image Factory serving the raw worker OS image (`<url>/image/<schematicID>/<version>/openstack-amd64.raw.xz`). Overridable per image. No trailing slash.
imageFactoryURL: "https://factory.talos.dev"
## @param {string} storageClass - Default StorageClass for the golden image(s). Must match the StorageClass of the worker disks that clone it: CDI cannot CSI-clone across StorageClasses and would silently fall back to a host-assisted copy over the pod network, so the tenant render rejects a mismatch outright. Overridable per image.
storageClass: "replicated"
## @param {quantity} storage - Default storage size allocated for each golden image. Must be >= the imported raw image virtual size and <= the smallest worker diskSize that clones it. The Talos v1.13.x openstack raw is ~4.15 GiB, so 6Gi leaves headroom. This size is also the unit of the transient cost every clone pays: CDI populates a worker disk larger than the golden by cloning at the golden's size and then growing it, so one worker in flight occupies the golden plus a temporary clone PVC plus its own diskSize, times the replica count of a replicated StorageClass. Keeping this at the raw image's size rather than rounding it up is therefore worth doing -- it is paid once per golden and again per concurrent clone. Overridable per image, but only when the entry is added: a DataVolume spec cannot be patched in place, so raising this on an entry whose golden already exists is rejected on the next upgrade — a Talos version whose raw outgrows it needs a new `images[]` entry, which it gets anyway since the name is keyed on the version.
storage: "6Gi"
## @typedef {struct} WorkerImage - A golden Talos worker image entry.
## @field {string} schematicID - Talos image-factory schematic ID.
## @field {string} version - Talos version (e.g. "v1.13.6").
## @field {string} [imageFactoryURL] - Override the global imageFactoryURL for this image.
## @field {quantity} [storage] - Override the global storage size for this image.
## @field {string} [storageClass] - Override the global StorageClass for this image.
## @param {[]WorkerImage} images - Catalog of golden Talos worker images to provision in the cozy-public namespace. Each entry is one (schematicID, version) flavor a tenant node group can clone via `osImage.builtin`.
images:
- schematicID: "ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515"
version: "v1.13.6"