Myasnikov Daniil
bc8c54d6e6
Every tenant worker VM imports the same Talos raw image from the Image Factory over HTTP, once per worker: a 20-worker cluster fetches the identical ~4.15 GiB artifact 20 times, and each fetch is a live dependency on the Factory at exactly the moment a node joins. packages/system/kubernetes-worker-image is a new opt-in catalog that holds one golden DataVolume per (schematicID, version) in cozy-public, named talos-worker-<schematicID>-<version>. A worker pool opts in with osImage.builtin on its own KubernetesNodes resource, and its disk-system DataVolume then carries source.pvc pointing at that golden, so CDI performs a storage-layer CSI clone instead of an HTTP import -- one import per flavor, shared by every worker of every tenant. Confirmed on a LINSTOR-backed cluster: the worker disk is a CSI clone plus a resize, with no host-assisted upload pod. Both halves are off unless asked for. The catalog is registered through the platform's optional-package helper and disabled by default, so a cluster that does not enable it creates no golden and keeps importing over HTTP. The disk source is derived entirely from osImage, which is absent by default, so a pool that does not set it renders byte-identical to the pre-feature chart and adopting the feature never rolls existing workers. Editing osImage on an existing pool re-images that pool, as changing any other image reference does. What the render refuses, rather than degrading into a silent fallback: - A pool whose golden has no catalog entry, or whose golden terminally failed to import. A golden that is merely mid-import is not an error; CDI holds the worker disk until the source populates. - A pool whose StorageClass differs from its golden's. CDI cannot CSI-clone across classes and falls back to a host-assisted copy over the pod network, which is the transfer this feature exists to remove. An empty class on either side means the cluster default and is resolved before the comparison, and a pool that has already cloned reads its class from its own bound PVC rather than from today's default -- the golden's side is frozen the same way, and an admin moving storageclass.kubernetes.io/is-default-class must not fail the release of a pool whose disks are all on the right class. That resolved class is then written onto the worker disk, so the disk binds where the guard checked instead of on whatever the default has become by then -- reading one class and binding another is how a pool at an empty storageClass passed the comparison and still took the host-assisted copy. The HTTP path leaves an empty class to the cluster as before, which is what keeps its render, and the KubevirtMachineTemplate hash named after it, byte-identical. - A diskSize below the golden's storage size, which a clone target cannot be. - A schematicID that is not a plain lowercase alphanumeric identifier, or a version that is not a vMAJOR.MINOR.PATCH release. Both are interpolated into a DataVolume name and an image URL, and values.schema.json types them as bare strings, so a malformed one otherwise reaches the render. The catalog applies the same patterns, since both sides build the same name from the same fields. - An images[] entry edited after its golden exists. A DataVolume spec cannot be patched in place, so the message names the entry and the supported move -- a new (schematicID, version) -- instead of leaving the API server to reject the patch on the next upgrade. Goldens carry helm.sh/resource-policy: keep, so dropping an entry stops managing a golden rather than deleting it: a missing golden aborts the whole Kubernetes HelmRelease of every tenant pinned to it, not just the disk. Each cloned worker disk annotates the golden it came from and the pool that owns it, so "does anything still clone this?" is answerable before a manual delete instead of after. A builtin pool needs a StorageClass whose volumes are reachable from any node, in practice a replicated/DRBD one. The golden in cozy-public has no consumer and binds as soon as it imports, wherever CDI's importer landed, while a worker disk binds where its VM is scheduled, and nothing here arranges for those to be the same node. The platform ships cloneStrategyOverride: csi-clone, which rules out the host-assisted strategy that would cross nodes. What a node-pinned class does in that situation is not measured, so values.yaml states the requirement and not a symptom. ComputePlane pools forward only the per-pool shape fields, and osImage is excluded there by name: the worker OS image, its Talos installer and its Image Factory are operator decisions about the platform's own images. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
ComputePlane
A managed, isolated environment for running code-executing (untrusted-code) catalog applications — notebooks, workflow "code" nodes, plugin systems.
This tenant module provisions a single-tenant, Cozystack-managed cluster by
deploying the ordinary apps/kubernetes chart (Kamaji control plane +
KubeVirt-VM worker nodes) with operator-fixed values, sourced from this
package's own source-only ExternalArtifact
(cozystack-computeplane-application-kubevirt-kubernetes). The tenant gets the
same substrate as any managed kind: Kubernetes cluster, but holds no admin
kubeconfig for it. The security posture (Cozystack-enablement addons, withheld
credentials) is baked into the chart and cannot be overridden; the one
tenant-facing knob is nodeGroups — the module forwards only the documented per-pool fields to the pool chart, so any other NodeGroup field (e.g. talos, version, images) is silently dropped: accepted but inert.
The cluster HelmRelease is named computeplane-cluster and its release name
is derived from it, so Kamaji writes the admin kubeconfig to the Secret
computeplane-cluster-admin-kubeconfig (key super-admin.svc) — the derived
contract consumed by the HelmReleases of placement: ComputePlane
applications. The tenant never sees that Secret: tenant-visible secrets are an
allowlist, and this module's ApplicationDefinition includes nothing, which
withholds every Secret in the module's lineage by construction.
Prerequisite: an etcd module must be available to the tenant — its own,
or one inherited from an ancestor tenant (the _namespace.etcd context
propagates down the tenant tree, so enabling a redundant local etcd is not
required). The tenant Kubernetes machinery — including the cluster-autoscaler
that brings up the default scale-from-zero md0 node group — is backed by
that etcd; without it the wrapped chart collapses to its awaiting-etcd
status beacon and the ComputePlane provisions no control plane or workers
(the HelmReleases still report Ready over the empty shell, so check for the
beacon ConfigMap when nothing comes up).
See the full design in cozystack/community design-proposals/compute-plane.
Parameters
Cluster shape
| Name | Description | Type | Value |
|---|---|---|---|
nodeGroups |
Worker node groups of the ComputePlane cluster (cluster shape only — the security posture is fixed by the module and cannot be overridden through this knob: the module forwards only the documented per-pool fields to the kubernetes-nodes pool chart, so any other NodeGroup field, e.g. talos/version/images, is silently dropped — accepted but inert). Same structure as the kubernetes application's nodeGroups. When left empty, the module itself materialises a default scale-from-zero md0 pool with roles: [ingress-nginx] (the wrapped kubernetes chart no longer injects one after the Phase 2 split), so the cluster sits idle until a pending Pod (e.g. the ingress-nginx controller) triggers the cluster-autoscaler. Supplied groups are authoritative (not merged with the default), so at least one of them must carry roles: [ingress-nginx] — enforced at render time. Each supplied pool key must be at most 15 characters so its kubernetes-nodes-computeplane-cluster-<pool> HelmRelease name fits Helm's 53-character limit — also enforced at render time. |
map[string]object |
{} |
nodeGroups[name].minReplicas |
Minimum number of replicas. | int |
0 |
nodeGroups[name].maxReplicas |
Maximum number of replicas. | int |
10 |
nodeGroups[name].instanceType |
Virtual machine instance type. | string |
u1.medium |
nodeGroups[name].diskSize |
System disk size for the worker VM. Carries the Talos OS image, kubelet state, containerd image cache, and any local-path PVCs. | quantity |
20Gi |
nodeGroups[name].storageClass |
StorageClass for worker node persistent disks. When empty, the kubernetes-nodes pool chart applies its own default (replicated). |
string |
"" |
nodeGroups[name].roles |
List of node roles. At least one group must carry ingress-nginx — the module pins the ingress-nginx addon on, and its controller only schedules onto nodes with that role (enforced at render time). |
[]string |
[] |
nodeGroups[name].resources |
Explicit CPU and memory for each worker node, as an alternative to instanceType sizing. Set both cpu and memory together or neither. |
object |
{} |
nodeGroups[name].resources.cpu |
CPU available. | quantity |
"" |
nodeGroups[name].resources.memory |
Memory (RAM) available. | quantity |
"" |
nodeGroups[name].gpus |
List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM). | []object |
[] |
nodeGroups[name].gpus[i].name |
Name of GPU, such as "nvidia.com/AD102GL_L40S". | string |
"" |
nodeGroups[name].kubelet |
Kubelet resource reservations for this node group. | object |
{} |
nodeGroups[name].kubelet.systemReservedMemory |
Memory reserved for host OS. Auto-computed from instanceType if empty. | string |
"" |
nodeGroups[name].kubelet.kubeReservedMemory |
Memory reserved for kubelet and container runtime. Auto-computed from instanceType if empty. | string |
"" |
nodeGroups[name].kubelet.systemReservedCpu |
CPU reserved for host OS. Auto-computed from instanceType if empty. | string |
"" |
nodeGroups[name].kubelet.kubeReservedCpu |
CPU reserved for kubelet and container runtime. Auto-computed from instanceType if empty. | string |
"" |
nodeGroups[name].kubelet.evictionHardMemory |
Hard eviction threshold for memory (absolute like 200Mi or percentage like 7%). | string |
7% |
nodeGroups[name].kubelet.evictionSoftMemory |
Soft eviction threshold for memory (absolute like 1Gi or percentage like 10%). | string |
10% |