Files
cozystack/packages/apps/kubernetes-nodes/tests/nodegroup_test.yaml
T
Mattia Eleuteri 9afe7f1293 fix(kubernetes): assemble the node group once in kubernetes-nodes
The TalosConfigTemplate name is a hash of the machine-config spec, and two
templates have to agree on it: talos-reconcile-job.yaml creates the object,
nodegroup.yaml points MachineDeployment.bootstrap.configRef at it. Sharing the
rendering was not enough — each file also assembled its own $group dict, one
carrying every pool value and one only the few the spec reads, so the two
names agreed only by coincidence of the keys the spec helper happens to read.
Dropping "gpus" from the Job's dict diverged them on a GPU pool with the whole
suite still green, which is the "templates do not exist" deadlock this fix
exists to make impossible.

Move the assembly into a single kubernetes-nodes.group helper, read back with
fromYaml at both call sites, so there is one input as well as one rendering.

Cover it too, since the seam can be reintroduced. The chart's helm-unittest
suites pin each name against its own literal in its own file, which catches a
divergence but reports it as one literal having moved — indistinguishable from
a stale fixture, so re-pasting the new value is the obvious and wrong repair.
hack/check-talosconfigtemplate-name-agreement.bats renders the chart once and
compares the two values against each other, so the failure states the invariant
instead of a number. It asserts on a GPU-shaped pool as well as the default
one: the default renders gpus [], kubelet {} and instanceType "" as falsy, so a
consumer reading a subset of the group is indistinguishable there from one
reading all of it. kernelModules gets a case of its own, because it has no
default and stays unset on both of those shapes. A last case pins that the
shapes hash differently, so the others cannot hold vacuously.

The same guard compares the talos-reconcile Role's resourceNames on
TalosConfigTemplates against that name. The Job applies server-side, and
that is a PATCH on the name even when it creates the object, so a Role
naming anything else leaves the Job Forbidden and the pool without a
template, with the same silent outcome.

Document that talos.registryMirrors and kernelModules sit inside the hashed
spec, so changing either rolls the pool's existing workers instead of only
affecting nodes created later. The kernelModules text described the
fixed-name template as the present state and the content-hash name as a
future one.

No rendered output changes: the golden render snapshots are unchanged.

Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
Assisted-by: LLM
2026-09-30 08:59:03 +02:00

148 lines
4.7 KiB
YAML

suite: worker node pool rendering
templates:
- templates/nodegroup.yaml
release:
name: kubernetes-nodes-myk8s-md0
namespace: tenant-test
set:
cluster: myk8s
_cluster:
cluster-domain: cozy.local
version: "v1.35"
minReplicas: 0
maxReplicas: 3
instanceType: ""
diskSize: 20Gi
storageClass: replicated
roles:
- ingress-nginx
resources:
cpu: "2"
memory: 4Gi
tests:
- it: names the MachineDeployment after the parent cluster and pool
documentSelector:
path: kind
value: MachineDeployment
asserts:
- equal:
path: metadata.name
value: kubernetes-myk8s-md0
- equal:
path: spec.clusterName
value: kubernetes-myk8s
- it: wires the MachineDeployment to the parent CAPI cluster via labels
documentSelector:
path: kind
value: MachineDeployment
asserts:
- equal:
path: spec.selector.matchLabels["cluster.x-k8s.io/cluster-name"]
value: kubernetes-myk8s
- equal:
path: spec.template.metadata.labels["cluster.x-k8s.io/deployment-name"]
value: kubernetes-myk8s-md0
- equal:
path: spec.template.metadata.labels["node-role.kubernetes.io/ingress-nginx"]
value: ""
- it: carries both cluster-autoscaler node-group size annotations and omits replicas
documentSelector:
path: kind
value: MachineDeployment
asserts:
- equal:
path: metadata.annotations["cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size"]
value: "0"
- equal:
path: metadata.annotations["cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size"]
value: "3"
- notExists:
path: spec.replicas
# The TalosConfigTemplate spec is immutable, so the object can only ever be
# created, never updated: the name carries a content hash of the machine config
# it holds (cozystack/cozystack#3515). The Job in talos-reconcile-job.yaml
# applies it under the same name — see the TCT_NAME assertion below.
- it: points the MachineDeployment bootstrap at the pool TalosConfigTemplate
documentSelector:
path: kind
value: MachineDeployment
asserts:
- equal:
path: spec.template.spec.bootstrap.configRef.name
value: kubernetes-myk8s-md0-ffdbe1
- matchRegex:
path: spec.template.spec.bootstrap.configRef.name
pattern: ^kubernetes-myk8s-md0-[0-9a-f]{6}$
- equal:
path: spec.template.spec.bootstrap.configRef.kind
value: TalosConfigTemplate
# Paired with the GPU-shaped test in talos_reconcile_job_test.yaml: a
# machine-config change rotates the name, and both templates must rotate it
# to the same value. See that file for why the fixture carries a truthy
# `gpus` rather than reusing the all-falsy defaults above.
- it: a machine-config change rotates the name the MachineDeployment references
documentSelector:
path: kind
value: MachineDeployment
set:
gpus:
- name: nvidia.com/GH200
asserts:
# Same literal as the paired test in talos_reconcile_job_test.yaml.
- equal:
path: spec.template.spec.bootstrap.configRef.name
value: kubernetes-myk8s-md0-0f8081
- it: names the KubevirtMachineTemplate with the parent cluster prefix and a content hash
documentSelector:
path: kind
value: KubevirtMachineTemplate
asserts:
- matchRegex:
path: metadata.name
pattern: ^kubernetes-myk8s-md0-[0-9a-f]{6}$
- it: emits a worker WorkloadMonitor selecting the pool
documentSelector:
path: kind
value: WorkloadMonitor
asserts:
- equal:
path: spec.type
value: worker
- equal:
path: spec.selector["cluster.x-k8s.io/cluster-name"]
value: kubernetes-myk8s
---
suite: worker node pool validation
templates:
- templates/nodegroup.yaml
tests:
- it: fails when cluster is not set
release:
name: kubernetes-nodes-myk8s-md0
namespace: tenant-test
set:
cluster: ""
_cluster:
cluster-domain: cozy.local
asserts:
- failedTemplate:
errorMessage: "kubernetes-nodes: .Values.cluster is required — set it to the parent Kubernetes CR name so the pool attaches to cluster kubernetes-<cluster>"
- it: fails when the release name is not prefixed with the cluster name
release:
name: kubernetes-nodes-wrong-md0
namespace: tenant-test
set:
cluster: myk8s
_cluster:
cluster-domain: cozy.local
asserts:
- failedTemplate:
errorMessage: 'kubernetes-nodes: release name "kubernetes-nodes-wrong-md0" must start with "kubernetes-nodes-myk8s-" — name the KubernetesNodes CR <cluster>-<pool> (cluster="myk8s")'