diff --git a/packages/apps/kubernetes-nodes/templates/nodegroup.yaml b/packages/apps/kubernetes-nodes/templates/nodegroup.yaml index 9c99eb551..eb705cbcd 100644 --- a/packages/apps/kubernetes-nodes/templates/nodegroup.yaml +++ b/packages/apps/kubernetes-nodes/templates/nodegroup.yaml @@ -107,6 +107,11 @@ spec: annotations: kubevirt.io/allow-pod-bridge-network-live-migration: "true" labels: + {{- /* The VM pod is intentionally attributed to the parent Kubernetes + application (kind Kubernetes, name ), not to this + KubernetesNodes release: these labels are part of the byte-identical + render the adoption migration relies on, and lineage/quota accounting + for worker VMs stays under the cluster they belong to. */}} apps.cozystack.io/application.group: apps.cozystack.io apps.cozystack.io/application.kind: Kubernetes apps.cozystack.io/application.name: {{ .clusterName | trimPrefix "kubernetes-" | quote }} diff --git a/packages/apps/kubernetes-nodes/templates/talos-reconcile-job.yaml b/packages/apps/kubernetes-nodes/templates/talos-reconcile-job.yaml index bca5af9d7..b56146001 100644 --- a/packages/apps/kubernetes-nodes/templates/talos-reconcile-job.yaml +++ b/packages/apps/kubernetes-nodes/templates/talos-reconcile-job.yaml @@ -3,6 +3,15 @@ and patches the parent KamajiControlPlane.spec.network.certSANs with the live apiserver Service ClusterIP. + The ClusterIP certSAN is a best-effort extra, not a worker-join requirement: + Talos workers dial the apiserver by DNS name (endpoint https://..svc + plus extraHostEntries below), so a cluster with zero pools — which never runs + this Job — is fine without the IP in the cert. A parent upgrade array-replaces + certSANs back to the two DNS names (a KamajiControlPlane merge-patch limitation + that predates the split); the next pool reconcile re-adds the ClusterIP. If a + future consumer ever needs the IP SAN on a pool-less control plane, that patch + belongs in the control-plane (kubernetes) chart, not here. + The Talos worker machineconfig needs values that only exist AFTER the parent control-plane release has applied: the apiserver Service ClusterIP (emitted by Kamaji), the Talos-CA Secret (issued by cert-manager from the parent's Talos diff --git a/packages/apps/kubernetes/README.md b/packages/apps/kubernetes/README.md index ea271cfb1..c496b367a 100644 --- a/packages/apps/kubernetes/README.md +++ b/packages/apps/kubernetes/README.md @@ -95,6 +95,8 @@ See the reference for components utilized in this service: - **Remote-accessible LINSTOR StorageClasses are auto-propagated to tenants** (v1.5): infra-cluster LINSTOR StorageClasses whose `linstor.csi.linbit.com/allowRemoteVolumeAccess` is not `"false"` are created inside each tenant under the same name (provisioned by `csi.kubevirt.io`). **Upgrade caveat:** delete any *manually created* tenant StorageClass whose name collides with a propagated infra class (e.g. a hand-made `replicated`) before upgrading, or the propagated class cannot be created and the tenant CSI release will not converge. Infra classes that must stay node-local need `allowRemoteVolumeAccess: "false"` set explicitly — an absent annotation is treated as remote-accessible. Propagation is evaluated at HelmRelease render time, so classes added or removed on the infra cluster after a tenant exists propagate only on that tenant's next reconcile. +- **Worker pools split into `KubernetesNodes` (Phase 2)**: `spec.nodeGroups` is removed from the `Kubernetes` CR — worker pools are now separate `KubernetesNodes` resources (one HelmRelease per pool; see the `kubernetes-nodes` chart). Existing pools are adopted automatically on upgrade by platform migration 52 with no worker-VM churn. Consequences: (1) the `Kubernetes` CR's `WorkloadsReady` condition is now **control-plane-only** — worker health is reported on each `KubernetesNodes` resource, so a dashboard or `kubectl wait` watching the parent no longer reflects worker readiness; (2) enabling `addons.ingressNginx` now requires a pool carrying `roles: [ingress-nginx]` (the implicit default `md0` is gone — without such a pool the controller DaemonSet schedules nothing); (3) do **not** edit `Kubernetes` CRs during a platform upgrade — a mid-window parent render can transiently fail Helm ownership validation while migration 52 re-annotates worker objects, and self-heals once the new chart artifact lands. + > The top-level `storageClass` field is annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it. The per-pool worker `storageClass` now lives on the `KubernetesNodes` resource (see the `kubernetes-nodes` chart). ## Parameters