The computeplane module deploys the kubernetes chart into the tenant
namespace, and that chart's dashboard Role lets the tenant's "use"
subjects read <release>-admin-kubeconfig. Withholding the Secret from
the tenant secrets view does not help when RBAC grants it directly, so
any tenant user at "use" level or above could fetch the cluster-admin
credential of a cluster the module promises to keep operator-owned.
The kubernetes chart now drops that rule when the internal value
withholdAdminKubeconfig is set, and the module sets it in the cluster
HelmRelease's inline values. The tenant cannot reach those values: the
module reads only nodeGroups from them and splices an allowlisted
subset into the pool releases, and Flux applies spec.values over
valuesFrom. A regular kubernetes app keeps the rule, so its dashboard
can still offer the kubeconfig.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Every tenant worker VM imports the same Talos raw image from the Image Factory
over HTTP, once per worker: a 20-worker cluster fetches the identical ~4.15 GiB
artifact 20 times, and each fetch is a live dependency on the Factory at exactly
the moment a node joins.
packages/system/kubernetes-worker-image is a new opt-in catalog that holds one
golden DataVolume per (schematicID, version) in cozy-public, named
talos-worker-<schematicID>-<version>. A worker pool opts in with osImage.builtin
on its own KubernetesNodes resource, and its disk-system DataVolume then carries
source.pvc pointing at that golden, so CDI performs a storage-layer CSI clone
instead of an HTTP import -- one import per flavor, shared by every worker of
every tenant. Confirmed on a LINSTOR-backed cluster: the worker disk is a CSI
clone plus a resize, with no host-assisted upload pod.
Both halves are off unless asked for. The catalog is registered through the
platform's optional-package helper and disabled by default, so a cluster that
does not enable it creates no golden and keeps importing over HTTP. The disk
source is derived entirely from osImage, which is absent by default, so a pool
that does not set it renders byte-identical to the pre-feature chart and
adopting the feature never rolls existing workers. Editing osImage on an
existing pool re-images that pool, as changing any other image reference does.
What the render refuses, rather than degrading into a silent fallback:
- A pool whose golden has no catalog entry, or whose golden terminally failed to
import. A golden that is merely mid-import is not an error; CDI holds the
worker disk until the source populates.
- A pool whose StorageClass differs from its golden's. CDI cannot CSI-clone
across classes and falls back to a host-assisted copy over the pod network,
which is the transfer this feature exists to remove. An empty class on either
side means the cluster default and is resolved before the comparison, and a
pool that has already cloned reads its class from its own bound PVC rather
than from today's default -- the golden's side is frozen the same way, and an
admin moving storageclass.kubernetes.io/is-default-class must not fail the
release of a pool whose disks are all on the right class. That resolved class
is then written onto the worker disk, so the disk binds where the guard
checked instead of on whatever the default has become by then -- reading one
class and binding another is how a pool at an empty storageClass passed the
comparison and still took the host-assisted copy. The HTTP path leaves an
empty class to the cluster as before, which is what keeps its render, and the
KubevirtMachineTemplate hash named after it, byte-identical.
- A diskSize below the golden's storage size, which a clone target cannot be.
- A schematicID that is not a plain lowercase alphanumeric identifier, or a
version that is not a vMAJOR.MINOR.PATCH release. Both are interpolated into
a DataVolume name and an image URL, and values.schema.json types them as bare
strings, so a malformed one otherwise reaches the render. The catalog applies
the same patterns, since both sides build the same name from the same fields.
- An images[] entry edited after its golden exists. A DataVolume spec cannot be
patched in place, so the message names the entry and the supported move -- a
new (schematicID, version) -- instead of leaving the API server to reject the
patch on the next upgrade.
Goldens carry helm.sh/resource-policy: keep, so dropping an entry stops
managing a golden rather than deleting it: a missing golden aborts the whole
Kubernetes HelmRelease of every tenant pinned to it, not just the disk. Each
cloned worker disk annotates the golden it came from and the pool that owns it,
so "does anything still clone this?" is answerable before a manual delete
instead of after.
A builtin pool needs a StorageClass whose volumes are reachable from any node,
in practice a replicated/DRBD one. The golden in cozy-public has no consumer and
binds as soon as it imports, wherever CDI's importer landed, while a worker disk
binds where its VM is scheduled, and nothing here arranges for those to be the
same node. The platform ships cloneStrategyOverride: csi-clone, which rules out
the host-assisted strategy that would cross nodes. What a node-pinned class does
in that situation is not measured, so values.yaml states the requirement and not
a symptom.
ComputePlane pools forward only the per-pool shape fields, and osImage is
excluded there by name: the worker OS image, its Talos installer and its Image
Factory are operator decisions about the platform's own images.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The nodeGroups docs still described the old denylist behaviour: "unlisted
NodeGroup fields pass through to the wrapped chart unvalidated". After the switch
to an allowlist (pick) both halves are wrong: fields go to the kubernetes-nodes
pool chart (not the wrapped kubernetes chart), and anything the pick does not name
is dropped, not passed through. A tenant that sets e.g. talos.version on a
nodeGroup gets no error and no effect (accepted-but-inert). Correct the source
@param and the hand-written README intro, regenerate schema/README/cozyrd, and
refresh the stale "omit" wording in the splice test comment to say allowlist.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The pool key becomes the pool HelmRelease name and its CAPI object names, which
must be RFC-1123 labels, but only the length was guarded. An uppercase/underscore
key rendered an invalid metadata.name the apiserver rejects, and a key containing
':' rendered invalid YAML that failed the whole module render -- a non-uniform,
late failure next to the guarded length case. Add a render-time RFC-1123 guard on
the key alongside the length guard (mirrors migration 56's child-name guard), and
pin the length guard's derived remedy budget (15) in the overflow test so a wrong
computation is caught, not just the presence of the error.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The pool-value splice used omit (a denylist), which leaks every key it does not
name. Since this module runs untrusted tenant code and the NodeGroup schema has
no additionalProperties:false, a tenant could pass the platform-injected
_cluster object (cluster-domain/oidc/dns01/proxy, delivered via cozystack-values
and merged UNDER spec.values, so a tenant key wins) or nameOverride straight
through to the pool chart -- and any future kubernetes-nodes top-level value
would silently re-open the same hole. Switch to pick: pass through only the
enumerated per-pool shape fields, so everything cluster-level or module-owned is
dropped by construction (fail-closed for new fields). Also derive the
length-guard's release-name prefix from $clusterRelease so it cannot drift from
the name actually rendered. Regression-tested: _cluster and nameOverride no
longer reach the child values; maxUnhealthy and the module wiring still do.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The per-pool child HelmRelease spliced the tenant nodeGroup dict verbatim
(toYaml) after setting cluster/clusterReleaseName, and the NodeGroup schema
has no additionalProperties:false. A tenant could therefore (a) set a
duplicate cluster/clusterReleaseName key that wins the YAML merge and
re-parents the pool onto a tenant-controlled cluster, or (b) pass talos/
version/images through a shape-only knob and pick the worker OS image,
installer and kubelet version. omit those five keys at the splice so only the
pool's own shape flows through (tolerated extras like maxUnhealthy still do).
Separately, each pool renders a 38-char fixed release-name prefix against
Helm's 53-char cap, so a nodeGroup key over 15 chars created a HelmRelease the
apiserver accepts but Flux never reconciles, with no other signal. Guard the
length at render time, next to the ingress-role guard, mirroring the migration
overflow branch and the aggregated API's validateNameLength.
Also refresh the generated values doc: the module now materialises its own
default md0 and pool storageClass falls back to the kubernetes-nodes chart
default, not the wrapped kubernetes app.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Phase 2 moved worker nodeGroups out of apps/kubernetes into apps/kubernetes-nodes,
but the ComputePlane module still passed nodeGroups to the wrapped kubernetes
chart, which now ignores it: a fresh ComputePlane cluster came up with zero
workers (and the pinned ingress-nginx addon stuck Pending), while existing ones
were unreachable by migration 56 (it only adopts kubernetes-* releases, not
computeplane-cluster).
The module now emits one kubernetes-nodes child HelmRelease per nodeGroup,
shaped like a natively created pool, and materialises the pre-split default md0
(roles: [ingress-nginx]) when nodeGroups is empty. The pool chart's clusterName
gains an internal clusterReleaseName override so pools attach to the module's
fixed computeplane-cluster release, which is off the aggregated API's
kubernetes-<cluster> convention; the default keeps every API-created pool
byte-identical.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The fluxcd addon pin was a holdover from the #3150-era model where app
charts rendered nested HelmReleases into the target cluster and needed an
in-cluster helm-controller. Consumers now keep their releases on the
management cluster and remote-apply via spec.kubeConfig, so the management
Flux resolves charts and applies manifests — nothing inside the ComputePlane
reconciles HelmReleases. This also decouples the module from the FluxCD
addon removal (#3379). certManager and ingressNginx stay pinned: Certificate
and Ingress objects land inside the cluster and need their controllers
there.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Gate the Proxied ingress Service on _namespace.etcd like the addon
HelmRelease it fronts, so an etcd-less tenant with the addon pinned on does
not get a selector-less Service (+ test). Correct the secrets framing: the
tenant-visible surface is an allowlist (include: [] withholds everything),
so drop the never-load-bearing exclude entry, the bats cross-check and the
rename-coupling comments, and state the real mechanism. Fix the etcd
prerequisite docs (an ancestor's etcd satisfies the wrapped chart; the
awaiting-etcd beacon is the diagnostic), reword the tenant toggle to stop
advertising the not-yet-shipped placement routing, require maxReplicas >= 1
on the ingress-role node group, and fix comment drift (Service name after
the release rename, placement-hook label, package-guard lookup semantics).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The tenant Kubernetes machinery — including the cluster-autoscaler that
brings up the default scale-from-zero md0 group — is backed by the tenant's
etcd module, so a ComputePlane without it provisions no workers and gives no
diagnostic; state the prerequisite in the README. Also state plainly that
unlisted NodeGroup fields pass through to the wrapped chart unvalidated (the
schema generator cannot close the object), with a test pinning that
behaviour.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The aggregated API cannot represent spec.releaseName: the conversion
rebuilds HelmRelease specs without it, so the first tenant edit through the
API dropped the module's explicit computeplane-module release name, let it
default to the object name computeplane, and collided with the Helm storage
record the cluster release owned — with the tenant chart writing the old
name back every interval. Drop releaseName from both HelmReleases and derive
names from object names, like every other tenant module. Kamaji now writes
the admin kubeconfig to computeplane-cluster-admin-kubeconfig; the
computeplane-rd secrets.exclude entry follows in the same change, and a bats
test pins the exclusion to the name the cluster release actually produces so
a future rename cannot silently re-expose the credential. Also add the
check-release-name guard the other extra modules carry, and pin the fluxcd
addon's distribution artifact to the flux-operator version vendored in
system/fluxcd-operator instead of :latest.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
nodeGroups was a bare object: typos surfaced as a stuck release deep in the
wrapped chart instead of an admission error, and groups without the
ingress-nginx role silently left the pinned ingress-nginx controller (and
with it the ComputePlane ingress) Pending forever, since supplied groups are
not merged with the wrapped chart's default md0. Reuse the kubernetes app's
nodeGroups typedefs for the schema and refuse rendering when no group
carries the role.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Add the operator-owned computeplane tenant module per the revised design
(cozystack/community design-proposals/compute-plane): an extra/computeplane
chart that provisions a single-tenant, Cozystack-managed cluster for
untrusted-code apps by deploying the ordinary apps/kubernetes chart with
operator-fixed values (fluxcd, cert-manager, ingress-nginx Proxied). The
package re-declares apps/kubernetes as its own source-only component, so the
module is self-contained; the cluster release is named computeplane so Kamaji
writes the admin kubeconfig to computeplane-admin-kubeconfig — the contract
consumed by placement: ComputePlane applications. The tenant holds no admin
kubeconfig and can set cluster shape (nodeGroups) only.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>