feat(kubernetes): configure per-pool Talos sysctls (#5254)

<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, security (a fix that comes through a security
advisory), docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Fixes merged to `main` reach the release lines the backport batch runs
for automatically, so do not open backport PRs by hand:
https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md
-->

## What this PR does

Adds an optional `KubernetesNodes.spec.talos.sysctls` string map,
rendered into `machine.sysctls` on that pool's guest workers. This lets
operators configure the user namespaces needed by the Talos gVisor
extension without adding a dedicated API field for each future sysctl.

Replaces the proposal in #4650, following [the suggestion to expose a
general sysctl
map](https://github.com/cozystack/cozystack/pull/4650#issuecomment-6743092045).
This is a new branch from `main`, not a stack on the original PR. The
suggestion is not treated as maintainer consensus on the broader API.

```yaml
spec:
  talos:
    sysctls:
      user.max_user_namespaces: "11255"
```

- Empty or omitted maps preserve the existing rendered machine
configuration and bootstrap hash. A literal `"0"` is an explicit
override; remove the entry to restore the Talos default.
- The existing content-hashed TalosConfigTemplate mechanism updates the
MachineDeployment bootstrap reference when overrides change. No new
rollout mechanism is introduced.
- Both keys and values are YAML-quoted, then shell-escaped for
backslashes, dollars and backticks before entering the unquoted
reconcile Job heredoc. Quoting scalars individually also keeps newlines
and Unicode line separators from breaking the surrounding YAML or
heredoc delimiter.
- Both the generated schema and a render-time guard enforce string
values. The template guard is tested with schema validation disabled
too.
- Regenerates the schema, parameter reference, Go API type/deepcopy and
ApplicationDefinition. The README table alignment change is generator
output.

This changes only the selected pool's guest configuration, not the
management hosts, control plane or other pools. Supported sysctls and
their semantics remain Talos/kernel-dependent; inappropriate settings
can weaken guest isolation or prevent boot. Installing the gVisor
extension, configuring its RuntimeClass and enforcing workload/network
restrictions remain separate steps.

### Validation

- `helm unittest packages/apps/kubernetes-nodes`: 40 suites, 245 tests
and 12 existing snapshots passed, without updating snapshots.
- `bats hack/check-talosconfigtemplate-name-agreement.bats
hack/talos-reconcile-heredoc_test.bats`: all 16 tests passed (5
name-agreement and 11 heredoc tests). Covers Job/Role/MachineDeployment
name agreement, changed and restored hashes, explicit zero, schema and
template type rejection, invalid UTF-8 file inputs, and a literal
round-trip through the actual manifest-producing heredoc under Bash and
Dash. Hostile keys/values include command substitutions, backticks,
variable references, backslashes, quotes, newlines and Unicode
separators; no marker command executes, and runtime placeholders still
expand. The UTF-8 regression was observed failing before the fix with
schema validation disabled; valid Unicode including U+FFFD remains
accepted.
- `bats hack/bats-no-exit-trap.bats hack/bats-no-negated-assert.bats`:
all 32 tests passed.
- Package `make generate` with cozyvalues-gen v1.7.0 and root `make
generate`: repeat runs are idempotent. Root `make manifests` passed.
- `go test ./kubernetesnodes` and `go vet ./kubernetesnodes` in
`api/apps/v1alpha1` passed; the generated API package has no test files.

No live cluster rollout or gVisor execution was tested for this PR.
`make show` could not run because `cozyhr` is unavailable locally;
offline chart rendering is exercised by the Helm and BATS tests above.
Worker replacement and rollback still need staging validation. This
main-branch change is not drop-in support for the release-1.6 nodeGroups
API.

### Screenshots

Not applicable; no UI changes.

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) - the
existing `K8S_NODES` reference imports this package's generated README
at release; no manual docs contract changes.
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
https://github.com/cozystack/terraform-provider-cozystack/issues/72;
adapt the existing follow-up to the general `talos.sysctls` map after
the upstream API is accepted, rather than implementing the superseded
dedicated field.
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The other trigger-map entries are unaffected: no management-node
prerequisite, package name, controller contract, tooling, image or
release convention changes. No downstream implementation or live
deployment is included.

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
feat(kubernetes): add an optional per-pool KubernetesNodes talos.sysctls string map for Talos worker machine.sysctls overrides, preserving defaults and using the existing bootstrap rollout mechanism.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Added optional string-valued sysctl overrides for worker nodes in a
pool. The default is empty and preserves existing behavior. Adding,
changing, or removing overrides updates the worker configuration and
rolls the pool.
* **Bug Fixes**
* Invalid sysctl configuration types and invalid UTF-8 values are
rejected during validation.
* **Documentation**
* Documented the setting’s scope, rollout behavior, and compatibility
considerations, including that it does not configure gVisor. Clarified
HTTPS certificate configuration and when it applies.
* **Improvements**
* Adjusted installation waiting behavior and timeout for Kubernetes node
pools. Updated the ordering of displayed configuration options.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Aleksei Sviridkin authored and GitHub committed 2026-11-16 21:30:35 +03:00
commit ecca3b2af5
10 files changed
+230 -25

No files matched your search

+3
View File
@@ -179,6 +179,9 @@ type Talos struct {
// Talos image-factory schematic ID. Defaults to the cozystack-tested vanilla schematic. Operators using custom schematics (system extensions, kernel args) override here. The pool can pin its own via `osImage.builtin.schematicID` or `osImage.factory.schematicID`; when `osImage` omits it, this value applies. Whichever of the two applies is interpolated into an image URL and a DataVolume name, so it must be a plain lowercase alphanumeric identifier -- the render refuses anything else. A custom schematic on `osImage.builtin` also needs its own catalog entry, since the golden is named from the schematic ID.
// +kubebuilder:default:="ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515"
SchematicID string `json:"schematicID"`
// Sysctl overrides rendered into Talos `machine.sysctls` on this pool's guest workers only. Values must be strings (quote numeric values); for example, `user.max_user_namespaces: "11255"` enables the user namespaces required by the Talos gVisor extension. An empty map preserves Talos defaults and the existing bootstrap hash. Adding, changing or removing an override changes the content-hashed TalosConfigTemplate and rolls the pool's workers. Supported names and values depend on the worker's Talos release and kernel; unsafe settings can weaken guest isolation or prevent workers from booting. This does not install gVisor or configure a RuntimeClass, and does not change the management hosts, control plane or other pools.
// +kubebuilder:default:={}
Sysctls map[string]string `json:"sysctls,omitempty"`
// Default Talos release for this pool's worker OS image and for the installer image written into the worker machine config. The pool can pin its own release via `osImage.builtin.version` or `osImage.factory.version`; when `osImage` omits it, this value applies. Both ways in are checked against the chart's Talos<->Kubernetes support matrix, against whichever release the pool actually resolves to, so an override cannot slip past a check `talos.version` would have failed. The matrix lists, for each Talos minor the chart knows, the Kubernetes minors it supports and refuses a pool `version` outside that list; a Talos minor it does not list passes unchecked, so pointing either field at a Talos release the chart does not know about is the operator's call, not a checked one. Whichever release applies is interpolated into an image URL and a DataVolume name, so it must be a full `vMAJOR.MINOR.PATCH` release, optionally with a pre-release suffix -- the render refuses anything else, a minor-only `v1.13` included. The values schema types this as a bare string, so a malformed one is accepted by the CR and fails the pool's HelmRelease instead.
// +kubebuilder:default:="v1.13.6"
Version string `json:"version"`
@@ -207,6 +207,13 @@ func (in *Resources) DeepCopy() *Resources {
func (in *Talos) DeepCopyInto(out *Talos) {
*out = *in
in.RegistryMirrors.DeepCopyInto(&out.RegistryMirrors)
if in.Sysctls != nil {
in, out := &in.Sysctls, &out.Sysctls
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Talos.
@@ -167,3 +167,21 @@ assert_agreement() {
fi
rm -rf "$TMP"
}
@test "sysctl additions changes and removal keep Job RBAC and bootstrap references in agreement" {
TMP=$(mktemp -d)
render_pool "$TMP/r.yaml"
baseline=$(job_tct_name "$TMP/r.yaml")
render_pool "$TMP/r.yaml" --set-string 'talos.sysctls.user\.max_user_namespaces=11255'
assert_agreement "$TMP" "sysctls enabled"
enabled=$(job_tct_name "$TMP/r.yaml")
[ "$enabled" != "$baseline" ]
render_pool "$TMP/r.yaml" --set-string 'talos.sysctls.user\.max_user_namespaces=0'
assert_agreement "$TMP" "explicit zero sysctl"
[ "$(job_tct_name "$TMP/r.yaml")" != "$enabled" ]
[ "$(job_tct_name "$TMP/r.yaml")" != "$baseline" ]
render_pool "$TMP/r.yaml" --set-json 'talos.sysctls={}'
assert_agreement "$TMP" "sysctls removed"
[ "$(job_tct_name "$TMP/r.yaml")" = "$baseline" ]
rm -rf "$TMP"
}
+105 -6
View File
@@ -2,8 +2,8 @@
# -----------------------------------------------------------------------------
# Execution-level regression test for the worker TalosConfigTemplate heredoc:
# it pins that the reconcile Job's unquoted heredoc still emits a valid config
# when talos.registryMirrors (and the Talos image coordinates) carry hostile
# free-form input.
# when talos.registryMirrors, talos.sysctls and the Talos image coordinates
# carry hostile free-form input.
#
# Since the Phase 2 split the worker pool, its talos-reconcile Job, and the
# TalosConfigTemplate it applies live only in the kubernetes-nodes chart; the
@@ -13,8 +13,8 @@
# The reconcile Job applies the TalosConfigTemplate via an UNQUOTED `cat <<EOF`
# heredoc, captured into a variable and then applied, so every line of its body
# is subject to shell parameter expansion and command substitution at Job
# runtime. The `talos.registryMirrors` knob and the Talos image coordinates
# render free-form tenant-facing input into that heredoc. A helm-unittest
# runtime. The `talos.registryMirrors` and `talos.sysctls` knobs and the Talos
# image coordinates render free-form tenant-facing input into that heredoc. A helm-unittest
# string `matchRegex` cannot catch a heredoc that the shell refuses to emit
# (e.g. an unbalanced backtick from an un-escaped value): it never runs the
# shell. This test does.
@@ -26,7 +26,7 @@
# escaped the value reaches the heredoc and is escaped for backslash,
# dollar and backtick. Assert it survives a real shell as a
# LITERAL. This covers talos.installerRepository and
# talos.registryMirrors, both genuinely free-form.
# talos.registryMirrors and talos.sysctls (both keys and values).
# validated the value is refused at render before it can reach the heredoc.
# Assert the refusal. This covers talos.schematicID and
# talos.version, which also land in the worker DataVolume name and
@@ -34,15 +34,32 @@
# wrong tool. templates/nodegroup.yaml pattern-checks them. It also
# covers kernelModules, whose names and parameters
# templates/_helpers.tpl checks before rebuilding the list.
# Sysctl map/value types and UTF-8 are validated before escaping.
#
# A field that is in neither arm is the regression this file exists to catch.
#
# Needs `helm` + `yq` and must run from the repo root.
# Needs `helm`, `yq`, Bash and Dash and must run from the repo root.
# Run with: bats hack/talos-reconcile-heredoc_test.bats
# -----------------------------------------------------------------------------
load test_helper
render_sysctl_pool() {
out="$1"; shift
helm template kubernetes-nodes-myk8s-md0 packages/apps/kubernetes-nodes \
--namespace tenant-test \
--set cluster=myk8s \
--set _cluster.cluster-domain=cozy.local \
--set version=v1.35 \
--set minReplicas=0 --set maxReplicas=3 \
--set instanceType="" \
--set diskSize=20Gi --set storageClass=replicated \
--set 'roles[0]=ingress-nginx' \
--set resources.cpu=2 --set resources.memory=4Gi \
"$@" > "$out" 2>"$out.err"
[ -s "$out" ] || { cat "$out.err" >&2; return 1; }
}
@test "kubernetes-nodes refuses hostile Talos image coordinates at render" {
work=$(mktemp -d)
cat > "$work/vals.yaml" <<'VALS'
@@ -334,3 +351,85 @@ VALS
|| { echo "kernelModules parameter was not preserved literally" >&2; printf '%s\n' "$out" | grep -A6 'kernel:' >&2; rm -rf "$work"; exit 1; }
rm -rf "$work"
}
@test "sysctl names and values survive the actual Job heredoc literally without shell execution" {
TMP=$(mktemp -d)
cat > "$TMP/values.yaml" <<'VALUES'
talos:
sysctls:
user.max_user_namespaces: "11255"
net.ipv4.ip_local_port_range: "1024\t65535"
kernel.panic: "0"
'$(touch "$SYSCTL_MARKER")': '$(touch "$SYSCTL_MARKER")'
'`touch "$SYSCTL_MARKER"`': '`touch "$SYSCTL_MARKER"`'
'${SYSCTL_MARKER}': '${SYSCTL_MARKER}'
'backslash\$(touch "$SYSCTL_MARKER")': 'backslash\$(touch "$SYSCTL_MARKER")'
"quotes\"\\\nEOF\n$(touch \"$SYSCTL_MARKER\")": "quotes\"\\\nEOF\n$(touch \"$SYSCTL_MARKER\")"
"unicode\u0085EOF\u2028EOF\u2029": "unicode\u0085EOF\u2028EOF\u2029"
"empty": ""
VALUES
render_sysctl_pool "$TMP/r.yaml" -f "$TMP/values.yaml"
yq -r 'select(.kind == "Job") | .spec.template.spec.containers[0].command[2]' "$TMP/r.yaml" > "$TMP/job.sh"
# Execute only the real manifest-producing heredoc, never the surrounding
# kubectl/Talos operations. YAML is parsed on both sides of the shell so
# escaping that prevents execution but corrupts the machine config also fails.
awk '/^TCT_MANIFEST=/{emit=1} emit{print} emit && /^\)"$/{exit}' "$TMP/job.sh" > "$TMP/heredoc.sh"
[ -s "$TMP/heredoc.sh" ]
printf '\nprintf "%%s\\n" "$TCT_MANIFEST"\n' >> "$TMP/heredoc.sh"
yq -o=json '.talos.sysctls | sort_keys(.)' "$TMP/values.yaml" > "$TMP/expected.json"
for shell in bash dash; do
SYSCTL_MARKER="$TMP/executed" TALOS_TOKEN=runtime-token "$shell" "$TMP/heredoc.sh" > "$TMP/tct.yaml"
[ ! -e "$TMP/executed" ]
yq -o=json '.spec.template.spec.data | from_yaml | .machine.sysctls | sort_keys(.)' "$TMP/tct.yaml" > "$TMP/actual.json"
diff -u "$TMP/expected.json" "$TMP/actual.json"
[ "$(yq '.spec.template.spec.data | from_yaml | .machine.token' "$TMP/tct.yaml")" = "runtime-token" ]
done
rm -rf "$TMP"
}
@test "sysctl types are rejected by both the schema and the render-time guard" {
TMP=$(mktemp -d)
for value in '11255' 'false' 'null' '["1"]' '{"evil":"$(id)"}'; do
input="talos.sysctls={\"kernel.panic\":$value}"
if render_sysctl_pool "$TMP/r.yaml" --set-json "$input"; then
echo "FAIL: schema accepted a non-string sysctl: $value"
false
fi
grep -q 'values don.t meet the specifications' "$TMP/r.yaml.err"
if render_sysctl_pool "$TMP/r.yaml" --skip-schema-validation --set-json "$input"; then
echo "FAIL: template accepted a non-string sysctl: $value"
false
fi
grep -Fq 'talos.sysctls["kernel.panic"] must be a string' "$TMP/r.yaml.err"
done
for value in '"$(id)"' 'false' '0' '[]' '["1"]'; do
input="talos.sysctls=$value"
if render_sysctl_pool "$TMP/r.yaml" --set-json "$input"; then
echo "FAIL: schema accepted a non-map sysctls: $value"
false
fi
grep -q 'values don.t meet the specifications' "$TMP/r.yaml.err"
if render_sysctl_pool "$TMP/r.yaml" --skip-schema-validation --set-json "$input"; then
echo "FAIL: template accepted a non-map sysctls: $value"
false
fi
grep -Fq 'talos.sysctls must be a map of strings' "$TMP/r.yaml.err"
done
rm -rf "$TMP"
}
@test "sysctl file values reject invalid UTF-8 rather than changing their bytes" {
TMP=$(mktemp -d)
# --set-file reads raw bytes, unlike the YAML/JSON-facing application API.
for bytes in '\377' '\300\257' '\342\202'; do
printf '%b' "$bytes" > "$TMP/value"
if render_sysctl_pool "$TMP/r.yaml" --skip-schema-validation --set-file "talos.sysctls.kernel\\.panic=$TMP/value"; then
echo "FAIL: accepted invalid UTF-8 sysctl bytes: $bytes"
false
fi
grep -Fq 'must be valid UTF-8' "$TMP/r.yaml.err"
done
printf '\357\277\275' > "$TMP/value"
render_sysctl_pool "$TMP/r.yaml" --skip-schema-validation --set-file "talos.sysctls.kernel\\.panic=$TMP/value"
rm -rf "$TMP"
}
+18 -17
View File
@@ -72,23 +72,24 @@ The pool's Kubernetes minor (`version`) may lag the cluster's during an upgrade
### Talos Worker Image
| Name | Description | Type | Value |
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | ------------------------------------------------------------------ |
| `osImage` | Worker OS image source for this pool. Default (omitted): import over HTTP from the `talos.*` Image Factory. Set `osImage.builtin` to clone a golden from the opt-in worker image catalog instead. Editing this on an existing pool re-images that pool. | `object` | `{}` |
| `osImage.builtin` | Clone a golden image from the worker image catalog. Requires a StorageClass whose volumes are reachable from any node, in practice a replicated/DRBD class: the clone is a storage-layer copy, and on a node-pinned class its two ends land on unrelated nodes -- the golden binds where CDI imported it, a worker disk where its VM is scheduled. | `*object` | `null` |
| `osImage.builtin.schematicID` | Talos schematic ID of the golden to clone. Defaults to `talos.schematicID`. | `string` | `""` |
| `osImage.builtin.version` | Talos version of the golden to clone. Defaults to `talos.version`. | `string` | `""` |
| `osImage.factory` | Import from an Image Factory / mirror (default). | `*object` | `null` |
| `osImage.factory.version` | Talos version. Defaults to `talos.version`. | `string` | `""` |
| `osImage.factory.schematicID` | Talos schematic ID. Defaults to `talos.schematicID`. | `string` | `""` |
| `osImage.factory.imageFactoryURL` | Base URL of the Image Factory serving the boot raw image. Defaults to `talos.imageFactoryURL`. No trailing slash. This redirects only the worker boot-disk import; the installer image in the worker machine config is always sourced from `talos.installerRepository`, so an air-gapped pool must also have `talos.installerRepository` pointed at a reachable registry. | `string` | `""` |
| `talos` | Talos worker image configuration for this pool. | `object` | `{}` |
| `talos.version` | Default Talos release for this pool's worker OS image and for the installer image written into the worker machine config. The pool can pin its own release via `osImage.builtin.version` or `osImage.factory.version`; when `osImage` omits it, this value applies. Both ways in are checked against the chart's Talos<->Kubernetes support matrix, against whichever release the pool actually resolves to, so an override cannot slip past a check `talos.version` would have failed. The matrix lists, for each Talos minor the chart knows, the Kubernetes minors it supports and refuses a pool `version` outside that list; a Talos minor it does not list passes unchecked, so pointing either field at a Talos release the chart does not know about is the operator's call, not a checked one. Whichever release applies is interpolated into an image URL and a DataVolume name, so it must be a full `vMAJOR.MINOR.PATCH` release, optionally with a pre-release suffix -- the render refuses anything else, a minor-only `v1.13` included. The values schema types this as a bare string, so a malformed one is accepted by the CR and fails the pool's HelmRelease instead. | `string` | `v1.13.6` |
| `talos.schematicID` | Talos image-factory schematic ID. Defaults to the cozystack-tested vanilla schematic. Operators using custom schematics (system extensions, kernel args) override here. The pool can pin its own via `osImage.builtin.schematicID` or `osImage.factory.schematicID`; when `osImage` omits it, this value applies. Whichever of the two applies is interpolated into an image URL and a DataVolume name, so it must be a plain lowercase alphanumeric identifier -- the render refuses anything else. A custom schematic on `osImage.builtin` also needs its own catalog entry, since the golden is named from the schematic ID. | `string` | `ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515` |
| `talos.imageFactoryURL` | Default base URL of the Talos Image Factory that serves the worker OS disk image (the `openstack-amd64.raw.xz` raw artifact streamed in by CDI over HTTP). Applies only when this pool imports over HTTP — the default and `osImage.factory`, which overrides it via `osImage.factory.imageFactoryURL`. A pool on `osImage.builtin` clones a golden image from the worker image catalog and does not use this field at all. Defaults to the public factory. Point at a self-hosted Image Factory, a caching mirror, or an internal HTTP file server for air-gapped, rate-limited, or flaky-egress environments. No trailing slash. It is interpolated into the worker DataVolume source URL, so on those HTTP paths it must be a plain `http(s)` URL and the render refuses anything else; a pool on `osImage.builtin` builds no such URL and is not held to it, so blanking this field there is legal. On `substrate: proxmox` the workers clone a VM template that already carries the OS, so this field has no effect there beyond that format check. | `string` | `https://factory.talos.dev` |
| `talos.installerRepository` | OCI repository prefix for the Talos installer image, written into the worker machine config as `machine.install.image`. Resolved as `<installerRepository>/<schematicID>:<version>` against whatever this pool's `osImage` pins, falling back to `talos.schematicID` and `talos.version` — so the installer matches the OS the pool actually booted. The prefix itself has no `osImage.*` override, so an air-gapped pool needs it pointed at a reachable registry as well. Defaults to the public factory's installer path. Override for air-gapped or mirrored registries. No trailing slash. | `string` | `factory.talos.dev/installer` |
| `talos.registryMirrors` | Talos `machine.registries.mirrors` passthrough for worker nodes: a map of upstream registry host to `{ endpoints: [ ... ] }`. Empty by default, so workers pull container images (the Talos `kubelet` image included) directly from the upstream registry. Point a host such as `ghcr.io` at an in-cluster pull-through mirror for air-gapped, rate-limited, or flaky-egress environments so a worker's boot does not depend on live public egress. Talos still falls back to the upstream registry unless a host also sets `skipFallback: true`, so a mirror alone does not enforce air-gap. This value is part of the worker machine config, whose content hash names the TalosConfigTemplate, so changing it rolls every existing worker in the pool rather than only affecting nodes created later. | `object` | `{}` |
| `talos.imageFactoryCA` | Complete PEM certificate block(s) CDI must trust when importing this pool's worker OS disk image over HTTPS — the no-osImage default and `osImage.factory` path. Empty by default, so the importer uses only its system trust store, which is what the public factory needs. Set this when the factory in use (`imageFactoryURL` or `osImage.factory.imageFactoryURL`) is a self-hosted factory or mirror served by an internal CA, which the importer otherwise rejects with `x509: certificate signed by unknown authority`. The chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references that from the worker DataVolume's `source.http.certConfigMap`; CDI appends it to the system pool rather than replacing it, so a publicly-trusted endpoint verifies either way. A pool on `osImage.builtin` clones a golden image and builds no HTTP source, so this field has no effect there. On `substrate: proxmox` it has no effect on the workers either: the ConfigMap is still rendered, but nothing references it. | `string` | `""` |
| Name | Description | Type | Value |
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------ |
| `osImage` | Worker OS image source for this pool. Default (omitted): import over HTTP from the `talos.*` Image Factory. Set `osImage.builtin` to clone a golden from the opt-in worker image catalog instead. Editing this on an existing pool re-images that pool. | `object` | `{}` |
| `osImage.builtin` | Clone a golden image from the worker image catalog. Requires a StorageClass whose volumes are reachable from any node, in practice a replicated/DRBD class: the clone is a storage-layer copy, and on a node-pinned class its two ends land on unrelated nodes -- the golden binds where CDI imported it, a worker disk where its VM is scheduled. | `*object` | `null` |
| `osImage.builtin.schematicID` | Talos schematic ID of the golden to clone. Defaults to `talos.schematicID`. | `string` | `""` |
| `osImage.builtin.version` | Talos version of the golden to clone. Defaults to `talos.version`. | `string` | `""` |
| `osImage.factory` | Import from an Image Factory / mirror (default). | `*object` | `null` |
| `osImage.factory.version` | Talos version. Defaults to `talos.version`. | `string` | `""` |
| `osImage.factory.schematicID` | Talos schematic ID. Defaults to `talos.schematicID`. | `string` | `""` |
| `osImage.factory.imageFactoryURL` | Base URL of the Image Factory serving the boot raw image. Defaults to `talos.imageFactoryURL`. No trailing slash. This redirects only the worker boot-disk import; the installer image in the worker machine config is always sourced from `talos.installerRepository`, so an air-gapped pool must also have `talos.installerRepository` pointed at a reachable registry. | `string` | `""` |
| `talos` | Talos worker image configuration for this pool. | `object` | `{}` |
| `talos.version` | Default Talos release for this pool's worker OS image and for the installer image written into the worker machine config. The pool can pin its own release via `osImage.builtin.version` or `osImage.factory.version`; when `osImage` omits it, this value applies. Both ways in are checked against the chart's Talos<->Kubernetes support matrix, against whichever release the pool actually resolves to, so an override cannot slip past a check `talos.version` would have failed. The matrix lists, for each Talos minor the chart knows, the Kubernetes minors it supports and refuses a pool `version` outside that list; a Talos minor it does not list passes unchecked, so pointing either field at a Talos release the chart does not know about is the operator's call, not a checked one. Whichever release applies is interpolated into an image URL and a DataVolume name, so it must be a full `vMAJOR.MINOR.PATCH` release, optionally with a pre-release suffix -- the render refuses anything else, a minor-only `v1.13` included. The values schema types this as a bare string, so a malformed one is accepted by the CR and fails the pool's HelmRelease instead. | `string` | `v1.13.6` |
| `talos.schematicID` | Talos image-factory schematic ID. Defaults to the cozystack-tested vanilla schematic. Operators using custom schematics (system extensions, kernel args) override here. The pool can pin its own via `osImage.builtin.schematicID` or `osImage.factory.schematicID`; when `osImage` omits it, this value applies. Whichever of the two applies is interpolated into an image URL and a DataVolume name, so it must be a plain lowercase alphanumeric identifier -- the render refuses anything else. A custom schematic on `osImage.builtin` also needs its own catalog entry, since the golden is named from the schematic ID. | `string` | `ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515` |
| `talos.imageFactoryURL` | Default base URL of the Talos Image Factory that serves the worker OS disk image (the `openstack-amd64.raw.xz` raw artifact streamed in by CDI over HTTP). Applies only when this pool imports over HTTP — the default and `osImage.factory`, which overrides it via `osImage.factory.imageFactoryURL`. A pool on `osImage.builtin` clones a golden image from the worker image catalog and does not use this field at all. Defaults to the public factory. Point at a self-hosted Image Factory, a caching mirror, or an internal HTTP file server for air-gapped, rate-limited, or flaky-egress environments. No trailing slash. It is interpolated into the worker DataVolume source URL, so on those HTTP paths it must be a plain `http(s)` URL and the render refuses anything else; a pool on `osImage.builtin` builds no such URL and is not held to it, so blanking this field there is legal. On `substrate: proxmox` the workers clone a VM template that already carries the OS, so this field has no effect there beyond that format check. | `string` | `https://factory.talos.dev` |
| `talos.installerRepository` | OCI repository prefix for the Talos installer image, written into the worker machine config as `machine.install.image`. Resolved as `<installerRepository>/<schematicID>:<version>` against whatever this pool's `osImage` pins, falling back to `talos.schematicID` and `talos.version` — so the installer matches the OS the pool actually booted. The prefix itself has no `osImage.*` override, so an air-gapped pool needs it pointed at a reachable registry as well. Defaults to the public factory's installer path. Override for air-gapped or mirrored registries. No trailing slash. | `string` | `factory.talos.dev/installer` |
| `talos.registryMirrors` | Talos `machine.registries.mirrors` passthrough for worker nodes: a map of upstream registry host to `{ endpoints: [ ... ] }`. Empty by default, so workers pull container images (the Talos `kubelet` image included) directly from the upstream registry. Point a host such as `ghcr.io` at an in-cluster pull-through mirror for air-gapped, rate-limited, or flaky-egress environments so a worker's boot does not depend on live public egress. Talos still falls back to the upstream registry unless a host also sets `skipFallback: true`, so a mirror alone does not enforce air-gap. This value is part of the worker machine config, whose content hash names the TalosConfigTemplate, so changing it rolls every existing worker in the pool rather than only affecting nodes created later. | `object` | `{}` |
| `talos.sysctls` | Sysctl overrides rendered into Talos `machine.sysctls` on this pool's guest workers only. Values must be strings (quote numeric values); for example, `user.max_user_namespaces: "11255"` enables the user namespaces required by the Talos gVisor extension. An empty map preserves Talos defaults and the existing bootstrap hash. Adding, changing or removing an override changes the content-hashed TalosConfigTemplate and rolls the pool's workers. Supported names and values depend on the worker's Talos release and kernel; unsafe settings can weaken guest isolation or prevent workers from booting. This does not install gVisor or configure a RuntimeClass, and does not change the management hosts, control plane or other pools. | `map[string]string` | `{}` |
| `talos.imageFactoryCA` | Complete PEM certificate block(s) CDI must trust when importing this pool's worker OS disk image over HTTPS — the no-osImage default and `osImage.factory` path. Empty by default, so the importer uses only its system trust store, which is what the public factory needs. Set this when the factory in use (`imageFactoryURL` or `osImage.factory.imageFactoryURL`) is a self-hosted factory or mirror served by an internal CA, which the importer otherwise rejects with `x509: certificate signed by unknown authority`. The chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references that from the worker DataVolume's `source.http.certConfigMap`; CDI appends it to the system pool rather than replacing it, so a publicly-trusted endpoint verifies either way. A pool on `osImage.builtin` clones a golden image and builds no HTTP source, so this field has no effect there. On `substrate: proxmox` it has no effect on the workers either: the ConfigMap is still rendered, but nothing references it. | `string` | `""` |
### Substrate
@@ -42,6 +42,24 @@
{{- define "kubernetes.talosConfigTemplateSpec" -}}
{{- $root := .root }}
{{- $group := .group }}
{{- $sysctls := $root.Values.talos.sysctls }}
{{- if not (kindIs "invalid" $sysctls) }}
{{- if not (kindIs "map" $sysctls) }}
{{- fail "talos.sysctls must be a map of strings" }}
{{- end }}
{{- range $key, $value := $sysctls }}
{{- if not (kindIs "string" $value) }}
{{- fail (printf "talos.sysctls[%q] must be a string" $key) }}
{{- end }}
{{- /* JSON replaces invalid UTF-8 with replacement runes. Comparing the
round-trip detects raw --set-file bytes that quote would change into
different Unicode characters in the machine config. */}}
{{- $roundTrip := dict "value" $value | toJson | fromJson }}
{{- if ne $value $roundTrip.value }}
{{- fail (printf "talos.sysctls[%q] must be valid UTF-8" $key) }}
{{- end }}
{{- end }}
{{- end }}
{{- $kubeletVersion := include "kubernetes.versionMap" $root | trim }}
{{- /* Resolve this pool's worker image (schematicID, version) from .Values.osImage
— the same selection templates/nodegroup.yaml uses for the boot disk — so the
@@ -140,6 +158,17 @@ spec:
machine:
type: worker
token: ${TALOS_TOKEN}
{{- if $sysctls }}
sysctls:
{{- /* Quote each scalar before shell-escaping it: unlike toYaml's
multiline scalars, quote keeps tenant line breaks (including
Unicode separators) out of the Job's command block and EOF
delimiter. The unquoted heredoc must preserve both names and
values literally; see the INVARIANT above. */}}
{{- range $key, $value := $sysctls }}
{{ $key | quote | replace "\\" "\\\\" | replace "$" "\\$" | replace "`" "\\`" }}: {{ $value | quote | replace "\\" "\\\\" | replace "$" "\\$" | replace "`" "\\`" }}
{{- end }}
{{- end }}
network:
# Management CoreDNS as the worker's host
# resolver, plus management cluster search
@@ -0,0 +1,38 @@
suite: worker sysctl overrides
templates:
- templates/talos-reconcile-job.yaml
release:
name: kubernetes-nodes-myk8s-md0
namespace: tenant-test
set:
cluster: myk8s
_cluster:
cluster-domain: cozy.local
version: v1.35
instanceType: ""
resources:
cpu: "2"
memory: 4Gi
tests:
- it: preserves Talos defaults when no sysctls are configured
asserts:
- notMatchRegex:
path: spec.template.spec.containers[0].command[2]
pattern: 'sysctls:'
documentSelector:
path: kind
value: Job
- it: renders multiple string overrides under machine.sysctls
set:
talos.sysctls:
user.max_user_namespaces: "11255"
net.ipv4.ip_local_port_range: "1024 65535"
kernel.kptr_restrict: "0"
asserts:
- matchRegex:
path: spec.template.spec.containers[0].command[2]
pattern: '\n {10}sysctls:\n {12}"kernel.kptr_restrict": "0"\n {12}"net.ipv4.ip_local_port_range": "1024 65535"\n {12}"user.max_user_namespaces": "11255"\n {10}network:'
documentSelector:
path: kind
value: Job
@@ -327,6 +327,14 @@
"type": "string",
"default": "ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515"
},
"sysctls": {
"description": "Sysctl overrides rendered into Talos `machine.sysctls` on this pool's guest workers only. Values must be strings (quote numeric values); for example, `user.max_user_namespaces: \"11255\"` enables the user namespaces required by the Talos gVisor extension. An empty map preserves Talos defaults and the existing bootstrap hash. Adding, changing or removing an override changes the content-hashed TalosConfigTemplate and rolls the pool's workers. Supported names and values depend on the worker's Talos release and kernel; unsafe settings can weaken guest isolation or prevent workers from booting. This does not install gVisor or configure a RuntimeClass, and does not change the management hosts, control plane or other pools.",
"type": "object",
"default": {},
"additionalProperties": {
"type": "string"
}
},
"version": {
"description": "Default Talos release for this pool's worker OS image and for the installer image written into the worker machine config. The pool can pin its own release via `osImage.builtin.version` or `osImage.factory.version`; when `osImage` omits it, this value applies. Both ways in are checked against the chart's Talos\u003c-\u003eKubernetes support matrix, against whichever release the pool actually resolves to, so an override cannot slip past a check `talos.version` would have failed. The matrix lists, for each Talos minor the chart knows, the Kubernetes minors it supports and refuses a pool `version` outside that list; a Talos minor it does not list passes unchecked, so pointing either field at a Talos release the chart does not know about is the operator's call, not a checked one. Whichever release applies is interpolated into an image URL and a DataVolume name, so it must be a full `vMAJOR.MINOR.PATCH` release, optionally with a pre-release suffix -- the render refuses anything else, a minor-only `v1.13` included. The values schema types this as a bare string, so a malformed one is accepted by the CR and fails the pool's HelmRelease instead.",
"type": "string",
@@ -144,6 +144,7 @@ osImage: {}
## @field {string} imageFactoryURL - Default base URL of the Talos Image Factory that serves the worker OS disk image (the `openstack-amd64.raw.xz` raw artifact streamed in by CDI over HTTP). Applies only when this pool imports over HTTP — the default and `osImage.factory`, which overrides it via `osImage.factory.imageFactoryURL`. A pool on `osImage.builtin` clones a golden image from the worker image catalog and does not use this field at all. Defaults to the public factory. Point at a self-hosted Image Factory, a caching mirror, or an internal HTTP file server for air-gapped, rate-limited, or flaky-egress environments. No trailing slash. It is interpolated into the worker DataVolume source URL, so on those HTTP paths it must be a plain `http(s)` URL and the render refuses anything else; a pool on `osImage.builtin` builds no such URL and is not held to it, so blanking this field there is legal. On `substrate: proxmox` the workers clone a VM template that already carries the OS, so this field has no effect there beyond that format check.
## @field {string} installerRepository - OCI repository prefix for the Talos installer image, written into the worker machine config as `machine.install.image`. Resolved as `<installerRepository>/<schematicID>:<version>` against whatever this pool's `osImage` pins, falling back to `talos.schematicID` and `talos.version` — so the installer matches the OS the pool actually booted. The prefix itself has no `osImage.*` override, so an air-gapped pool needs it pointed at a reachable registry as well. Defaults to the public factory's installer path. Override for air-gapped or mirrored registries. No trailing slash.
## @field {object} registryMirrors - Talos `machine.registries.mirrors` passthrough for worker nodes: a map of upstream registry host to `{ endpoints: [ ... ] }`. Empty by default, so workers pull container images (the Talos `kubelet` image included) directly from the upstream registry. Point a host such as `ghcr.io` at an in-cluster pull-through mirror for air-gapped, rate-limited, or flaky-egress environments so a worker's boot does not depend on live public egress. Talos still falls back to the upstream registry unless a host also sets `skipFallback: true`, so a mirror alone does not enforce air-gap. This value is part of the worker machine config, whose content hash names the TalosConfigTemplate, so changing it rolls every existing worker in the pool rather than only affecting nodes created later.
## @field {map[string]string} sysctls - Sysctl overrides rendered into Talos `machine.sysctls` on this pool's guest workers only. Values must be strings (quote numeric values); for example, `user.max_user_namespaces: "11255"` enables the user namespaces required by the Talos gVisor extension. An empty map preserves Talos defaults and the existing bootstrap hash. Adding, changing or removing an override changes the content-hashed TalosConfigTemplate and rolls the pool's workers. Supported names and values depend on the worker's Talos release and kernel; unsafe settings can weaken guest isolation or prevent workers from booting. This does not install gVisor or configure a RuntimeClass, and does not change the management hosts, control plane or other pools.
## @field {string} [imageFactoryCA] - Complete PEM certificate block(s) CDI must trust when importing this pool's worker OS disk image over HTTPS — the no-osImage default and `osImage.factory` path. Empty by default, so the importer uses only its system trust store, which is what the public factory needs. Set this when the factory in use (`imageFactoryURL` or `osImage.factory.imageFactoryURL`) is a self-hosted factory or mirror served by an internal CA, which the importer otherwise rejects with `x509: certificate signed by unknown authority`. The chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references that from the worker DataVolume's `source.http.certConfigMap`; CDI appends it to the system pool rather than replacing it, so a publicly-trusted endpoint verifies either way. A pool on `osImage.builtin` clones a golden image and builds no HTTP source, so this field has no effect there. On `substrate: proxmox` it has no effect on the workers either: the ConfigMap is still rendered, but nothing references it.
## @param {Talos} talos - Talos worker image configuration for this pool.
@@ -153,6 +154,7 @@ talos:
imageFactoryURL: "https://factory.talos.dev"
installerRepository: "factory.talos.dev/installer"
registryMirrors: {}
sysctls: {}
imageFactoryCA: ""
##
@@ -21,7 +21,7 @@ spec:
singular: kubernetesnodes
plural: kubernetesnodeses
openAPISchema: |-
{"title":"Chart Values","type":"object","properties":{"cluster":{"description":"Name of the parent Kubernetes cluster (the `Kubernetes` CR) in the same namespace this node pool attaches to. Required. The pool's CAPI objects are wired to the parent cluster `kubernetes-<cluster>`; the KubernetesNodes CR must be named `<cluster>-<pool>`. The pool's objects are named `kubernetes-<cluster>-<pool>`; when a MachineDeployment of that name already exists and belongs to another Helm release, the render fails instead of taking it over.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"cluster is immutable"}]},"storageClass":{"description":"StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.","type":"string","default":"replicated","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}},"minReplicas":{"description":"Minimum number of replicas in this pool. Used as the cluster-autoscaler floor.","type":"integer","default":0},"maxReplicas":{"description":"Maximum number of replicas in this pool. Used as the cluster-autoscaler ceiling.","type":"integer","default":10},"instanceType":{"description":"KubeVirt instance type that sizes each worker VM when `resources` is not set. Ignored when `resources.cpu` and `resources.memory` are both set, and therefore on `substrate: proxmox`, which requires them.","type":"string","default":"u1.medium","x-cozystack-options":{"source":"instancetype"}},"diskSize":{"description":"System disk size for the worker VM. Carries the Talos OS image, kubelet state, containerd image cache, and any local-path PVCs. How the OS image reaches the disk follows this pool's `osImage`: the default and `osImage.factory` stream the Image Factory raw artifact in over HTTP, while `osImage.builtin` makes the disk a storage-layer CDI clone of a shared golden image, with no HTTP fetch. A `builtin` pool must keep diskSize at or above the golden's storage size (6Gi in the shipped worker image catalog), because a CDI clone target cannot be smaller than its source -- the chart rejects a smaller value at render time rather than letting the clone stall. Sizing a `builtin` pool also has to allow for transient headroom the finished disk does not show: CDI populates a larger target by cloning the golden at the source size and then growing it, so a worker being provisioned occupies a temporary clone PVC at the golden's size on top of this diskSize, and both are multiplied by the replica count of a replicated StorageClass. The golden itself is a separate one-time cost for the whole cluster, not a per-worker one. So the peak is the golden, plus `diskSize + golden storage` for each worker cloning at that moment. A pool whose storage backend cannot spare that peak does not fail: the clone waits for space, the VM never boots, and the reason is visible only in the storage layer's own events. Has no effect on `substrate: proxmox`, where the worker disk is a clone of the VM template's disk and keeps its size.","default":"20Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"roles":{"description":"List of node roles. Each role `r` labels nodes with `node-role.kubernetes.io/<r>`. Use `[ingress-nginx]` for a pool that hosts the tenant ingress-nginx controller.","type":"array","default":[],"items":{"type":"string"}},"resources":{"description":"Explicit CPU and memory for each worker node, as an alternative to `instanceType` sizing. Optional: when omitted, the node is sized by `instanceType`. When both `cpu` and `memory` are set, they take precedence and `iLine truncated
{"title":"Chart Values","type":"object","properties":{"cluster":{"description":"Name of the parent Kubernetes cluster (the `Kubernetes` CR) in the same namespace this node pool attaches to. Required. The pool's CAPI objects are wired to the parent cluster `kubernetes-<cluster>`; the KubernetesNodes CR must be named `<cluster>-<pool>`. The pool's objects are named `kubernetes-<cluster>-<pool>`; when a MachineDeployment of that name already exists and belongs to another Helm release, the render fails instead of taking it over.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"cluster is immutable"}]},"storageClass":{"description":"StorageClass for the worker node system disk. When empty, the cluster default applies. Worker VMs live-migrate, so their disks need ReadWriteMany — the RWX access mode is supplied by the chosen StorageClass's CDI StorageProfile — and linstor-csi rejects RWX volumes that are not on a DRBD-backed StorageClass, so prefer a replicated/DRBD class. On an `osImage.builtin` pool this field also decides whether the pool renders at all: CDI takes the storage-layer clone path only when the worker disk and the golden share a StorageClass, and falls back to a host-assisted copy over the pod network across classes, so the chart refuses a pool whose class differs from its golden's. Leaving it empty resolves to the cluster's default StorageClass, and the same comparison is made against that. On a `builtin` pool the resolved class is then written onto the worker disk rather than left for the cluster to fill in at bind time, so a pool that has already cloned keeps binding on the class its disks are on even after an admin moves the default annotation; on the HTTP path an empty class is still left to the cluster. Has no effect on `substrate: proxmox`, where the worker disk is cloned from the VM template onto the template's storage or `proxmox.storage`.","type":"string","default":"replicated","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}},"minReplicas":{"description":"Minimum number of replicas in this pool. Used as the cluster-autoscaler floor.","type":"integer","default":0},"maxReplicas":{"description":"Maximum number of replicas in this pool. Used as the cluster-autoscaler ceiling.","type":"integer","default":10},"instanceType":{"description":"KubeVirt instance type that sizes each worker VM when `resources` is not set. Ignored when `resources.cpu` and `resources.memory` are both set, and therefore on `substrate: proxmox`, which requires them.","type":"string","default":"u1.medium","x-cozystack-options":{"source":"instancetype"}},"diskSize":{"description":"System disk size for the worker VM. Carries the Talos OS image, kubelet state, containerd image cache, and any local-path PVCs. How the OS image reaches the disk follows this pool's `osImage`: the default and `osImage.factory` stream the Image Factory raw artifact in over HTTP, while `osImage.builtin` makes the disk a storage-layer CDI clone of a shared golden image, with no HTTP fetch. A `builtin` pool must keep diskSize at or above the golden's storage size (6Gi in the shipped worker image catalog), because a CDI clone target cannot be smaller than its source -- the chart rejects a smaller value at render time rather than letting the clone stall. Sizing a `builtin` pool also has to allow for transient headroom the finished disk does not show: CDI populates a larger target by cloning the golden at the source size and then growing it, so a worker being provisioned occupies a temporary clone PVC at the golden's size on top of this diskSize, and both are multiplied by the replica count of a replicated StorageClass. The golden itself is a separate one-time cost for the whole cluster, not a per-worker one. So the peak is the golden, plus `diskSize + golden storage` for each worker cloning at that moment. A pool whose storage backend cannot spare that peak does not fail: the clone waits for space, the VM never boots, and the reason is visible only in the storage layer's own events. Has no effect on `substrate: proxmox`, where the worker disk is a clone of the VM template's disk and keeps its size.","default":"20Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"roles":{"description":"List of node roles. Each role `r` labels nodes with `node-role.kubernetes.io/<r>`. Use `[ingress-nginx]` for a pool that hosts the tenant ingress-nginx controller.","type":"array","default":[],"items":{"type":"string"}},"resources":{"description":"Explicit CPU and memory for each worker node, as an alternative to `instanceType` sizing. Optional: when omitted, the node is sized by `instanceType`. When both `cpu` and `memory` are set, they take precedence and `iLine truncated
release:
labels:
sharding.fluxcd.io/key: tenants
@@ -39,7 +39,7 @@ spec:
- compute
description: Worker node pool for a managed Kubernetes cluster
icon: PHN2ZyB3aWR0aD0iMTQ0IiBoZWlnaHQ9IjE0NCIgdmlld0JveD0iMCAwIDE0NCAxNDQiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CjxyZWN0IHdpZHRoPSIxNDQiIGhlaWdodD0iMTQ0IiByeD0iMjQiIGZpbGw9InVybCgjcGFpbnQwX2xpbmVhcl82ODFfMjg0NSkiLz4KPHBhdGggZD0iTTcxLjk5NjggMTlDNzAuMzAzOSAxOS4wMDAyIDY4LjkzMTIgMjAuNTMyMiA2OC45MzE0IDIyLjQyMjFDNjguOTMxNCAyMi40NTExIDY4LjkzNzMgMjIuNDc4OCA2OC45Mzc5IDIyLjUwNzZDNjguOTM1NCAyMi43NjQ0IDY4LjkyMzEgMjMuMDczNyA2OC45MzE0IDIzLjI5NzNDNjguOTcxNyAyNC4zODczIDY5LjIwODIgMjUuMjIxNiA2OS4zNTA2IDI2LjIyNThDNjkuNjA4NCAyOC4zNzUyIDY5LjgyNDUgMzAuMTU2OSA2OS42OTEyIDMxLjgxM0M2OS41NjE1IDMyLjQzNzUgNjkuMTAzNyAzMy4wMDg2IDY4LjY5NTYgMzMuNDA1Nkw2OC42MjM1IDM0LjcwODZDNjYuNzgzOSAzNC44NjE3IDY0LjkzMTkgMzUuMTQyMSA2My4wODIxIDM1LjU2NDFDNTUuMTIyNiAzNy4zNzk4IDQ4LjI2OTUgNDEuNDk5MSA0My4wNTIgNDcuMDYwOUM0Mi43MTM0IDQ2LjgyODggNDIuMTIxMSA0Ni40MDE5IDQxLjk0NSA0Ni4yNzEyQzQxLjM5NzcgNDYuMzQ1NCA0MC44NDQ1IDQ2LjUxNTEgNDAuMTI0MSA0Ni4wOTM1QzM4Ljc1MjIgNDUuMTY1NyAzNy41MDI4IDQzLjg4NTEgMzUuOTkxIDQyLjM0MjRDMzUuMjk4MiA0MS42MDQ0IDM0Ljc5NjYgNDAuOTAxOCAzMy45NzM1IDQwLjE5MDRDMzMuNzg2NiA0MC4wMjg5IDMzLjUwMTQgMzkuODEwNCAzMy4yOTIzIDM5LjY0NDJDMzIuNjQ4OSAzOS4xMjg4IDMxLjg5IDM4Ljg2IDMxLjE1NyAzOC44MzQ4QzMwLjIxNDcgMzguODAyNCAyOS4zMDc1IDM5LjE3MjUgMjguNzEzOCAzOS45MjA2QzI3LjY1ODQgNDEuMjUwNiAyNy45OTYzIDQzLjI4MzMgMjkuNDY3MSA0NC40NjE0QzI5LjQ4MiA0NC40NzM0IDI5LjQ5NzkgNDQuNDgyNyAyOS41MTI5IDQ0LjQ5NDNDMjkuNzE1IDQ0LjY1ODkgMjkuOTYyNSA0NC44Njk4IDMwLjE0ODMgNDUuMDA3NkMzMS4wMjE3IDQ1LjY1NTUgMzEuODE5NSA0NS45ODcyIDMyLjY4OTcgNDYuNTAxNUMzNC41MjMxIDQ3LjYzOTEgMzYuMDQzIDQ4LjU4MjMgMzcuMjQ4NiA0OS43MTk2QzM3LjcxOTQgNTAuMjIzNyAzNy44MDE2IDUxLjExMjIgMzcuODY0MyA1MS40OTY0TDM4Ljg0NjggNTIuMzc4MkMzMy41ODcyIDYwLjMzMDggMzEuMTUzIDcwLjE1MzkgMzIuNTkxNSA4MC4xNjI3TDMxLjMwNzcgODAuNTM3OEMzMC45NjkzIDgwLjk3NjggMzAuNDkxMiA4MS42Njc2IDI5Ljk5MTEgODEuODczOEMyOC40MTM4IDgyLjM3MjkgMjYuNjM4NyA4Mi41NTYyIDI0LjQ5NTYgODIuNzgxOUMyMy40ODk0IDgyLjg2NiAyMi42MjEzIDgyLjgxNTggMjEuNTU0NiA4My4wMTg4QzIxLjMxOTggODMuMDYzNSAyMC45OTI3IDgzLjE0OTEgMjAuNzM1OCA4My4yMDk3QzIwLjcyNjkgODMuMjExNiAyMC43MTg2IDgzLjIxNDIgMjAuNzA5NiA4My4yMTYyQzIwLjY5NTYgODMuMjE5NSAyMC42NzcyIDgzLjIyNjMgMjAuNjYzOCA4My4yMjk0QzE4Ljg1NyA4My42NjggMTcuNjk2MyA4NS4zMzY1IDE4LjA2OTkgODYuOTgwNUMxOC40NDM3IDg4LjYyNDggMjAuMjA4NiA4OS42MjQ4IDIyLjAyNjIgODkuMjMxMkMyMi4wMzkzIDg5LjIyODIgMjIuMDU4NCA4OS4yMjc3IDIyLjA3MiA4OS4yMjQ2QzIyLjA5MjYgODkuMjE5OSAyMi4xMTA2IDg5LjIwOTkgMjIuMTMxIDg5LjIwNDlDMjIuMzg0NCA4OS4xNDkgMjIuNzAxOSA4OS4wODY4IDIyLjkyMzYgODkuMDI3MkMyMy45NzIzIDg4Ljc0NTEgMjQuNzMxOCA4OC4zMzA2IDI1LjY3NDYgODcuOTY3N0MyNy43MDI5IDg3LjIzNjggMjkuMzgyOCA4Ni42MjYyIDMxLjAxOTUgODYuMzg4M0MzMS43MDMgODYuMzM0NSAzMi40MjMyIDg2LjgxMiAzMi43ODE0IDg3LjAxMzRMMzQuMTE3NyA4Ni43ODMxQzM3LjE5MjYgOTYuMzYxMyA0My42MzY2IDEwNC4xMDMgNTEuNzk2MyAxMDguOTYxTDUxLjIzOTYgMTEwLjMwM0M1MS40NDAzIDExMC44MjQgNTEuNjYxNiAxMTEuNTMgNTEuNTEyMSAxMTIuMDQ1QzUwLjkxNzEgMTEzLjU5NSA0OS44OTggMTE1LjIzMSA0OC43Mzc0IDExNy4wNTVDNDguMTc1NSAxMTcuODk4IDQ3LjYwMDQgMTE4LjU1MiA0Ny4wOTM0IDExOS41MTZDNDYuOTcyIDExOS43NDcgNDYuODE3NSAxMjAuMTAyIDQ2LjcwMDQgMTIwLjM0NkM0NS45MTI1IDEyMi4wMzkgNDYuNDkwNCAxMjMuOTkgNDguMDAzOCAxMjQuNzIyQzQ5LjUyNjggMTI1LjQ1OSA1MS40MTcxIDEyNC42ODIgNTIuMjM1MiAxMjIuOTg1QzUyLjIzNjQgMTIyLjk4MiA1Mi4yNDA2IDEyMi45OCA1Mi4yNDE3IDEyMi45NzhDNTIuMjQyNiAxMjIuOTc2IDUyLjI0MDkgMTIyLjk3MyA1Mi4yNDE3IDEyMi45NzFDNTIuMzU4MiAxMjIuNzMxIDUyLjUyMzMgMTIyLjQxNSA1Mi42MjE2IDEyMi4xODhDNTMuMDU2IDEyMS4xODkgNTMuMjAwNSAxMjAuMzMyIDUzLjUwNTkgMTE5LjM2NUM1NC4zMTcgMTE3LjMxOCA1NC43NjI2IDExNS4xNyA1NS44NzkxIDExMy44MzJDNTYuMTg0OSAxMTMuNDY2IDU2LjY4MzMgMTEzLjMyNSA1Ny4yMDAxIDExMy4xODZMNTcuODk0NCAxMTEuOTIyQzY1LjAwOCAxMTQuNjY1IDcyLjk3MDUgMTE1LjQwMiA4MC45MjQ1IDExMy41ODdDODIuNzM5MSAxMTMuMTczIDg0LjQ5MDggMTEyLjYzNyA4Ni4xODQzIDExMS45OTRDODYuMzc5NCAxMTIuMzQyIDg2Ljc0MiAxMTMuMDExIDg2LjgzOTMgMTEzLjE3OUM4Ny4zNjQ0IDExMy4zNTEgODcuOTM3NyAxMTMuNDM5IDg4LjQwNDcgMTE0LjEzM0M4OS4yNDAxIDExNS41NjcgODkuODExNCAxMTcuMjYzIDkwLjUwNzMgMTE5LjMxMkM5MC44MTI4IDEyMC4yNzkgOTAuOTYzOCAxMjEuMTM2IDkxLjM5ODEgMTIyLjEzNkM5MS40OTcxIDEyMi4zNjMgOTEuNjYxNCAxMjIuNjg0IDkxLjc3OCAxMjIuOTI1QzkyLjU5NDQgMTI0LjYyOCA5NC40OTA3IDEyNS40MDcgOTYuMDE1OSAxMjQuNjY5Qzk3LjUyOTIgMTIzLjkzNyA5OC4xMDc3IDEyMS45ODYgOTcuMzE5NCAxMjAuMjkzQzk3LjIwMjMgMTIwLjA0OSA5Ny4wNDEyIDExOS42OTUgOTYuOTE5OCAxMTkuNDY0Qzk2LjQxMjcgMTE4LjQ5OSA5NS44Mzc3IDExNy44NTIgOTUuMjc1OCAxMTcuMDA5Qzk0LjExNTIgMTE1LjE4NSA5My4xNTI2IDExMy42NyA5Mi41NTc1IDExMi4xMkM5Mi4zMDg3IDExMS4zMiA5Mi41OTk1IDExMC44MjMgOTIuNzkzMyAxMTAuMzAzQzkyLjY3NzIgMTEwLjE3IDkyLjQyODggMTA5LjQxNCA5Mi4yODI0IDEwOS4wNTlDMTAwLjc2MiAxMDQuMDI5IDEwNy4wMTcgOTUuOTk4NSAxMDkuOTU1IDg2LjcyMzlDMTEwLjM1MSA4Ni43ODY1IDExMS4wNDEgODYuOTA5MSAxMTEuMjY1IDg2Ljk1NDJDMTExLjcyNiA4Ni42NDg3IDExMi4xNDkgODYuMjUwMSAxMTIuOTgxIDg2LjMxNTlDMTE0LjYxNyA4Ni41NTM3IDExNi4yOTcgODcuMTY0NSAxMTguMzI2IDg3Ljg5NTNDMTE5LjI2OCA4OC4yNTgxIDEyMC4wMjggODguNjc5MyAxMjEuMDc3IDg4Ljk2MTRDMTIxLjI5OCA4OS4wMjEgMTIxLjYxNiA4OS4wNzY2IDEyMS44NjkgODkuMTMyNUMxMjEuODg5IDg5LjEzNzUgMTIxLjkwOCA4OS4xNDc1IDEyMS45MjggODkuMTUyMkMxMjEuOTQyIDg5LjE1NTMgMTIxLjk2MSA4OS4xNTU4IDEyMS45NzQgODkuMTU4OEMxMjMuNzkyIDg5LjU1MiAxMjUuNTU3IDg4LjU1MjYgMTI1LjkzIDg2LjkwODFDMTI2LjMwMyA4NS4yNjQxIDEyNS4xNDMgODMuNTk1MLine truncated
keysOrder: [["apiVersion"], ["appVersion"], ["kind"], ["metadata"], ["metadata", "name"], ["spec", "cluster"], ["spec", "storageClass"], ["spec", "minReplicas"], ["spec", "maxReplicas"], ["spec", "instanceType"], ["spec", "diskSize"], ["spec", "roles"], ["spec", "resources"], ["spec", "gpus"], ["spec", "kubelet"], ["spec", "logSerialConsole"], ["spec", "maxUnhealthy"], ["spec", "nodeStartupTimeout"], ["spec", "readyUnknownTimeout"], ["spec", "readyFalseTimeout"], ["spec", "maxNodeProvisionTime"], ["spec", "version"], ["spec", "osImage"], ["spec", "talos"], ["spec", "talos", "version"], ["spec", "talos", "schematicID"], ["spec", "talos", "imageFactoryURL"], ["spec", "talos", "installerRepository"], ["spec", "talos", "registryMirrors"], ["spec", "talos", "imageFactoryCA"], ["spec", "substrate"], ["spec", "proxmox"], ["spec", "proxmox", "templateTags"], ["spec", "proxmox", "full"], ["spec", "proxmox", "network"], ["spec", "proxmox", "network", "bridge"], ["spec", "proxmox", "storage"], ["spec", "proxmox", "pool"], ["spec", "proxmox", "dnsServers"]]
keysOrder: [["apiVersion"], ["appVersion"], ["kind"], ["metadata"], ["metadata", "name"], ["spec", "cluster"], ["spec", "storageClass"], ["spec", "minReplicas"], ["spec", "maxReplicas"], ["spec", "instanceType"], ["spec", "diskSize"], ["spec", "roles"], ["spec", "resources"], ["spec", "gpus"], ["spec", "kubelet"], ["spec", "logSerialConsole"], ["spec", "maxUnhealthy"], ["spec", "nodeStartupTimeout"], ["spec", "readyUnknownTimeout"], ["spec", "readyFalseTimeout"], ["spec", "maxNodeProvisionTime"], ["spec", "version"], ["spec", "osImage"], ["spec", "talos"], ["spec", "talos", "version"], ["spec", "talos", "schematicID"], ["spec", "talos", "imageFactoryURL"], ["spec", "talos", "installerRepository"], ["spec", "talos", "registryMirrors"], ["spec", "talos", "sysctls"], ["spec", "talos", "imageFactoryCA"], ["spec", "substrate"], ["spec", "proxmox"], ["spec", "proxmox", "templateTags"], ["spec", "proxmox", "full"], ["spec", "proxmox", "network"], ["spec", "proxmox", "network", "bridge"], ["spec", "proxmox", "storage"], ["spec", "proxmox", "pool"], ["spec", "proxmox", "dnsServers"]]
secrets:
exclude: []
include: []