mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
Aleksei Sviridkin
a349c753ad
A node group is sized either by instanceType or by explicit resources, but the generated values schema required resources on every node group. A node group with instanceType and no resources — or an empty resources dropped during typed-CR to HelmRelease translation — failed schema validation with "missing property 'resources'" even though sizing was fully specified. Mark resources optional in the node group definition and regenerate the schema, API types, README, and ApplicationDefinition. The chart template emits domain.cpu/memory only when resources is present, so a node group sized purely by instanceType now passes validation and installs. When a node group sets both instanceType and explicit resources, explicit resources take precedence: the instancetype is omitted from the rendered VM and the per-group instancetype lookup is skipped, so KubeVirt accepts the explicit domain.cpu/memory with no instancetype to conflict with. instanceType stays required and defaulted (no schema change), so the console's required instanceType dropdown keeps working — sending the default instanceType plus explicit resources simply sizes the VM by resources. This replaces an earlier render-time hard failure on the both-set case, which was unreachable to clear from the dashboard (the required dropdown cannot be emptied). Explicit resources are all-or-nothing alongside an instanceType: a partial spec (only cpu, or only memory) on a group that carries a non-empty instanceType would emit one domain override next to the instancetype — which KubeVirt rejects — so it now fails at render time with a clear error. Partial resources on a group without an instanceType (used to tune kubelet reservations) are unaffected. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
304 lines
15 KiB
YAML
304 lines
15 KiB
YAML
##
|
|
## @section Common Parameters
|
|
##
|
|
|
|
## @param {string} storageClass - Default StorageClass inside the tenant cluster. Remote-accessible LINSTOR classes are auto-propagated to the tenant under the same name.
|
|
## @immutable
|
|
## @x-cozystack-options {source: storageclass}
|
|
storageClass: replicated
|
|
|
|
##
|
|
## @section Application-specific Parameters
|
|
##
|
|
|
|
## @enum {string} ResourcesPreset - Default sizing preset.
|
|
## @value t1.nano
|
|
## @value t1.micro
|
|
## @value t1.small
|
|
## @value t1.medium
|
|
## @value t1.large
|
|
## @value t1.xlarge
|
|
## @value t1.2xlarge
|
|
## @value t1.4xlarge
|
|
## @value c1.nano
|
|
## @value c1.micro
|
|
## @value c1.small
|
|
## @value c1.medium
|
|
## @value c1.large
|
|
## @value c1.xlarge
|
|
## @value c1.2xlarge
|
|
## @value c1.4xlarge
|
|
## @value s1.nano
|
|
## @value s1.micro
|
|
## @value s1.small
|
|
## @value s1.medium
|
|
## @value s1.large
|
|
## @value s1.xlarge
|
|
## @value s1.2xlarge
|
|
## @value s1.4xlarge
|
|
## @value u1.nano
|
|
## @value u1.micro
|
|
## @value u1.small
|
|
## @value u1.medium
|
|
## @value u1.large
|
|
## @value u1.xlarge
|
|
## @value u1.2xlarge
|
|
## @value u1.4xlarge
|
|
## @value m1.nano
|
|
## @value m1.micro
|
|
## @value m1.small
|
|
## @value m1.medium
|
|
## @value m1.large
|
|
## @value m1.xlarge
|
|
## @value m1.2xlarge
|
|
## @value m1.4xlarge
|
|
## @value nano
|
|
## @value micro
|
|
## @value small
|
|
## @value medium
|
|
## @value large
|
|
## @value xlarge
|
|
## @value 2xlarge
|
|
|
|
## @typedef {struct} Resources - Explicit CPU and memory configuration for a node or component.
|
|
## @field {quantity} [cpu] - CPU available.
|
|
## @field {quantity} [memory] - Memory (RAM) available.
|
|
|
|
## @typedef {struct} GPU - GPU configuration.
|
|
## @field {string} name - Name of GPU, such as "nvidia.com/AD102GL_L40S".
|
|
## @x-cozystack-options {source: gpu}
|
|
|
|
## @typedef {struct} Kubelet - Kubelet resource reservations for worker nodes.
|
|
## @field {string} [systemReservedMemory] - Memory reserved for host OS. Auto-computed from instanceType if empty.
|
|
## @field {string} [kubeReservedMemory] - Memory reserved for kubelet and container runtime. Auto-computed from instanceType if empty.
|
|
## @field {string} [systemReservedCpu] - CPU reserved for host OS. Auto-computed from instanceType if empty.
|
|
## @field {string} [kubeReservedCpu] - CPU reserved for kubelet and container runtime. Auto-computed from instanceType if empty.
|
|
## @field {string} [evictionHardMemory]="7%" - Hard eviction threshold for memory (absolute like 200Mi or percentage like 7%).
|
|
## @field {string} [evictionSoftMemory]="10%" - Soft eviction threshold for memory (absolute like 1Gi or percentage like 10%).
|
|
|
|
## @typedef {struct} NodeGroup - Worker node group configuration.
|
|
## @field {int} minReplicas=0 - Minimum number of replicas.
|
|
## @field {int} maxReplicas=10 - Maximum number of replicas.
|
|
## @field {string} instanceType="u1.medium" - Virtual machine instance type.
|
|
## @x-cozystack-options {source: instancetype}
|
|
## @field {quantity} diskSize="20Gi" - System disk size for the worker VM. Carries the Talos OS image (factory.talos.dev raw artifact streamed in by CDI), kubelet state, containerd image cache, and any local-path PVCs. Pre-Talos installs used a separate disk-kubelet PVC for kubelet/containerd state; on Talos this is consolidated onto the single system disk imaged from the factory artifact.
|
|
## @field {string} [storageClass] - StorageClass for worker node persistent disks. When empty, uses the management cluster default StorageClass (the one annotated storageclass.kubernetes.io/is-default-class: true). NOTE: deliberately not marked immutable — the field is optional and undefaulted, so a strict `self == oldSelf` rule would block any future attempt to set it on an existing node group.
|
|
## @x-cozystack-options {source: storageclass}
|
|
## @field {[]string} roles - List of node roles.
|
|
## @field {Resources} [resources] - Explicit CPU and memory for each worker node, as an alternative to `instanceType` sizing. Optional: when omitted, the node is sized by `instanceType`. When both `cpu` and `memory` are set, they take precedence and `instanceType` is ignored for that node group (the instancetype is omitted from the VM, since KubeVirt cannot override an instancetype's CPU/memory). Set both `cpu` and `memory` together or neither; setting only one is rejected at render time.
|
|
## @field {[]GPU} gpus - List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM).
|
|
## @field {Kubelet} [kubelet] - Kubelet resource reservations for this node group.
|
|
|
|
## @param {map[string]NodeGroup} nodeGroups - Worker nodes configuration map. When left empty, a single default node group `md0` (one `ingress-nginx` worker) is provisioned. Provide your own groups to take full control — they are not merged with the default, so you may name and omit groups freely.
|
|
nodeGroups: {}
|
|
|
|
##
|
|
## @enum {string} Version
|
|
## @value v1.35
|
|
## @value v1.34
|
|
## @value v1.33
|
|
## @value v1.32
|
|
## @value v1.31
|
|
|
|
## @param {Version} version - Kubernetes major.minor version to deploy
|
|
version: "v1.35"
|
|
|
|
|
|
## @param {string} host - External hostname for Kubernetes cluster. Defaults to `<cluster-name>.<tenant-host>` if empty.
|
|
host: ""
|
|
|
|
##
|
|
## @section Cluster Addons
|
|
##
|
|
|
|
## @typedef {struct} CertManagerAddon - cert-manager addon.
|
|
## @field {bool} enabled - Enable cert-manager.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} CiliumAddon - Cilium CNI plugin.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} GatewayAPIAddon - Gateway API addon.
|
|
## @field {bool} enabled - Enable Gateway API.
|
|
|
|
## @enum {string} IngressNginxExposeMethod - Method to expose the controller
|
|
## @value Proxied
|
|
## @value LoadBalancer
|
|
|
|
## @typedef {struct} IngressNginxAddon - Ingress-NGINX controller.
|
|
## @field {bool} enabled - Enable the controller (requires nodes labeled `ingress-nginx`).
|
|
## @field {IngressNginxExposeMethod} exposeMethod - Method to expose the controller. Allowed values: `Proxied`, `LoadBalancer`.
|
|
## @field {[]string} hosts - Domains routed to this tenant cluster when `exposeMethod` is `Proxied`.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} GPUOperatorAddon - NVIDIA GPU Operator.
|
|
## @field {bool} enabled - Enable GPU Operator.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} FluxCDAddon - FluxCD GitOps operator.
|
|
## @field {bool} enabled - Enable FluxCD.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} HAMiAddon - HAMi GPU virtualization middleware.
|
|
## @field {bool} enabled - Enable HAMi (requires GPU Operator).
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} MonitoringAgentsAddon - Monitoring agents (Fluent Bit, VMAgents).
|
|
## @field {bool} enabled - Enable monitoring agents.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} VerticalPodAutoscalerAddon - Vertical Pod Autoscaler.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} VeleroAddon - Velero backup and recovery addon.
|
|
## @field {bool} enabled - Enable Velero.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} CoreDNSAddon - CoreDNS addon.
|
|
## @field {object} valuesOverride - Custom Helm values overrides.
|
|
|
|
## @typedef {struct} OuroborosAddon - Hairpin-NAT fix for ingress-nginx with PROXY-protocol. See cozystack/website docs/networking/hairpin-proxy-protocol for the operator guide.
|
|
## @field {bool} enabled - Enable ouroboros. Requires addons.ingressNginx.enabled (chart-render fail otherwise). Only useful when PROXY-protocol is wired on the tenant ingress-nginx via valuesOverride.
|
|
## @field {object} valuesOverride - Custom Helm values overrides. Operator-key wins over cozystack defaults.
|
|
|
|
## @typedef {struct} Addons - Cluster addons configuration.
|
|
## @field {CertManagerAddon} certManager - Cert-manager addon.
|
|
## @field {CiliumAddon} cilium - Cilium CNI plugin.
|
|
## @field {GatewayAPIAddon} gatewayAPI - Gateway API addon.
|
|
## @field {IngressNginxAddon} ingressNginx - Ingress-NGINX controller.
|
|
## @field {GPUOperatorAddon} gpuOperator - NVIDIA GPU Operator.
|
|
## @field {HAMiAddon} hami - HAMi GPU virtualization middleware.
|
|
## @field {FluxCDAddon} fluxcd - FluxCD GitOps operator.
|
|
## @field {MonitoringAgentsAddon} monitoringAgents - Monitoring agents.
|
|
## @field {VerticalPodAutoscalerAddon} verticalPodAutoscaler - Vertical Pod Autoscaler.
|
|
## @field {VeleroAddon} velero - Velero backup/restore addon.
|
|
## @field {CoreDNSAddon} coredns - CoreDNS addon.
|
|
## @field {OuroborosAddon} ouroboros - Hairpin-NAT fix for ingress-nginx with PROXY-protocol.
|
|
|
|
## @param {Addons} addons - Cluster addons configuration.
|
|
addons:
|
|
certManager:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
cilium:
|
|
valuesOverride: {}
|
|
gatewayAPI:
|
|
enabled: false
|
|
ingressNginx:
|
|
enabled: false
|
|
exposeMethod: Proxied
|
|
hosts: []
|
|
valuesOverride: {}
|
|
gpuOperator:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
hami:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
fluxcd:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
monitoringAgents:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
verticalPodAutoscaler:
|
|
valuesOverride: {}
|
|
velero:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
coredns:
|
|
valuesOverride: {}
|
|
ouroboros:
|
|
enabled: false
|
|
valuesOverride: {}
|
|
|
|
##
|
|
## @section Kubernetes Control Plane Configuration
|
|
##
|
|
|
|
## @typedef {struct} APIServer - API Server configuration.
|
|
## @field {Resources} resources - CPU and memory resources for API Server.
|
|
## @field {ResourcesPreset} resourcesPreset="c1.medium" - Preset if `resources` omitted.
|
|
## @field {[]string} extraArgs - Extra command-line flags appended to the tenant kube-apiserver, passed through to KamajiControlPlane `spec.apiServer.extraArgs`. Use for OIDC SSO (`--oidc-issuer-url`, `--oidc-client-id`) or structured authentication (`--authentication-config=/etc/kubernetes/authentication/config.yaml`, `--requestheader-uid-headers=X-Remote-Uid`). Empty by default (no change to current behavior).
|
|
## @field {[]object} extraVolumes - Extra volumes added to the control-plane Deployment, passed through to KamajiControlPlane `spec.deployment.extraVolumes`. Use to mount a ConfigMap or Secret holding an AuthenticationConfiguration file referenced by `extraArgs`. Each item is a core/v1 Volume, but because the control-plane pod runs on the management cluster only `configMap` and `secret` sources are allowed (host-reaching sources like `hostPath`/`csi` and token-projection via `projected` are rejected); each volume must have a unique, non-empty name and exactly one source. The names `talos-ca` and `talos-tls-cert` are reserved by the chart. Empty by default.
|
|
## @field {[]object} extraVolumeMounts - Extra volume mounts added to the tenant kube-apiserver container, passed through to KamajiControlPlane `spec.apiServer.extraVolumeMounts`. Each `name` must reference a volume declared in `extraVolumes`; the chart-managed talos secret volumes cannot be mounted. Each item is a core/v1 VolumeMount. Empty by default.
|
|
|
|
## @typedef {struct} ControllerManager - Controller Manager configuration.
|
|
## @field {Resources} resources - CPU and memory resources for Controller Manager.
|
|
## @field {ResourcesPreset} resourcesPreset="t1.micro" - Preset if `resources` omitted.
|
|
|
|
## @typedef {struct} Scheduler - Scheduler configuration.
|
|
## @field {Resources} resources - CPU and memory resources for Scheduler.
|
|
## @field {ResourcesPreset} resourcesPreset="t1.micro" - Preset if `resources` omitted.
|
|
|
|
## @typedef {struct} KonnectivityServer - Konnectivity Server configuration.
|
|
## @field {Resources} resources - CPU and memory resources for Konnectivity.
|
|
## @field {ResourcesPreset} resourcesPreset="t1.micro" - Preset if `resources` omitted.
|
|
|
|
## @typedef {struct} Konnectivity - Konnectivity configuration.
|
|
## @field {KonnectivityServer} server - Konnectivity Server configuration.
|
|
|
|
## @typedef {struct} ControlPlane - Kubernetes control plane configuration.
|
|
## @field {int} replicas=2 - Number of control-plane replicas.
|
|
## @field {APIServer} apiServer - API Server configuration.
|
|
## @field {ControllerManager} controllerManager - Controller Manager configuration.
|
|
## @field {Scheduler} scheduler - Scheduler configuration.
|
|
## @field {Konnectivity} konnectivity - Konnectivity configuration.
|
|
|
|
## @param {ControlPlane} controlPlane - Kubernetes control-plane configuration.
|
|
controlPlane:
|
|
replicas: 2
|
|
apiServer:
|
|
resources: {}
|
|
resourcesPreset: "c1.medium"
|
|
extraArgs: []
|
|
extraVolumes: []
|
|
extraVolumeMounts: []
|
|
controllerManager:
|
|
resources: {}
|
|
resourcesPreset: "t1.micro"
|
|
scheduler:
|
|
resources: {}
|
|
resourcesPreset: "t1.micro"
|
|
konnectivity:
|
|
server:
|
|
resources: {}
|
|
resourcesPreset: "t1.micro"
|
|
|
|
## @typedef {struct} Images - Optional image overrides for chart-internal helpers.
|
|
## @field {string} [waitForKubeconfig] - Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.
|
|
## @field {string} [kubectl] - Image used by the bootstrap-token tenant Job (kubectl). Empty falls back to images/kubectl.tag.
|
|
## @field {string} [talosCsrSigner] - Image used by the talos-csr-signer sidecar in the Kamaji control plane. Empty falls back to images/talos-csr-signer.tag.
|
|
|
|
## @param {Images} images - Optional image overrides for air-gapped or rate-limited registries.
|
|
images:
|
|
waitForKubeconfig: ""
|
|
kubectl: ""
|
|
talosCsrSigner: ""
|
|
|
|
##
|
|
## @section Talos Worker Image
|
|
##
|
|
|
|
## @typedef {struct} Talos - Talos worker image configuration.
|
|
## @field {string} version - Talos release used for worker OS image and installer. Must satisfy the chart's Talos<->Kubernetes support matrix against the chosen `version`.
|
|
## @field {string} schematicID - Talos image-factory schematic ID. Defaults to the cozystack-tested vanilla schematic. Operators using custom schematics (system extensions, kernel args) override here.
|
|
|
|
## @param {Talos} talos - Talos worker image configuration.
|
|
talos:
|
|
version: "v1.13.0"
|
|
schematicID: "ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515"
|
|
|
|
##
|
|
## @section Node Health Check
|
|
##
|
|
|
|
## @typedef {struct} NodeHealthCheck - MachineHealthCheck configuration for worker node groups.
|
|
## @field {string} maxUnhealthy="50%" - Maximum number of unhealthy nodes tolerated per node group before remediation is paused. The MHC admission webhook accepts either a bare integer ("0", "1", ...) or a percentage ("0%", "50%"); bare numeric strings are rejected, so the safer default is to express the value as a percentage. Default "50%" leaves headroom for transient unhealthy nodes during the kubeadm-to-Talos rollover and slow first boots from factory.talos.dev. Drop to "0%" once the fleet is stable on Talos workers.
|
|
## @field {string} nodeStartupTimeout="10m" - Maximum time a Machine is allowed to spend reaching the Ready condition before it is remediated. Raise for slow first boots (Talos image fetch from factory.talos.dev or a busy storage class on the kubevirt-csi PVC populator).
|
|
|
|
## @param {NodeHealthCheck} nodeHealthCheck - MachineHealthCheck tuning for worker node groups.
|
|
nodeHealthCheck:
|
|
maxUnhealthy: "50%"
|
|
nodeStartupTimeout: "10m"
|