mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-04 02:57:27 +09:00
Aleksei Sviridkin
9352d3ee0e
The two existing variants assume the host has neither the NVIDIA driver
nor the container toolkit: default unbinds the host driver to bind
vfio-pci for KubeVirt VMs, vgpu requires the proprietary vGPU host
driver and a license server. Neither path produces a working setup on a
node that already ships the driver and nvidia-container-toolkit via the
distro package manager — the most common shape of a Linux GPU node.
The new container variant surfaces the third architectural mode that
upstream NVIDIA gpu-operator already supports: sandboxWorkloads
disabled, devicePlugin enabled, with every component that would
conflict with the host install switched off.
- driver.enabled=false use the host driver
- toolkit.enabled=false host already has nvidia-container-toolkit;
enabling overwrites the containerd
config.toml the host configured via
nvidia-ctk runtime configure
- devicePlugin.enabled=true publish nvidia.com/gpu to the kubelet
- vfioManager.enabled=false do not unbind the host driver
- cdi.enabled=false CDI spec generation is serviced by the
disabled toolkit; an apt host uses the
legacy nvidia-container-runtime hook, so
pin off to match the host runtime wiring
- sandboxWorkloads.enabled=false regular pod workloads, not VMs
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
44 lines
2.2 KiB
YAML
44 lines
2.2 KiB
YAML
# Defaults applied to every variant of the gpu-operator package.
|
|
# The variant-specific files (values-passthrough.yaml, values-vgpu.yaml,
|
|
# values-container.yaml) layer on top.
|
|
gpu-operator:
|
|
# Upstream chart v26.x flips ccManager.enabled to true and
|
|
# ccManager.defaultMode to "on". On Hopper-class hardware
|
|
# (H100/H200/B100) with Confidential Computing capability this would
|
|
# auto-enable CC mode after a chart upgrade — a non-trivial behavior
|
|
# change. On non-CC-capable GPUs (L40S, A100, T4, …) the DaemonSet
|
|
# deploys, probes the hardware, and no-ops, but it still enters the
|
|
# cluster's resource graph. Pin to false so the default footprint
|
|
# stays as it was on v25.x; users who want CC enable it explicitly.
|
|
ccManager:
|
|
enabled: false
|
|
# Upstream v26.x also flips vgpuDeviceManager.enabled to true. The
|
|
# primary reason this is pinned back to false: the device manager
|
|
# walks /sys/class/mdev_bus/, which does not exist on Ada Lovelace
|
|
# (L4/L40/L40S, …) or Blackwell (B100, …) on the vGPU 17/20 driver
|
|
# branch — it errors with «no parent devices found». Keeping the
|
|
# default-on would crashloop the DaemonSet on any modern GPU.
|
|
# Secondary consideration: in passthrough mode the operator does not
|
|
# apply the nvidia.com/gpu.deploy.vgpu-device-manager=true label, so
|
|
# the DaemonSet would stay at zero pods, but it would still render
|
|
# into the cluster. The vgpu variant explicitly re-affirms this in
|
|
# values-vgpu.yaml.
|
|
vgpuDeviceManager:
|
|
enabled: false
|
|
|
|
# Optional ConfigMap holding extra nvidia kernel module parameters.
|
|
# The upstream chart only references an existing ConfigMap by name via
|
|
# driver.kernelModuleConfig.name — it never creates it. When create is
|
|
# true this wrapper chart renders the ConfigMap into the release
|
|
# namespace so the params actually land. The driver entrypoint reads
|
|
# ${base}/nvidia.conf and appends each line to the nvidia modprobe
|
|
# options. Used by tenant Kubernetes clusters to set
|
|
# NVreg_NvLinkDisable=1 (see packages/apps/kubernetes gpu-operator
|
|
# HelmRelease defaults); off by default for host-level passthrough/vgpu
|
|
# variants where the in-host driver is disabled.
|
|
kernelModuleConfig:
|
|
create: false
|
|
name: nvidia-kernel-module-params
|
|
content: |
|
|
NVreg_NvLinkDisable=1
|