The NVIDIA and mediated entries of the KubeVirt permittedHostDevices
table are GPU resource names besides the hoster's gpu.resourceNames
list, whether the hoster sets one or not. Without that, a
GPU name only the table carries, on a node whose agent reports nothing
and with no class of its own, would not be a GPU resource name: under
forbidden the raw-name policy would let a VM or a pod request it
directly, and the VMI webhook would take it for a non-GPU host device.
The binder adds those names to the effective set it publishes in the
settings ConfigMap, which the policy and cozystack-api read, and the VMI
webhook adds them when it looks for a VMI's own GPU entries. A PCI entry
of another vendor, such as a ConnectX VF, stays out. Whether a class's
resource name is in the table at all is still checked against every
PCI and mediated entry.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder, the VMI webhook, the catalog option source and the usage
records each leave out of a pod's GPU usage the units its VMI's devices
already hold. They matched the pod to the VMI by its owner reference,
the usage records also by the device-reference label, and whoever
creates the pod writes both, so a pod naming a VMI with attached
devices could hide its GPUs from capacity, raw-key budgets and the
usage records.
A pod now counts as a VMI's launcher only when that VMI lists the pod's
UID in status.activePods; any other pod's GPUs count as raw-name usage.
One shared reader decides it for all four. The binder takes the active
pods from the VMIs it already lists, the webhook from the VMI informer
every replica warms, the usage records from a read of the named VMI,
and cozystack-api reads the named VMI only for a pod that requests a
raw GPU name, for which it gains get on virtualmachineinstances.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The chart writes the device references and the webhook reads them, in
two packages with no shared code, so a renamed field on either side
would leave VMs the webhook refuses to start. The vm-instance
chart tests already pin the rendered annotation; this test reads each
pinned value back through the webhook's parser.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder, the admission webhooks, the catalog option source and the
tenant quota gate carried copies of the tenant release name mapping,
the resourceQuotas reader, the permittedHostDevices reader and the
per-VMI device coverage, and the copies had started to drift: the
catalog skipped a budget it could not read, where the others failed.
The VMI webhook still counted one unit per healthy GPU function when a
node had lost capacity, so it refused a VM whose devices an mdev
parent's remaining slots still served while the binder would have kept
them; the units a node serves are now counted once, for both.
They now live in the shared model, next to the tenant walk they depend
on. Coverage keeps two named readers on purpose: for budgets every
attached device covers its pod, since a Lost device still counts
itself, while for node capacity only a device holding units does.
Merging them would count a Lost device twice or not at all.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The GPU admission webhooks need a Service, a serving certificate and
webhook configurations, and the binder's metrics need a port a scrape
can reach. All of it renders only with GPU enabled, which the platform
sets where KubeVirt serves GPUs, so clusters without KubeVirt change
nothing.
The webhooks fail closed. The two mutating ones select only VMIs and
virt-launcher pods carrying the device-reference label, the pod webhook
also requiring the virt-launcher label, so of VMIs and pods a
controller outage blocks only the start of VMs with device references;
the GPUDevice webhook blocks new orders as well. The controller then
runs two replicas behind a PDB, since a single fail-closed replica
would block those starts on every rollout.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A tenant references held devices and device templates on the VMI
template; something has to turn those references into GPUs on one node
before KubeVirt renders the launcher pod, and refuse a start that cannot
work instead of leaving a VMI Pending with no pod.
The GPUDevice webhook refuses an order that cannot be met now: an
unknown or deleted class, maxCount, the class's free units or one
node's, a budget of the tenant or an ancestor, or a vGPU class without
the entitlement.
The VMI webhook runs every refusal check before it writes anything, so
a dry run and a refused start leave no trace. It then creates the
template devices, attaches the whole set with optimistic concurrency so
two VMIs never take one device, and waits a bounded time for the
leader's binder to place the set on one node. The API server calls it
again after virt-api's mutator changes the VMI, so a recent answered
request for the same VMI is taken as its own and nothing is written
twice. It adds a gpus entry per GPU and never a resources key as
well, since the launcher pod would request the sum.
The pod webhook pins the launcher pod to the devices' node with a
required node affinity on the pod, because LiveUpdate rewrites the
VMI's affinity from the VM template.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder, the admission webhooks, the tenant chart, the policies and
cozystack-api all have to agree on the same label, annotation,
finalizer and quota keys and on the settings ConfigMap the policies
take as a parameter, so they are defined once here.
The binder in the leader is the only writer of placement; webhooks
running in every replica record an attachment and a bind request in
the device status and wait for the binder's answer, so capacity and
budgets are decided in one serialized loop. Status is the carrier
because no tenant role can write it. The template-device mark carries
no VMI name, since a VMI name can exceed a label value.
Register and Setup exist as no-ops so the controller can be wired to
them before the webhooks and reconcilers land.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Aleksei Sviridkin