On a fresh install the platform turns GPUs for VMs on, and so passes
--enable-gpu, while KubeVirt's operator may not have installed the VMI,
VM and KubeVirt CRDs yet. An informer on a kind no CRD serves cannot
start, and cozystack-controller exited on it, taking every reconciler
down with the GPU paths.
Now nothing outside the GPU paths waits for KubeVirt. The binder starts
with its other watches and reads no VMI, VM or KubeVirt object while
their kinds are not served, so finalizers, usage records and placement
keep running, and joins the KubeVirt watches from a runnable once the
kinds appear. While a device is attached, an unserved kind is a gap in
discovery rather than an absent VMI, and the pass fails instead of
detaching a running VM's device.
Every replica warms the informers the webhooks read, and readiness
fails until those that need no KubeVirt have synced. Readiness does not
wait for KubeVirt itself: cozystack-api, the dashboard and the lineage
webhook wait for cozystack-controller to be ready, and must not wait
for KubeVirt, nor hang when it is not installed. The VMI and KubeVirt
informers start once their kinds are served.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The NVIDIA and mediated entries of the KubeVirt permittedHostDevices
table are GPU resource names besides the hoster's gpu.resourceNames
list, whether the hoster sets one or not. Without that, a
GPU name only the table carries, on a node whose agent reports nothing
and with no class of its own, would not be a GPU resource name: under
forbidden the raw-name policy would let a VM or a pod request it
directly, and the VMI webhook would take it for a non-GPU host device.
The binder adds those names to the effective set it publishes in the
settings ConfigMap, which the policy and cozystack-api read, and the VMI
webhook adds them when it looks for a VMI's own GPU entries. A PCI entry
of another vendor, such as a ConnectX VF, stays out. Whether a class's
resource name is in the table at all is still checked against every
PCI and mediated entry.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder, the VMI webhook, the catalog option source and the usage
records each leave out of a pod's GPU usage the units its VMI's devices
already hold. They matched the pod to the VMI by its owner reference,
the usage records also by the device-reference label, and whoever
creates the pod writes both, so a pod naming a VMI with attached
devices could hide its GPUs from capacity, raw-key budgets and the
usage records.
A pod now counts as a VMI's launcher only when that VMI lists the pod's
UID in status.activePods; any other pod's GPUs count as raw-name usage.
One shared reader decides it for all four. The binder takes the active
pods from the VMIs it already lists, the webhook from the VMI informer
every replica warms, the usage records from a read of the named VMI,
and cozystack-api reads the named VMI only for a pod that requests a
raw GPU name, for which it gains get on virtualmachineinstances.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder, the admission webhooks, the catalog option source and the
tenant quota gate carried copies of the tenant release name mapping,
the resourceQuotas reader, the permittedHostDevices reader and the
per-VMI device coverage, and the copies had started to drift: the
catalog skipped a budget it could not read, where the others failed.
The VMI webhook still counted one unit per healthy GPU function when a
node had lost capacity, so it refused a VM whose devices an mdev
parent's remaining slots still served while the binder would have kept
them; the units a node serves are now counted once, for both.
They now live in the shared model, next to the tenant walk they depend
on. Coverage keeps two named readers on purpose: for budgets every
attached device covers its pod, since a Lost device still counts
itself, while for node capacity only a device holding units does.
Merging them would count a Lost device twice or not at all.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder is the only writer of a device's placement, so budget and
capacity checks that admission already ran cannot be raced past by two
replicas or two webhook calls. Every event collapses into one pass over
all GPU objects, which serializes the decisions without per-object
locks, and devices are read past the cache so a unit booked in the last
pass is never booked again from a stale view.
A device that carries an unanswered bind request also queues a pass
that only answers requests, so the webhook, which waits seconds, is not
held behind moves, usage records and status of the whole cluster. The
full pass moves held sets off nodes that cannot keep them, rebinds Lost
sets before new orders, and binds new orders onto the fullest node with
room. Finalizers and usage records run with the feature disabled too,
so turning GPU off never hangs a namespace or class deletion.
The devices one VM names move, go Lost and are rebound as a set. Every
pass restores that invariant before it places anything, gathering a set
left partly Lost, by a write that failed half way or by a VMI that ended
next to held members, onto one node or sending all of it to Lost, so
each later step only ever changes whole sets.
A Lost device already counts against its tenant's budgets, so the pod of
its still running VMI is not counted again under a raw GPU key; for node
capacity it is, since the device itself holds no units.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Assisted-by: LLM
The binder, the admission webhooks, the tenant chart, the policies and
cozystack-api all have to agree on the same label, annotation,
finalizer and quota keys and on the settings ConfigMap the policies
take as a parameter, so they are defined once here.
The binder in the leader is the only writer of placement; webhooks
running in every replica record an attachment and a bind request in
the device status and wait for the binder's answer, so capacity and
budgets are decided in one serialized loop. Status is the carrier
because no tenant role can write it. The template-device mark carries
no VMI name, since a VMI name can exceed a label value.
Register and Setup exist as no-ops so the controller can be wired to
them before the webhooks and reconcilers land.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Aleksei Sviridkin