On a fresh install the platform turns GPUs for VMs on, and so passes
--enable-gpu, while KubeVirt's operator may not have installed the VMI,
VM and KubeVirt CRDs yet. An informer on a kind no CRD serves cannot
start, and cozystack-controller exited on it, taking every reconciler
down with the GPU paths.
Now nothing outside the GPU paths waits for KubeVirt. The binder starts
with its other watches and reads no VMI, VM or KubeVirt object while
their kinds are not served, so finalizers, usage records and placement
keep running, and joins the KubeVirt watches from a runnable once the
kinds appear. While a device is attached, an unserved kind is a gap in
discovery rather than an absent VMI, and the pass fails instead of
detaching a running VM's device.
Every replica warms the informers the webhooks read, and readiness
fails until those that need no KubeVirt have synced. Readiness does not
wait for KubeVirt itself: cozystack-api, the dashboard and the lineage
webhook wait for cozystack-controller to be ready, and must not wait
for KubeVirt, nor hang when it is not installed. The VMI and KubeVirt
informers start once their kinds are served.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The binder is the only writer of a device's placement, so budget and
capacity checks that admission already ran cannot be raced past by two
replicas or two webhook calls. Every event collapses into one pass over
all GPU objects, which serializes the decisions without per-object
locks, and devices are read past the cache so a unit booked in the last
pass is never booked again from a stale view.
A device that carries an unanswered bind request also queues a pass
that only answers requests, so the webhook, which waits seconds, is not
held behind moves, usage records and status of the whole cluster. The
full pass moves held sets off nodes that cannot keep them, rebinds Lost
sets before new orders, and binds new orders onto the fullest node with
room. Finalizers and usage records run with the feature disabled too,
so turning GPU off never hangs a namespace or class deletion.
The devices one VM names move, go Lost and are rebound as a set. Every
pass restores that invariant before it places anything, gathering a set
left partly Lost, by a write that failed half way or by a VMI that ended
next to held members, onto one node or sending all of it to Lost, so
each later step only ever changes whole sets.
A Lost device already counts against its tenant's budgets, so the pod of
its still running VMI is not counted again under a raw GPU key; for node
capacity it is, since the device itself holds no units.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Assisted-by: LLM
The binder, the admission webhooks, the tenant chart, the policies and
cozystack-api all have to agree on the same label, annotation,
finalizer and quota keys and on the settings ConfigMap the policies
take as a parameter, so they are defined once here.
The binder in the leader is the only writer of placement; webhooks
running in every replica record an attachment and a bind request in
the device status and wait for the binder's answer, so capacity and
budgets are decided in one serialized loop. Status is the carrier
because no tenant role can write it. The template-device mark carries
no VMI name, since a VMI name can exceed a label value.
Register and Setup exist as no-ops so the controller can be wired to
them before the webhooks and reconcilers land.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Aleksei Sviridkin