Every release-1.5 PR Build job dies at the 30-minute timeout, and all of
it is spent in one target. `image-ubuntu-container-disk` builds a disk
per Kubernetes minor -- six of them -- and each spends ~4-5min inside a
libguestfs appliance installing that minor's kubelet/kubeadm into its own
copy of the cloud image. Serially that is ~28min of the 30min budget:
v1.35 6:00, v1.34 4:21, v1.33 4:27, v1.32 4:39, v1.31 4:42, v1.30 cancelled
None of it is cached, and no warmer will fix that on its own. Only
build-main.yaml sets WRITE_CACHE=1, and main dropped this image from
`make image` in 39cdc8866 six days before release-1.5 was cut, so the
:*-buildcache refs it reads have never been written -- the CI log shows
all six importers failing with "not found". release-1.5 is now the only
branch that builds this image at all.
The six builds share no per-version work, so run them concurrently
instead. A sub-make carries its own -j, keeping the root `make build`
serial, and buildkit dedups the shared guestfish and cloud-image stages
across the concurrent solves. --output-sync=target groups each version's
log under its target name; six interleaved buildx progress streams have
no per-solve identifier and are unreadable. Verified the per-version
buildx/echo/rm command set is byte-identical to the loop it replaces
across PUBLISH_VERSIONED, PUBLISH_FLOATING and WRITE_CACHE combinations.
Moving off the 4cpu/16gb shape because six concurrent libguestfs
appliances want ~9gb of RAM and a core apiece. The comment justifying
the small shape assumed a warm cache, which no longer holds here.
This also stops swallowing failures. The loop separated versions with
`;`, so make saw only the last version's exit status: a failed build for
any other version left its committed .tag file stale and the job green.
Under the sub-make a failure in any version fails the target.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The flux-instance template rendered spec.distribution.artifact
unconditionally, unlike the sibling optional fields (artifactPullSecret,
imagePullSecret, variant). The umbrella default is empty to use the
operator's embedded manifests (the air-gapped default from #964), but an
empty value renders as YAML null. In the flux-operator v0.50.0 CRD the
field is optional yet typed 'string' with pattern ^oci://.*$ and is not
nullable, so the rendered null (like an empty string) fails schema
validation and the API rejects the FluxInstance.
Guest Kubernetes clusters with the fluxcd addon enabled could therefore
never install it: the sub-release looped install/uninstall, the parent
kubernetes release stayed InProgress, and the cluster never reached Ready.
Guard the field so an empty value is omitted (the operator falls back to
its embedded manifests, preserving the air-gapped default) while a
non-empty artifact still renders. The template lives in the vendored
subchart, so the change is carried as a patch applied by 'make update'.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
(cherry picked from commit da47b70e2c)
Two related issues caused the dashboard SPA to break with CORS errors
after the kc-access cookie expired:
1. KeycloakClient for dashboard had no webOrigins set, so the operator
never touched the field in Keycloak after initial creation. After
any rootUrl change the existing webOrigins value became stale and
blocked CORS from the SPA. Setting "+" tells Keycloak to derive
allowed origins from redirectUris.
2. oauth2-proxy returned 302 -> Keycloak /auth for unauthenticated
XHR/fetch on /api and /apis. The login page does not emit
Access-Control-Allow-Origin, so the SPA's fetch failed with CORS.
--api-route makes oauth2-proxy answer 401 without redirect, so the
SPA can handle the re-auth itself via top-level navigation.
Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
(cherry picked from commit be8f6b4739)
A leftover debug field "foo" in the <release>-urls Secret template
violates the core/v1 Secret schema. It went unnoticed under client-side
apply but breaks Helm upgrades under server-side apply with
".foo: field not declared in schema", leaving the release in
UpgradeFailed state. Drop the debug block.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
(cherry picked from commit 70a2284551)
Refresh the vendored virt-operator manifest from v1.8.2 to v1.8.4. The
release backports validation of the VMI checksum status fields as a
uint32 range, required on Kubernetes 1.36 where strict CRD numeric
format validation otherwise leaves VMIs stuck in Scheduled. Remaining
changes are bug fixes and CVE remediations; the embedded CRDs and RBAC
are unchanged, so only the operator image tag moves.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit 192246e8b6)
Reverts #3006. Dropping persistent EFI/TPM was motivated by a live-migration
concern that rested on an outdated assumption — that an RWO Filesystem
backend-storage PVC pins the VM to its node. KubeVirt has migrated
RWO-Filesystem backend storage since v1.4 (kubevirt/kubevirt#12629): on
migration it creates a fresh target state PVC and copies the small state blob,
so persistent-EFI/TPM VMs on the default replicated storage live-migrate fine
(verified: the VM reports LiveMigratable=True and the copy-on-target PVC is
created on migration). Restore upstream's persistent firmware state for the
windows.* preferences.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
(cherry picked from commit 59f035ff53)
The vendored seaweedfs chart ships connection pool settings only for the
mysql store, which cozystack does not use. The postgres2 store, which is
enabled, has no pool settings, so the Go sql pool keeps zero idle
connections and every filer metadata lookup opens a fresh PostgreSQL
connection (TCP + TLS + SCRAM, ~300ms each). With ~5 path lookups per S3
operation this adds roughly 2 seconds of latency to every S3 request,
regardless of cluster load.
Measured on a production cluster (12 nodes, 2 filers): PUT of a 4KiB
object went from 10s (p50) to 0.28s, HEAD from 2.6s to 0.16s after
setting these values; pg_stat_activity confirmed connections were
previously recycled every ~200ms and are now reused.
max_open is capped at 40 so that two filer replicas stay below the CNPG
default max_connections of 100.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
(cherry picked from commit 49cbea8c73)
Deleting a Tenant whose etcd DataStore has 0 referencing TenantControlPlanes left the DataStore stuck Terminating (finalizer kamaji.clastix.io/TenantControlPlane), hanging the etcd helm uninstall and wedging the tenant namespace. Backports clastix/kamaji#1122 (fixes clastix/kamaji#1115) as a source patch on the pinned 26.3.6-edge build. Refs #3018.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 47638f8569)
The victoria-metrics-operator HelmRelease is gated on
monitoringAgents.enabled but listed cert-manager-crds in dependsOn
unconditionally, while the cert-manager-crds HelmRelease is only
created when certManager.enabled is true. The valid combination
monitoringAgents.enabled=true + certManager.enabled=false therefore
left vmop blocked on a non-existent dependency
("...cert-manager-crds ... not found"), cascading to monitoring-agents
and vertical-pod-autoscaler. Gate the dependency on the same condition
that creates it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 2b1d3e904d)
## What this PR does
Backport of #3121 to `release-1.5`.
Makes node group `resources` optional, so a node group sized purely by
`instanceType` (with `resources` omitted) passes `values.schema.json`
validation and installs. When a group sets both `instanceType` and
explicit `resources` (both `cpu` and `memory`), the explicit resources
take precedence: the instancetype is omitted from the rendered VM and
the per-group instancetype lookup is skipped, so KubeVirt does not
reject a VM that references an instancetype while overriding
`domain.cpu`/`memory`. A partial spec (only one of `cpu`/`memory`)
alongside an `instanceType` fails fast at render time.
Cherry-picked from `a349c753`; conflicts resolved against the
`release-1.5` tree (the management-cluster Talos worker bootstrap stack
present on `main` is not on this branch, so the schema/template/test
layout differ). Generated files (`values.schema.json`, `README.md`,
cozyrds, API types) regenerated with `make generate` to match. `helm
unittest` passes (134 tests), including the four new node-group sizing
cases adapted to this branch's document layout.
### Release note
```release-note
fix(kubernetes): make node group resources optional when instanceType is set
```
The failure-path dump queried mariadbs.k8s.mariadb.com with $name, but the
operator CR is named after the Helm release ({{ .Release.Name }} = mariadb-$name),
so the dump would hit NotFound instead of showing the rejected CR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
(cherry picked from commit f8e4222b7d)
PR #2279 made `replication.enabled: true` unconditional so the operator would
always create -primary/-secondary services. But the mariadb-operator validating
webhook (vmariadb-v1alpha1.kb.io) rejects replication when spec.replicas == 1,
so single-replica MariaDB could no longer be created — the HelmRelease never
became Ready.
Re-guard replication on replicas>1 and teach the service-name consumers about
the bare <release> service the operator provisions in both topologies:
- mariadb.yaml: gate `replication` on replicas>1; restore single-replica
external access via `service.type: LoadBalancer` and keep `primaryService`
LoadBalancer for replicas>1 only.
- backup-cronjob.yaml: route single-replica dumps to the bare service
(no -secondary exists without replication).
- dashboard-resourcemap.yaml: always grant RBAC on the bare service, plus
-primary/-secondary when replicas>1, matching the ApplicationDefinition.
- mariadb-rd: list the bare service in services.include so single-replica
installs are visible in the dashboard.
Adds a single-replica e2e test that reproduces the webhook rejection and
guards against regression.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
(cherry picked from commit c7e8f4d975)
The tenant etcd, ingress and monitoring HelmReleases create VM* custom
resources (VMPodScrape directly in etcd/ingress; VMCluster, VMAlert,
VLCluster, VMAlertmanager and VMServiceScrape via the monitoring-system
child of monitoring) guarded by the victoria-metrics-operator validating
webhook with failurePolicy: Fail. On a cold install the operator pod can
be briefly unavailable on :9443, so these releases reconcile before the
webhook backend is serving and are rejected with connection refused. They
carry install.remediation.retries: -1 and would eventually recover, but
the failed-install -> rollback -> retry churn can outlast a cold-install
readiness deadline and surfaces as spurious install failures.
Add a Flux spec.dependsOn from each of these releases to the
victoria-metrics-operator HelmRelease in cozy-victoria-metrics-operator,
so they wait for the operator to be Ready before creating VM* resources.
The system monitoring components gate on the operator the same way, via a
variant-level dependsOn. The cross-shard reference (tenant releases are
sharded, the operator release is not) resolves via helm-controller's
uncached dependency check (APIReader, v1.1.0+).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit 6db5ae9c6f)
A node group is sized either by instanceType or by explicit resources, but the
generated values schema required resources on every node group. A node group
with instanceType and no resources — or an empty resources dropped during
typed-CR to HelmRelease translation — failed schema validation with "missing
property 'resources'" even though sizing was fully specified.
Mark resources optional in the node group definition and regenerate the schema,
API types, README, and ApplicationDefinition. The chart template emits
domain.cpu/memory only when resources is present, so a node group sized purely
by instanceType now passes validation and installs.
When a node group sets both instanceType and explicit resources, explicit
resources take precedence: the instancetype is omitted from the rendered VM and
the per-group instancetype lookup is skipped, so KubeVirt accepts the explicit
domain.cpu/memory with no instancetype to conflict with. instanceType stays
required and defaulted (no schema change), so the console's required
instanceType dropdown keeps working — sending the default instanceType plus
explicit resources simply sizes the VM by resources. This replaces an earlier
render-time hard failure on the both-set case, which was unreachable to clear
from the dashboard (the required dropdown cannot be emptied).
Explicit resources are all-or-nothing alongside an instanceType: a partial spec
(only cpu, or only memory) on a group that carries a non-empty instanceType
would emit one domain override next to the instancetype — which KubeVirt rejects
— so it now fails at render time with a clear error. Partial resources on a
group without an instanceType (used to tune kubelet reservations) are unaffected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit a349c753ad)
The seaweedfs COSI driver shipped at v0.1.2 hardcodes read-write S3
actions (Read, Write, List, Tagging) for every BucketAccess and never
reads the accessPolicy parameter. The chart already defines a -readonly
BucketAccessClass carrying parameters.accessPolicy: readonly, but v0.1.2
ignores it, so a credential meant to be read-only can still write
(upload and delete) objects.
Bump the driver to v0.3.1, which honours accessPolicy: readonly by
issuing Read,List-only credentials (read-write and the empty default
stay Read,Write,List,Tagging). The override lives in the package
values.yaml rather than the vendored chart so it survives make update;
the sister extra/seaweedfs chart already runs v0.3.1.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit 0c6426feaa)
The cozy-harbor wrapper never passed storageClass to the jobservice
jobLog PVC, so it was created without a storageClassName and the
default-StorageClass admission stamped the cluster default. Because PVC
storageClassName is immutable, this caused rendered-vs-live drift on
upgrade. Preserve an existing PVC's class via lookup and fall back to the
configured storageClass for fresh installs.
The trivy StatefulSet's volumeClaimTemplate had the same latent drift --
.Values.storageClass was forwarded blindly into a VCT that cannot be
patched at all -- so it gets the same lookup-preserve treatment, keyed
off data-<release>-trivy-0.
A live object pinned to an explicit "" (no dynamic provisioning) now
round-trips through upstream's "-" sentinel instead of collapsing to an
omitted storageClassName, and the spec lookup is guarded with
`| default dict` so a PVC carrying no storageClassName key stays omitted.
Adds a helm-unittest regression suite and wires a test target so
make unit-tests stops skipping packages/apps/harbor.
Fixes#2368
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit bdf520bf74)
The s3-service-name.patch renamed the S3 Service to <name>-s3
(seaweedfs-s3), but several consumers still resolved it through
componentName -> <fullname>-s3. name and fullname only coincide when the
release name equals the chart name; the shipped release is
seaweedfs-system, so they pointed at seaweedfs-system-s3 while the real
service stayed seaweedfs-s3:
- the S3 ingress backend and the iceberg ingress backend, which left
ingress-nginx returning 503 on the external S3 endpoint.
The COSI provisioner in-cluster ENDPOINT was wrong on the same axis and
also on the port. Both the s3-gateway branch and the filer-S3 branch
rendered a portless URL that defaults to 443, while the s3 service only
listens on the S3 port (8333); the filer branch additionally targeted
the filer service, which serves no S3. Both in-cluster branches now
resolve seaweedfs-s3 on the configured S3 port.
The default path (s3.ingress.enabled=true) is repaired by the ingress
fix; the COSI in-cluster paths are latent until an operator disables the
ingress. Resolve every consumer through the same name helper the Service
uses, and pin the contract with unittests rendered under the production
release name so the mismatch cannot recur.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit 8a14de9abf)
backupstrategy-controller creates the cozy-backups Bucket into the
tenant-root namespace, which cozystack-basics provisions. On a fresh
install the Bucket patch can race namespace creation and fail with
`namespaces "tenant-root" not found`, which then races the install
wait deadline and fails the run — on changes that have nothing to do
with backup ordering. Add the cozystack-basics dependsOn edge so the
target namespace exists first, pinned with a helm-unittest regression
guard.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit ee853b00bd)
entityOperator.topicOperator/userOperator carried no resources, so in tenant
namespaces (the LimitRange defaults containers to 128Mi) the JVM topic-operator
and user-operator are OOMKilled on startup. The entity-operator stays in
CrashLoopBackOff, KafkaTopics/KafkaUsers never reconcile, and on teardown topics
get stuck on the strimzi.io/topic-operator finalizer. Set explicit requests/limits
(256Mi/512Mi) so they no longer inherit the namespace default; add a template test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 1ab5f17cac)
TEMPORARY revert. Restoring persistent EFI/TPM state (4cc60e3) makes
windows.11/2k22/2k25 VMs get a backend-storage PVC that is ReadWriteOnce on the
default StorageClass, which pins the VM to its node and blocks live-migration and
node drains (with evictionStrategy: LiveMigrate this can stall cluster upgrades).
See #3005.
Strip persistent EFI/TPM from the preferences again, but with `yq del` so an
emptied preferredTPM stays `{}` (the v1.8 CRD rejects null) — the bug the original
sed strip caused. Secure Boot and the vTPM stay present, just non-persistent.
Stopgap until vm-state can request an RWX backend (RWX-Block via the upstream
KubeVirt change, or an RWX-Filesystem vmStateStorageClass).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
(cherry picked from commit 6ac2601a68)
Merge the AI-generated v1.5.0 changelog with a fuller pass:
- Add an "Upgrade Notes and Required Actions" section (K8s 1.33+, removal
of upgrade.force, GPU permittedHostDevices ownership, MetalLB FRR-K8s /
HTTPS metrics, automatic TLS on externally-published services).
- Expand "Platform Components" with per-component upstream highlights and
CVE references (Flux v2.8 CVE-2026-23990, MetalLB v0.16 CVE-2025-22874),
plus the new frr-k8s subchart and kuberture package.
- Scope the storageClass-immutability entry (#2639) to UI-only enforcement,
with apiserver enforcement tracked in #2657.
- Broaden Development/Testing/CI coverage and add the v1.4.x website docs
backports (#565, #566, #567, #580).
- Correct the loop-device LVM-filter attribution to talm#215 (the prior
ansible-cozystack#51 reference was a dependency bump, not this change).
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
The `Release E2E` workflow's build job logged into `iad.ocir.io` using
`secrets.OCI_REGISTRY_USERNAME` / `secrets.OCI_REGISTRY_PASSWORD` —
secret names that don't exist in the repo. `docker/login-action`
therefore received empty credentials and failed at the very first step
with `Username and password required`, before any image was built or any
test ran.
The workflow has **never** succeeded since it was added (2026-05-02):
all three tag runs to date died at this step — `v1.5.0-rc.2`,
`v1.5.0-rc.3`, and `v1.5.0`.
This switches the login to the configured `OCIR_USER` / `OCIR_TOKEN`
secrets — the same ones `build-main.yaml` and `pull-requests.yaml` use
against the same registry — and adds the per-job Docker config isolation
those workflows already use (`Set up Docker config` step +
`DOCKER_CONFIG` env on the login and build steps), so the login doesn't
clobber the shared `~/.docker/config.json` on the self-hosted runner.
### Release note
```release-note
ci(tests): fix OCIR registry login in the Release E2E workflow so it can build and run the full suite on release tags
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Improved the release workflow with enhanced Docker credential
isolation and configuration handling during the build process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Release E2E build job referenced secrets OCI_REGISTRY_USERNAME /
OCI_REGISTRY_PASSWORD, which do not exist in the repo, so
docker/login-action received empty credentials and failed every run
with "Username and password required". The workflow has never built
since it was added (2026-05-02): v1.5.0-rc.2, v1.5.0-rc.3 and v1.5.0
all died at this step before any image was built or test run.
Use the configured OCIR_USER / OCIR_TOKEN secrets, matching
build-main.yaml and pull-requests.yaml, and isolate the Docker config
per job (Set up Docker config step + DOCKER_CONFIG on the login and
build steps) so login does not clobber the shared ~/.docker/config.json
on the self-hosted runner.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Implements the tenant Flux sharding design from #2817: a new
`flux-shard-operator` system package that spreads tenant HelmReleases
across multiple helm-controller shards, so a noisy tenant (e.g. a
HelmRelease stuck in infinite remediation) cannot degrade the others.
- **Placement controller** owns the tenant→shard assignment: the unit of
placement is the tenant (all of its HelmReleases share one shard label),
distribution is greedy least-loaded weighted by HelmRelease count, with
pinned tenants, threshold-gated rebalance, paced moves, and never moving
deleting tenants. Watches are metadata-only, so the controller does not
decode the helm-controller status-patch firehose. Assignments are
recorded on tenant namespaces; HelmRelease labels remain the source of
truth on restarts.
- **Shard runtime** is provisioned by the operator: `shardCount`
helm-controller Deployments cloned from the flux-aio `flux` Deployment
and sanitised, so the image and feature-gates stay version-synced
automatically. flux-aio itself is unchanged and keeps owning everything
without a shard key.
- **CREATE-only mutating webhook** (`failurePolicy: Ignore`) stamps the
tenant's shard onto every HelmRelease at admission, so each one is born
on the correct shard regardless of creation path; any miss degrades
gracefully to the placement controller.
- The hand-rolled `flux-tenants` Deployment is retired: the operator
drains it (backfill `tenants` → `shard<i>`) and deletes it once empty;
migration 44 removes a leftover only under the same guard.
Ships with `shardCount: auto` (the default): the operator sizes the
shard count from the tenant HelmRelease count — `K = clamp(ceil(H/100),
1, min(16, T))` with hysteresis (scale up past 120 HR/shard, down under
60) — so small clusters stay at one shard (≈ today's behaviour) and
large fleets shard out automatically. An integer value pins the count
explicitly.
One deviation from the design doc: the webhook/controller scope is the
`sharding.fluxcd.io/key` label rather than
`internal.cozystack.io/tenantmodule` — every tenant HelmRelease is born
with `key=tenants` (stamped by ApplicationDefinition `release.labels`
and chart templates), so the webhook rewrites it and the placement
controller continuously relabels stragglers; `tenantmodule` covers only
the 6 tenant chart modules.
### Screenshots
Not a UI change.
### Release note
```release-note
feat(fluxcd): tenant HelmReleases are now reconciled by operator-managed helm-controller shards; the flux-shard-operator balances tenants across `shardCount` instances and retires the legacy flux-tenants deployment
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced **flux-shard-operator** to shard helm-controller workloads
with deterministic tenant placement, optional rebalance controls, and
autosized shard count.
* Added a **CREATE-only mutating webhook** to stamp the correct shard
label on tenant HelmReleases.
* **Bug Fixes**
* Application updates now preserve existing **flux-shard** label
assignments to prevent unwanted shard re-stamping.
* **Chores**
* Added platform packaging for the new component, including an image
build step.
* Removed the legacy tenants deployment and added a migration to retire
it safely.
* **Tests**
* Expanded unit and chart tests for placement, provisioning, webhook
behavior, and configuration parsing.
* **Documentation**
* Added operator documentation describing configuration and metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The `Build cache (main)` workflow (`build-main.yaml`, added in #2948)
failed on
its first run:
https://github.com/cozystack/cozystack/actions/runs/27929277423
The `Set up Buildx` step ran without `DOCKER_CONFIG`, so
`docker/setup-buildx-action` created the docker-container builder under
the
runner's default `~/.docker`. The build step runs with
`DOCKER_CONFIG=$RUNNER_TEMP/.docker` (seeded earlier, holding the OCIR
auth), so
buildx couldn't find the builder and failed immediately:
```
ERROR: no builder "builder-6fb46242-…" found
```
The post-job `docker buildx rm` still succeeded — because the action's
cleanup
runs under the default config where the builder actually lived — which
confirms
the builder existed but in the wrong config store.
This sets `DOCKER_CONFIG` on the `Set up Buildx` step so the builder
instance is
created in the same config the build step reads.
Impact: only the main cache-warmer was broken. PR builds `--cache-from`
this
ref, so a missing cache is a cold build, not a failure — but the
warm-cache
speedup from #2948 wasn't being delivered until this fix.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved build pipeline stability by ensuring consistent Docker
configuration across workflow steps. This enhances the reliability of
the container build and deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
During node reprovisioning in tenant Kubernetes clusters, the VMI (and
its hot-plug pod) can outlive the parent VirtualMachine. The upstream
kubevirt-csi-driver treats a missing VM as "detach succeeded", so the
hot-plug pod stays alive and keeps the infra storage device exclusively
attached to the source host — every subsequent attach of that volume
fails, on DRBD with `failed to set source device readwrite`. This was
hit repeatedly in production tenant clusters (see
kubevirt/csi-driver#182).
This PR:
- updates the pinned `kubevirt.io/csi-driver` module to the latest
upstream commit;
- backports kubevirt/csi-driver#184 into the controller wrapper: after
the upstream unpublish returns success, the wrapper re-checks
`VMI.status.volumeStatus` and, if the hot-plug is still reported there
(e.g. because the parent VM is already gone), detaches it from the VMI
directly via the `removevolume` subresource and waits until it
disappears.
The backport is self-contained in the wrapper and can be dropped once
kubevirt/csi-driver#184 merges upstream and the module pin is bumped
past it.
Credit to @mattia-eleuteri for the root-cause analysis and the upstream
fix.
### Screenshots
Not a UI change.
### Release note
```release-note
fix(csi): tenant CSI driver no longer leaves orphan hot-plug volumes exclusively attached to the source host when the VMI outlives its deleted VirtualMachine during node reprovisioning
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced hotplug volume unpublishing with automatic verification to
ensure volumes are fully detached before completion, including retry
logic with up to 2-minute timeout.
* **Dependencies**
* Updated KubeVirt API and CSI driver dependencies to latest versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Myasnikov Daniil