Commit Graph
4078 Commits
Author SHA1 Message Date
Aleksei Sviridkin 09a658d43b test(dashboard): pin JWT verification in the token proxy
Only the cold-start path of verifyAndParseJWT was tested; every other
test served an empty JWKS, so no token was ever verified and a change
that weakened verification would have passed. Cover a valid token and
the ones that must be refused: expired, not yet valid, signed by
another key, unknown or missing kid, alg none, and HS256 keyed with
the RSA public key.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:02 +03:00
Aleksei Sviridkin e66f134564 fix(e2e-sandbox): install mc from its GitHub release
dl.min.io now answers 410 Gone for every community mc build, since
MinIO archived the project, so the sandbox image no longer builds.
The final mc release is still published as a GitHub release asset.
Pin it and verify the per-arch checksum, because the old install
trusted whatever the URL served.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:02 +03:00
Aleksei Sviridkin 3800f63513 fix(ingress): trust CF-Connecting-IP only from Cloudflare ranges
cloudflareProxy passed the Cloudflare ranges to the controller under
set_real_ip_from, which is an nginx directive and not an ingress-nginx
ConfigMap key, so the controller dropped it without a warning. With
use-forwarded-headers on, the realip module then trusted every source
from the default proxy-real-ip-cidr of 0.0.0.0/0, and any client that
reached the load balancer directly could set CF-Connecting-IP to an
address of its choice. That address is what whitelist-source-range
checks, so the tenant whitelist, and any per-Ingress one, could be
walked past with one header.

proxy-real-ip-cidr is the key the controller reads for the trusted
ranges.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:04:27 +03:00
Aleksei Sviridkin 76b3652297 fix(kubevirt)!: evict non-migratable VMs instead of blocking the drain (#4403)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

A node drain now completes when a VM on that node cannot live-migrate.
The VM is shut down and started again instead of holding the drain
forever. VMs that can migrate still migrate. The change is the
cluster-wide `evictionStrategy` in the KubeVirt CR, from `LiveMigrate`
to `LiveMigrateIfPossible`.

The drain is held by virt-api's eviction webhook, not by a
PodDisruptionBudget. With `LiveMigrate` and a VMI that is not
live-migratable, the webhook denies every eviction with HTTP 429
([pod-eviction-admitter.go#L147-L150](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-api/webhooks/validating-webhook/admitters/pod-eviction-admitter.go#L147-L150)).
`kubectl drain` retries a 429 until `--timeout`, which is infinite by
default. KubeVirt v1.9.0 does not create PDBs for VMIs at all; its
disruption-budget controller only deletes old ones
([disruptionbudget.go#L353-L406](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-controller/watch/drain/disruptionbudget/disruptionbudget.go#L353-L406)).
With `LiveMigrateIfPossible` the same webhook lets the eviction through
for such a VMI
([L152-L155](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-api/webhooks/validating-webhook/admitters/pod-eviction-admitter.go#L152-L155)).
The VM then comes back according to its `runStrategy`.

RWO disks are not the only case. A VM with a passthrough GPU is not
migratable either, unless the `VGPULiveMigration` gate is on and the GPU
is a mediated device
([vm.go#L1310-L1319](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-handler/vm.go#L1310-L1319)).
The gate is off in Cozystack, so any `vm-instance` with `gpus` set
blocked the drain of its node until now.

In KubeVirt a drain and a launcher update go through the same eviction.
After a KubeVirt upgrade the workload updater live-migrates outdated
VMIs that can migrate and evicts the rest, because
`workloadUpdateMethods` lists `Evict` as the fallback
([workload-updater.go#L421-L424](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-controller/watch/workload-updater/workload-updater.go#L421-L424)).
The eviction call
([L614-L620](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-controller/watch/workload-updater/workload-updater.go#L614-L620))
hits the same webhook as a drain. So a `vm-instance` VM that cannot
migrate either blocks both, which is what happens today, or is restarted
by both, which is what happens after this PR once the VM runs under the
new strategy.

This has a cost. A non-migratable VM now restarts on a drain. That
covers VMs on RWO storage and VMs with GPU or host device passthrough.
The webhook does not look at who asked for the eviction, so this holds
for any eviction through the API, not only a drain. A VM whose disk is
bound to the drained node, for example on a LINSTOR class without DRBD,
stays down until the node is back.

The upgrade itself restarts nothing, and existing VMs switch only after
their next restart. KubeVirt writes the cluster strategy into the VMI
spec when the VMI is created
([defaults.go#L208-L211](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/defaults/defaults.go#L208-L211)),
and the webhook reads that field first
([migrations.go#L116-L122](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/util/migrations/migrations.go#L116-L122)).
A VMI that was running before the upgrade keeps `LiveMigrate`, so a
drain still waits on it and the updater's eviction still gets 429. After
one restart the VMI carries `LiveMigrateIfPossible`. From then on every
KubeVirt bump shuts it down through the updater, in batches of 10 every
minute, the `batchEvictionSize` and `batchEvictionInterval` set in the
CR
([kubevirt-cr.yaml#L139-L140](https://github.com/cozystack/cozystack/blob/25d4a8c951b9fa7249b56ae79328d2cfd4f3a9c0/packages/system/kubevirt/templates/kubevirt-cr.yaml#L139-L140)).
Reading the source, a CPU socket increase on such a VM under
`vmRolloutStrategy: LiveUpdate` goes the same way, since the
`VCPUChange` condition sends it to the updater
([L320-L325](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-controller/watch/workload-updater/workload-updater.go#L320-L325)).
I have not run that one.

I kept `Evict` even so, because `kubernetes-nodes` workers depend on it.
They set `evictionStrategy: External`, and for that strategy the webhook
marks the VMI for evacuation instead of evicting it
([pod-eviction-admitter.go#L156-L157](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-api/webhooks/validating-webhook/admitters/pod-eviction-admitter.go#L156-L157)).
CAPK sees the mark on a VMI that cannot migrate, drains the tenant node
and deletes the VMI
([machine.go#L414-L454](https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt/blob/v0.1.10/pkg/kubevirt/machine.go#L414-L454)).
The worker VM has `runStrategy: Always`, so KubeVirt starts it again on
the new launcher. That is how a GPU worker or any other non-migratable
worker gets the new launcher. Without `Evict` those workers would stay
on the old launcher for good. Worker behaviour does not change with this
PR, and the new test pins the method list so the fallback is not dropped
later.

`vm-instance` renders no `evictionStrategy` and has no value for it, so
a tenant has no way to opt out of either restart. KubeVirt would honour
a per-VM value over the cluster one. I filed #4419 to expose it.

I picked this over the options in the issue because KubeVirt already
decides migratability per VMI at eviction time. The platform CR does not
need to know anything about storage, and a chart shipping a VM does not
have to remember an override. Deriving the strategy from the disk access
mode in a chart would also miss the GPU case.

To get the new drain behaviour for a non-migratable VM that is already
running, restart it once after the upgrade. Until then a drain of its
node still blocks, as it does today.

The comment in the CR is on the same line as the key on purpose. `make
update` inserts blocks right before the `evictionStrategy:` line, so a
comment above the key would get separated from it.

A VM with `runStrategy: RerunOnFailure`, `Manual` or `Once` may stay off
after an eviction. If the guest shuts down cleanly on the ACPI request
within its grace period, the VMI ends `Succeeded`. `RerunOnFailure`
restarts only a failed VMI, and `Manual` and `Once` restart nothing on
their own
([vm.go#L1039-L1149](https://github.com/kubevirt/kubevirt/blob/v1.9.0/pkg/virt-controller/watch/vm/vm.go#L1039-L1149)).
`Once` stays off even after an unclean stop. Only the default `Always`
brings the VM back in every case.

What I did not verify. I have not drained a node with such a VM on a
live cluster, and I have not watched the updater restart them during an
upgrade. Everything above comes from the KubeVirt v1.9.0 and CAPK
v0.1.10 source. I did not check how `talosctl upgrade` handles a drain
that keeps getting 429. I also did not check how the cluster autoscaler
treats a node with a non-migratable VM now that its launcher pod can be
evicted. The pod has no `safe-to-evict` annotation, so for a VM on
node-local storage that likely comes down to the autoscaler's
`skip-nodes-with-local-storage` setting. Finally, I did not check
whether the upstream `VMCannotBeEvicted` alert is deployed in Cozystack.
For non-migratable VMs it stops firing after this change, since they are
evictable now.

Closes #4360

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

No UI change.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(kubevirt)!: VMs that cannot live-migrate no longer block a node drain. The cluster-wide eviction strategy is now LiveMigrateIfPossible instead of LiveMigrate. VMs that can migrate still migrate. VMs that cannot, for example with a ReadWriteOnce disk or GPU or host device passthrough, are shut down and started again by their runStrategy. This applies to any pod eviction, not only a drain. A VM whose disk is bound to the drained node stays down until the node returns. A VM that is already running keeps the old strategy until its next restart, so restart non-migratable VMs once after upgrading if their nodes need to drain. The KubeVirt workload update uses the same eviction: once a non-migratable vm-instance VM runs under the new strategy, every later KubeVirt upgrade shuts it down, 10 VMs per minute. A VM with runStrategy RerunOnFailure, Manual or Once that shuts down cleanly on eviction stays off until it is started again. kubernetes-nodes workers are not affected.
```
2026-09-24 15:57:44 +04:00
Aleksei Sviridkin 08c535c295 fix(monitoring): give kube-apiserver its own scrape size limit (#4427)
## What this PR does

kube-apiserver's `/metrics` grows with the number of resources and CRDs,
and on a full install it outgrows the 32MB `promscrape.maxScrapeSize`
that monitoring-agents sets globally. vmagent then refuses every scrape
of the API server, `up` drops to 0, and `KubeAPIDown` and `TargetDown`
fire against a healthy control plane while every rule over API server
metrics goes blind.

The apiserver `VMServiceScrape` endpoint now sets its own
`max_scrape_size` (128MB by default, `vmagent.apiserverMaxScrapeSize`).
vmagent uses a per-job value in place of the global flag for that job
only, so the 32MB guard for every other target is unchanged. The value
stays finite on purpose: vmagent reads the whole uncompressed response
into memory before parsing, `streamParse` included, so this limit is
also a memory ceiling.

A new `VMAgentScrapeSizeExceeded` warning fires on
`vm_promscrape_max_scrape_size_exceeded_errors_total`, so an oversized
target is reported as a size problem instead of only as a target that is
down. The counter has no target label, so the description points at the
vmagent targets page to find it.

Upgrade note: the per-job value replaces the global flag in both
directions. An install that raised `promscrape.maxScrapeSize` above
128MB through `vmagent.extraArgs` should raise
`vmagent.apiserverMaxScrapeSize` as well.

Verified on a dev cluster with the same operator (v0.68.4) and vmagent
(v1.136.0): a test vmagent with a 1MB global limit scraped all three API
servers with `max_scrape_size: 64MB` on the endpoint and refused all
three without it. The new counter reached storage with the `namespace`
and `pod` labels the alert uses, and the alert expression returned a
series. The `VMRule` passes the operator's admission webhook, which
rejects a malformed expression. Guest clusters get the same default,
since their HelmRelease overrides other `vmagent` keys only.

Fixes #4343

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(monitoring): kube-apiserver is scraped with its own 128MB size limit (`vmagent.apiserverMaxScrapeSize` in the monitoring-agents component) instead of the global 32MB, so a large `/metrics` no longer makes KubeAPIDown fire on a healthy API server. A new VMAgentScrapeSizeExceeded alert reports any target whose response exceeds the scrape size limit.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Kubernetes API server metrics can now use a configurable scrape-size
limit, set to 128 MB by default. Leaving the setting empty uses the
global limit.
* **Bug Fixes**
* Added a warning alert when vmagent reports scrape-size-limit errors.
The alert identifies the affected vmagent pod and service and provides
guidance for finding the target and adjusting its limit.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-24 14:30:46 +04:00
Aleksei Sviridkin 15035a915b fix(vm-instance): disable helm readiness wait on the VMInstance release (#3735)
## What this PR does

A VMInstance is served as a Flux HelmRelease that renders a KubeVirt
VirtualMachine. Since v1.5.0 (helm-controller v1.4.3 to v1.5.0) that
release is health-assessed by the kstatus poller. vm-instance sets no
`healthCheckExprs`, so the VirtualMachine goes through kstatus's own
rules in `fluxcd/cli-utils` (`pkg/kstatus/status`): `checkGeneration`
returns InProgress for any object whose `status.observedGeneration`
differs from `metadata.generation`, and past that check a `Ready=False`
condition is InProgress too, so a halted VM is never Current even with
its generation in sync. KubeVirt keeps a VirtualMachine's
`observedGeneration` behind its `generation` whenever a `spec.template`
change is pending a restart, the VM is stopped, or a live-updatable
change never converges (kubevirt/kubevirt#18700). So a healthy, Running
VM is read as permanently InProgress, the upgrade waits its full
timeout, and fails with `timeout waiting for: [VirtualMachine/...
status: 'InProgress']`. On releases still carrying the pre-1.5.0
`remediation.retries: -1` + rollback spec, the timeout then reverts the
previous manifest, dropping just-applied changes such as opened ports.

This sets `release.cozystack.io/helm-install-disable-wait: "true"` on
the vm-instance ApplicationDefinition, so cozystack-api generates the
HelmRelease with Install/Upgrade DisableWait. The release settles once
the VirtualMachine is applied, and VM health is read off the
VirtualMachine itself (`status.printableStatus` / `Ready`) and off the
WorkloadMonitor the chart already renders, which is what
`WorkloadsReady` on the Application reports. This restores the pre-1.5.0
behaviour, under which the VM was never a helm readiness gate. The same
mechanism is already used by `kubernetes-rd` and `kubernetes-nodes-rd`.

On its own that annotation does not reach a running cluster, so two gaps
in cozystack-controller are closed with it.

The first is the rollout. `applicationdefinition-controller` hashed each
definition's name and spec only, so a change confined to a
`release.cozystack.io/*` annotation left the hash unmoved and
cozystack-api kept serving the configuration it read at its last
start-up, which is the only place those annotations are ever read. The
hash now covers them too. Only that prefix, because kubectl and Helm
rewrite annotations of their own on every apply and hashing those would
roll the api Deployment on every reconcile.

The second is the releases that already exist.
`applicationdefinition-helm-reconciler` brings chartRef, valuesFrom and
labels on existing HelmReleases in line with their definition; it now
does the same for the readiness wait. Without it the annotation reaches
new instances only, which on an affected cluster is the wrong half:
#3734 reports 134 of 136 releases already looping upgrade and timeout,
and re-applying every VMInstance by hand was the only way out. The
setting is driven in both directions, since cozystack-api rebuilds the
whole spec on every write and emits `DisableWait=false` once the
annotation is gone. An absent install or upgrade block already behaves
as `DisableWait=false` and is materialised only when the definition asks
to disable the wait, so a release with nothing to change is not
rewritten.

Two alternatives were considered. A per-kind `healthCheckExprs` keyed on
`status.printableStatus` cannot work: setting it moves the
VirtualMachine from kstatus to the CEL status evaluator
(`fluxcd/pkg/runtime/cel`), which runs the same `observedGeneration`
check ahead of the expressions, and any `healthCheckExprs` added to this
kind later is a no-op while the wait is disabled. The typed
`spec.release.waitStrategy: legacy` is the narrower option, since the
legacy checker never waits on a custom resource, but it still gates the
release on the chart's Service, and with `external: true` that is a
LoadBalancer the legacy checker holds not ready until it has ingress. So
an external VM would still wait on IP allocation, which `WorkloadsReady`
already watches. Disable-wait avoids that and is the mechanism this
repository already uses for the same class of problem.

Tests: a helm-unittest suite (`tests/readiness_gate_test.yaml`) guards
that the annotation stays on the rendered ApplicationDefinition, and Go
unit tests cover both cozystack-controller paths: the hash moves on a
release annotation and stands still on an unrelated one, and the wait
lands on an existing release, is withdrawn when the annotation goes,
does not materialise empty install/upgrade blocks, does not rewrite a
release already in sync, leaves a release alone when the annotation does
not parse, and does not reach a release of another kind.

Note on existing clusters: no manual step. The definition changes on
upgrade, the config-hash rolls cozystack-api, and the helm reconciler
brings existing VMInstance releases in line. Releases that predate
v1.5.0 also carry a `remediation.retries: -1` + rollback spec, which is
left alone: with the wait disabled the upgrade no longer times out, so
that remediation no longer fires, and any write through the apps API
drops it.

Closes #3734.

### Downstream repositories

- [x] No downstream repository is affected by this change

<!-- Walked the trigger map against the diff. The change is a readiness
annotation on an ApplicationDefinition plus two cozystack-controller
paths. It does not touch openAPISchema, kind, plural, release.prefix,
any app package, any values.schema.json or any installer/platform values
key. The website's application-definitions.md describes the config-hash
as a checksum "over the sorted definitions", which stays accurate now
that the hash also covers the release.cozystack.io/* annotations, and no
page documents those annotations today. -->

### Release note

```release-note
fix(vm-instance): VMInstance HelmReleases no longer hang InProgress or roll back healthy VMs; the release no longer waits on VirtualMachine readiness, which helm-controller's kstatus poller cannot assess correctly for KubeVirt VMs. Existing VMInstance releases are brought in line automatically on upgrade.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* VM installations no longer wait indefinitely when Kubernetes reports
the VM’s observed generation behind its current generation.
* Changes to release settings are now detected and applied to generated
Helm releases, including install and upgrade wait behavior.
* **Tests**
  * Added coverage for VM readiness and release-setting reconciliation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-24 14:30:23 +04:00
myasnikovdaniil 9bec21461d feat(kubernetes): boot tenant worker disks from a shared golden Talos image (CDI clone) (#4171)
## What this PR does

Tenant worker boots from talos OS disk image that CDI streams over HTTP
from image factory, and it does that once per worker. Same bytes N
times, and N chances for a fetch to fail. In closed or rate-limited
environment that is wrong shape.

This PR adds opt-in catalog and per-pool selector that reads from it.

`packages/system/kubernetes-worker-image` imports one golden DataVolume
into `cozy-public` per `(schematicID, version)`, named
`talos-worker-<schematicID>-<version>`. Pool opts in with
`osImage.builtin` on its own `KubernetesNodes`, and its `disk-system`
DataVolume then carries `source.pvc` pointing at that golden, so CDI
does storage layer CSI clone instead of HTTP import. One import per
flavor, shared by every worker of every tenant.

Catalog is disabled by default, enable by adding
`cozystack.kubernetes-worker-image` to `bundles.enabledPackages`. Its
knobs (`imageFactoryURL`, `storageClass`, `storage`, `images`) are set
through `spec.components.platform.values.kubernetesWorkerImage` on
cozystack-platform Package. That needed optional package helper to learn
components argument, which `cozystack.platform.package.optional.default`
did not have, so every optional package registered through that wrapper
could name variant and nothing else. Thirteen other packages get same
path for free.

`osImage` has two mutually exclusive forms:

- `osImage.builtin` boots pool workers by CDI-cloning golden from
catalog. Clone is storage layer CSI copy, so OS image never crosses pod
network and factory answers once for whole fleet instead of once per
worker.
- `osImage.factory` keeps HTTP import and lets pool pin own
`imageFactoryURL`, `schematicID` and `version`. This is bring-your-own
path for operator running own factory or caching mirror, and it is what
pool gets when `osImage` is omitted.

In-guest talos installer written by reconcile job follows same
resolution, so pool that pins release boots it and upgrades into it
instead of being flipped back to cluster default halfway.

This is pool half of what #3950 asks for. That issue wants platform
level defaults for two channels, worker OS image and in-guest registry
mirrors. Here image channel gets per-pool source and shared cache,
platform default plumbing is left to the issue.

## Existing workers do not roll

With `osImage` omitted rendered `KubevirtMachineTemplate` is byte
identical to what chart produced before. KMT is named by hash of its own
content, so any drift would re-image every worker VM in fleet on next
upgrade. Stored render snapshot is untouched by this PR, that is the
evidence and not the claim. Editing `osImage` on existing pool re-images
that pool, like changing any other image reference.

## What fails at render

Clone that would silently degrade into something slower or wrong is
refused instead of rendered.

Golden absent, or its import in terminal `Failed` phase. Pool
`storageClass` different from golden one, because CDI cant CSI-clone
across storage classes and falls back to host assisted copy over pod
network, which is exactly the transfer `osImage.builtin` removes. Empty
class on either side resolves against cluster default class before
comparing, so pool at `storageClass: ""` is checked too. `diskSize`
below golden storage, because clone target cannot be smaller than
source. Talos release the matrix lists as unsupported, whether pool
reached it through `talos.version` or through `osImage.*` override.
Matrix passes talos minor it does not list, so what it promises is that
known-bad pairing is refused, not that every pairing is known.
`schematicID` that is not plain lowercase alphanumeric, or version that
is not `vMAJOR.MINOR.PATCH`, both are interpolated into DataVolume name
and image URL while `values.schema.json` types them as bare strings.

Golden that is only mid-import is not an error. CDI holds worker disk
until source populates, and failing there would abort whole pool
HelmRelease for length of a download. Same reasoning covers pool that
already cloned: absent or `Failed` golden fails render only for pool
still waiting for its first disk, otherwise reconcile of healthy pool
would abort over source it no longer reads.

Golden that declares no StorageClass is read from its PVC, not from
whatever default is in force today, because those differ once cluster
default moves. Cluster with two default classes resolves same way
Kubernetes does, newest wins. Pool that already cloned is identified by
annotation naming exactly that pool, not by disk name prefix, because
pool `md0-gpu` produces disks carrying `md0` prefix, and it reads its
own class from its own bound PVC, so admin moving
`storageclass.kubernetes.io/is-default-class` does not fail release of
pool whose disks are all on right class.

That resolved class is then written onto worker disk. Reading one class
and binding another is how pool at empty `storageClass` passed the
comparison and still took host assisted copy on its next worker, so
guard and disk now come from one value. HTTP path leaves empty class to
cluster as before, which is what keeps its render byte identical.

Catalog side refuses duplicate `(schematicID, version)` entries, entries
whose schematic or version cannot produce name pool can reconstruct, and
edit to `storage`, `storageClass` or source URL of entry whose golden
already exists. Last one is not patchable in place, so without guard it
lands as rejected patch on next upgrade.

Goldens carry `helm.sh/resource-policy: keep`, so dropping entry stops
managing golden instead of deleting it. Missing golden aborts whole
Kubernetes HelmRelease of every tenant pinned to it, not just the disk,
and each cloned worker disk annotates golden it came from, so "does
anything still clone this" is answerable before manual delete.

Three drift guards live in
`hack/check-worker-image-catalog-defaults.bats`, because values that
have to agree sit in files nobody edits together. One-sided talos bump
trips all three.

## Upgrade

Three pool values that never produced working image rendered before and
hard-fail now, so pool carrying any of them flips its HelmRelease to
`InstallFailed` on upgrade and not on next edit: `talos.version` in
`vMAJOR.MINOR` form, `schematicID` that is not plain lowercase
alphanumeric, and empty `talos.imageFactoryURL`, which used to render
schemeless `/image/...` source URL. None of the three could boot worker,
so this is failing earlier and not a regression, but it surfaces on pool
nobody touched and fix in each case is to correct the value. CR carrying
such value is admitted by API with value intact, and later unrelated
edit reaches `.spec.values` of HelmRelease while it stays
`InstallFailed`, so edit is taken everywhere and applied nowhere.

## Why e2e does not exercise the clone path

Clone is storage layer copy, so it needs StorageClass whose volumes are
reachable from any node. Merge-gating e2e has none: those lanes run on
talos containers, which share runner kernel and cannot load DRBD, so
node-local `local` is only class there. Nothing would arrange the
co-location such a class requires either, golden in `cozy-public` has no
consumer and binds where CDI importer landed, while worker disk binds
where its VM is scheduled.

So the suites keep importing over HTTP with `osImage` omitted, which is
also the render the no-roll guarantee rests on. This PR touches no e2e
harness file: `hack/` differs from main only by the two guards this
feature owns, one tying catalog default golden to pool chart talos and
StorageClass defaults, one on in-guest installer heredoc.

Switching suites over does not work on that substrate. Golden asks for
`replicated`, lane does not have it, CDI resolves no StorageProfile, and
catalog HelmRelease times out on DataVolume stuck in `ImportInProgress`.
Passing lane own class would fix the class and leave the co-location.

Nightly and tag build run same suites on QEMU with DRBD and
`replicated`, which is the substrate clone path was verified on. That is
where e2e coverage belongs, as its own change with that lane storage
headroom measured rather than assumed.

Worth being straight about what this leaves, because it bears on whether
the feature earns its place. The e2e argument was never the strong one:
golden fetches from image factory exactly once, and #3513 established
that node-join stall tracks host `kvm_amd` `vgif` flag rather than
registry egress. What clone genuinely removes is the fanout, N workers
streaming over pod network becomes N storage layer copies. That is
property of storage layer, so it is worth having where DRBD is and worth
nothing where it is not. This ships as production capability with the
constraint stated on the field, not as CI improvement.

## Testing

On this tree:

- full `make unit-tests` plus `make test-controllers`: 95 charts, 2520
helm-unittest cases, 1710 bats cases, `go test ./pkg/...` and
`./internal/...`, exit 0
- `pre-commit run --all-files`: exit 0, root `make generate` and per-app
generate both clean, so no generated-file drift
- render snapshot unchanged, which is what pins byte identical render
for pool that does not adopt `osImage`
- mutation pass over render guards, each term deleted or inverted in
turn and suite required to notice it by name. Restoring presence gate on
catalog StorageClass comparison reddens exactly one case, inverting its
`ne` to `eq` reddens five. Reverting worker disk to omit its resolved
class reddens three cases that pin where builtin disk binds, and forcing
that class non-empty on every path reddens HTTP-path case and all three
render snapshots, with snapshot diff naming the changed KMT hash

`E2E Tests` passed on run 35831830568. That run was on the previous
head, 4fd8e7ba, and this head differs from it only by one test comment
and the commit message.

## Verified on a live cluster

Throwaway three-node talos stand (OCI, talos v1.13.6, LINSTOR/ZFS,
`replicated` StorageClass) installed from this branch `packages/` tree.
This covers what neither `helm template` nor unit suites can answer. It
predates the change that writes resolved StorageClass onto worker disk,
and that stand ran with explicit `replicated` on both sides, which is
the value that change now renders.

- The catalog wires up from `bundles.enabledPackages` alone:
`PackageSource cozystack.kubernetes-worker-image` reconciled, its
HelmRelease installed, and the golden `talos-worker-<schematic>-v1.13.6`
imported to `Succeeded` in 108s (6Gi, RWX, `replicated`).
- A pool on `osImage.builtin` clones it on the storage layer. Both
worker DataVolumes went `CSICloneInProgress` then `Succeeded` in 43s
with `Running=False reason=Completed message="Clone Complete"`, carrying
`source.pvc` and no `source.http`, and no importer or upload Pod
appeared in the tenant namespace.
- The clone yields a bootable talos root disk. Both worker VMs reached
`Running`/`Ready`, joined the tenant cluster and went `Ready` about 66s
after creation, and the tenant cilium, coredns, csi, ingress-nginx and
metrics-server addons all installed on them.
- Access modes match across the hop: golden `ReadWriteMany` 6Gi to
worker `ReadWriteMany` 20Gi on the same `replicated` class, grown by
CDI.
- The written annotations are correct in practice: each worker disk
carries `kubernetes-worker-image.cozystack.io/golden` naming the golden
and `.../pool` naming its pool.
- helm-controller converges on a builtin pool: forcing a reconcile left
the content-hashed KMT name unchanged and the VMI and DataVolume UIDs
identical, so a reconcile does not roll the pool.
- Cross-namespace clone RBAC is in place as shipped: `create datavolumes
--subresource=source -n cozy-public` resolves to `yes` for a tenant
ServiceAccount through the `cdi-clone-dv` RoleBinding, and CDI runs with
`cloneStrategyOverride: csi-clone`. This PR adds no RBAC of its own.

### Screenshots

Not applicable, no UI change.

### Downstream repositories

Trigger map points at two repositories.

Provider models every pool field by hand, so `osImage` needs schema,
model and expand/flatten pair there. Already tracked as
cozystack/terraform-provider-cozystack#39, filed when the field was
still called `image`.

`content/en/docs/next/operations/configuration/platform-package.md` is a
hand-written table of `spec.components.platform.values.*` keys and
nothing generates it, so `kubernetesWorkerImage` needs a row there.
Filed as cozystack/website#707.

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/issues/707
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
https://github.com/cozystack/terraform-provider-cozystack/issues/39
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(kubernetes): tenant worker pools can boot their system disk from a shared golden talos image instead of importing it from the Image Factory once per worker. Enable the opt-in `cozystack.kubernetes-worker-image` package via `bundles.enabledPackages` to publish one golden DataVolume per `(schematicID, version)` in `cozy-public`, then set `osImage.builtin` on a `KubernetesNodes` pool to have CDI clone it on the storage layer. A pool that omits `osImage` keeps importing over HTTP and renders exactly as before. The clone path needs the pool and the golden on the same StorageClass, and one whose volumes are reachable from any node, in practice a replicated/DRBD class.
```
2026-09-24 12:10:07 +05:00
Timofei Larkin 146e06e75e feat(api): take Application name limits from the application's own schema (#4367)
## What this PR does

The aggregated API server capped two kinds' names by knowing what their
charts render. A `Kubernetes` cluster name was held to 32 characters
because every worker pool is a `KubernetesNodes` release named
`kubernetes-nodes-<name>-<pool>`, and the default `md0` pool has to fit
the 53-character Helm release name limit. A `Kafka` name was held to 44
because the KRaft controller pod `kafka-<name>-c-<hash>-<id>` has to fit
the 63-character DNS-1123 label limit. The arithmetic is right, but it
is the charts' knowledge living in the platform: the next application
with a sub-resource budget needs another `if r.kindName == ...` branch,
and an application served from outside this repository — the whole point
of `ApplicationDefinition` — cannot get one at all.

An application now declares the limits on its own name in its own
schema, and the server applies whatever it finds there for whatever kind
the definition describes. The chart's `values.yaml` carries a `## @name`
annotation, `cozyvalues-gen` v1.7.0 publishes it at the root of
`values.schema.json` as `x-cozystack-name`, and `hack/update-crd.sh`
carries it into the `ApplicationDefinition`:

```yaml
## @name {string} - Cluster name. Every worker pool is a separate KubernetesNodes release named `kubernetes-nodes-<name>-<pool>`, so the name must leave room for the default `md0` pool inside the 53-character Helm release name limit.
## @maxLength 32
```

```json
"x-cozystack-name": {
  "type": "string",
  "description": "Cluster name. Every worker pool is a separate KubernetesNodes release named `kubernetes-nodes-<name>-<pool>`, so the name must leave room for the default `md0` pool inside the 53-character Helm release name limit.",
  "maxLength": 32
}
```

`maxLength`, `minLength` and `pattern` are honoured; `pattern` is
unanchored, as JSON Schema specifies. A declaration only tightens:
`prefix + name` must still fit the 53-character Helm limit. A rejected
name is answered with the chart's own description, or with "limit
declared by the <Kind> application" when the chart gives none.

The schema is untrusted input — definitions from other repositories are
written by hand — so a declaration that cannot be enforced as written
(not an object, a type error, an unknown keyword such as a misspelled
`maxLenght`, `maxLength` below 1, a negative `minLength`, `minLength`
above `maxLength` or above the Helm budget, a pattern that is not RE2)
refuses every create of that kind with `configuration error: the <Kind>
ApplicationDefinition declares an invalid x-cozystack-name: …`, and logs
the same at start-up. Dropping the declaration would admit names the
chart cannot render; applying it would reject every name and blame the
user. Other kinds are unaffected, and existing objects of the affected
kind can still be read, updated and deleted.

An upgrade brings `cozystack-api` up before `kubernetes-rd` and
`kafka-rd` re-render (`kubernetes-rd` depends on `cozystack-engine`), so
the new server can briefly read a definition that predates its
declaration. For exactly that case — a definition that declares nothing
— the two previous caps stay in `legacyNameCaps` as a floor, to be
removed once no supported upgrade starts from such a definition. That is
the only per-kind knowledge left in the name checks.

Also in this PR:

- The `pre-commit` job pins `cozyvalues-gen` v1.7.0, the first release
that understands `@name`
([cozystack/cozyvalues-gen#33](https://github.com/cozystack/cozyvalues-gen/pull/33)).
- The API review gate now diffs the root `x-cozystack-name` of every
definition as `metadata.name`, so a chart that lowers its cap or adds a
pattern is reported like any other tightened field. On this PR it
reports the Kubernetes and Kafka caps as added; their effective limits
are unchanged.
- The migration 56 comment that pointed at the removed
`maxKubernetesClusterName` now points at the chart's declaration.

Not in scope, because none of it is a static limit a chart can declare:
the `Tenant` name pattern in `pkg/apis/apps/validation`, the computed
tenant-namespace length check (it depends on the parent namespace at
request time), the tenant quota hierarchy, and the Phase 2 removed-field
warning for `Kubernetes`.

### How it is tested

`TestNameCapComesFromTheShippedSchema` reads the shipped `kubernetes`
and `kafka` ApplicationDefinitions off disk, builds storage through
`NewREST` as the server does, and checks the cap and the chart's
explanation; it then rewrites `maxLength` inside the shipped schema and
checks the limit moves with it, which a cap held in code would not.
`TestInvalidNameSchemaFailsClosed` covers each malformed declaration
through the same path, `TestLegacyNameCapsCoverUndeclaredDefinitions`
pins the upgrade floor and that a declaration replaces it, and
`TestClassifyNameDeclaration` covers the gate.

### Downstream repositories

Two repositories are affected and neither follow-up is filed yet, so no
box is ticked.

- `cozystack/website` — `content/en/docs/next/development.md` documents
the `values.yaml` annotations and needs `@name`;
`content/en/docs/next/cozystack-api/application-definitions.md`
describes `openAPISchema` and needs `x-cozystack-name`.
- `cozystack/external-apps-example` — it pins no `cozyvalues-gen`
version and regenerates against whatever is current; nothing in it
declares a name constraint, so it is unaffected until it chooses to.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(api): Application name limits now come from the application's own schema instead of being hardcoded per kind in the aggregated API server. A chart declares them with a `## @name` annotation in `values.yaml`, which cozyvalues-gen v1.7.0 publishes as `x-cozystack-name`; the API server enforces `maxLength`, `minLength` and `pattern` on `metadata.name` for any kind, answers a rejected name with the chart's own explanation, and refuses creates of a kind whose declaration it cannot enforce with a configuration error. The Kubernetes (32) and Kafka (44) name caps are unchanged in effect and are now declared by their charts.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Applications can declare minimum and maximum name lengths and
formatting patterns.
* Kubernetes cluster names are limited to 32 characters, and Kafka
cluster names to 44 characters, to meet generated resource naming
limits.
* Names that violate declared constraints are rejected with an
explanatory message. Tighter name constraints are also reported as
breaking changes.
* Declared maximum lengths cannot exceed the platform’s existing Helm
release-name budget.

* **Documentation**
* Added guidance on declaring and enforcing chart-specific resource-name
limits.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-24 10:29:27 +04:00
Andrey Kolkov ff52aeb44e chore(etcd-operator): bump to v0.5.6 (#4425)
## What this PR does

Bumps the vendored `etcd-operator` from **v0.5.5** to
**[v0.5.6](https://github.com/cozystack/etcd-operator/releases/tag/v0.5.6)**.

v0.5.6 is a controller-only bug-fix release; the two changes are:

- **Terminal-phase `EtcdMember` Pods are replaced**
([etcd-operator#370](https://github.com/cozystack/etcd-operator/pull/370)).
A graceful node shutdown SIGTERMs etcd, which exits 0, and the kubelet
parks the Pod in `Succeeded` without deleting it. The operator manages
bare Pods and only recreated on `NotFound`, so after a whole-cluster
reboot every member stayed down until the Pods were deleted by hand. The
reconciler now deletes an owned Pod in a terminal phase and lets the
next pass recreate it against the same PVC (or, for memory-backed
members, through the existing pod-loss re-add path).
- **`EtcdCluster` status stays truthful while etcd is unreachable**
([etcd-operator#369](https://github.com/cozystack/etcd-operator/pull/369)).
On a converged cluster the learner-promote and auth reconcile paths
returned their transient requeue before `updateStatus`, so
`Available`/`Degraded` and `readyMembers` froze at the last healthy
value and a fully down cluster kept advertising `Available=True`. The
status write now always happens, with the requeue carried through it.

Package changes are pin-only:

- **`etcd-operator`**: `appVersion` → v0.5.6 (image tag derives from
it), `ETCD_OPERATOR_REF` → v0.5.6, `tests/deployment_test.yaml` image
pins → v0.5.6.
- **`etcd-operator-crds`**: `ETCD_OPERATOR_REF` → v0.5.6. `make update`
at the new ref produces an empty diff: the five CRDs are byte-identical
to v0.5.5, and `charts/etcd-operator/files/manager-role-rules.yaml` did
not change, so `templates/rbac.yaml` is untouched.

Verified locally: `helm unittest` 18/18 (`etcd-operator`) and 1/1
(`etcd-operator-crds`), `helm lint` clean on both, `helm template`
renders `ghcr.io/cozystack/etcd-operator:v0.5.6`, and that tag resolves
in ghcr.io.

### Screenshots

n/a — no UI changes.

### Downstream repositories

Walked the trigger map against the diff. The change is confined to
`packages/system/etcd-operator{,-crds}` and touches only version pins:
etcd-operator is a system component (not a
`packages/apps`/`packages/extra` package the website and TF provider
track), no CRD, schema, values, `hack/`, installer or platform values
changed.

- [x] No downstream repository is affected by this change

### Release note

```release-note
chore(etcd-operator): bump to v0.5.6, which replaces EtcdMember Pods left in a terminal phase after a node shutdown and keeps EtcdCluster Available/Degraded conditions truthful while etcd is unreachable
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Updates**
* Updated the etcd operator to version 0.5.6. This version is used
consistently across the operator deployment and its associated custom
resource definitions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 22:14:16 +04:00
IvanHunters 380c446cda fix(monitoring): pin the scrape size rendering and sharpen the alert
The per-job limit replaces the global flag downward as well, so an
install that raised promscrape.maxScrapeSize past it through extraArgs
has to raise this value too; say so where the value is set. Pin that a
numeric value still renders as a string, which the CRD field requires,
and pin the alert severity. Name the cluster in the alert and say where
the apiserver value lives, since a guest cluster cannot set it.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-23 20:56:07 +03:00
IvanHunters a17c5b2667 feat(monitoring): alert when vmagent refuses an oversized scrape
A target whose response exceeds the scrape size limit reports up == 0,
which is indistinguishable from the target being down: TargetDown and
KubeAPIDown fire and nothing names the real cause. Alert on
vm_promscrape_max_scrape_size_exceeded_errors_total so the first alert
says what is wrong. The counter has no target label, so the description
points at the vmagent targets page to find the target.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-23 20:28:29 +03:00
IvanHunters 4a268f6c01 fix(monitoring): give the kube-apiserver scrape its own size limit
kube-apiserver's /metrics grows with the number of resources and CRDs,
and on a full install it outgrows the global 32MB
-promscrape.maxScrapeSize. vmagent then refuses every scrape of it, so
up is 0 and KubeAPIDown and TargetDown fire against a healthy API
server while every rule over API server metrics goes blind.

Set max_scrape_size on the apiserver endpoint instead of raising the
global flag. vmagent uses it in place of the flag for that job only, so
the 32MB guard for every other target stays as it is. vmagent reads the
whole uncompressed body into memory even with streamParse, so the
per-job value is a memory ceiling and stays finite (128MB), exposed as
vmagent.apiserverMaxScrapeSize.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-23 20:27:00 +03:00
Andrey Kolkov e60fbc3f37 chore(etcd-operator): bump to v0.5.6
Controller-only release: EtcdMember Pods that reach a terminal phase
are replaced instead of being left behind, and EtcdCluster status is no
longer rewritten while etcd itself is unreachable. No CRD, RBAC or
values changes; re-vendoring the CRDs produces an empty diff.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-23 21:22:43 +04:00
IvanHunters 70ea242e1b docs(vm-instance-rd): trim the disable-wait comment and correct the mechanism
The annotation comment repeated the PR description and the test header;
two lines on why are enough. The hang comes from the kstatus generation
check in cli-utils, not the CEL status evaluator, since vm-instance sets
no healthCheckExprs. The config-hash rollout runs in cozystack-controller,
not cozystack-operator.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-23 20:18:06 +03:00
Aleksei Sviridkin 45f5698bce fix(cdi): validate kubevirt-cdi values against a schema (#4417)
## What this PR does

This gives the kubevirt-cdi chart a values schema, so a bad clone
strategy fails when the chart renders instead of when the CDI object is
applied. Today a typo like `csi_clone` renders fine, the CDI CRD refuses
it on apply, and the operator finds out from a failed HelmRelease.

The schema is generated by `cozyvalues-gen` from annotations in
`values.yaml`, through a new `make generate` target, the same way
`talos-log-collector` does it. There is no README here, so the target
only writes the schema.

The clone strategy is checked with a pattern, not with the CRD's enum.
The chart uses the empty string to omit the field and let CDI pick a
strategy per StorageProfile, and the CRD enum has no member for that.
`cozyvalues-gen` also can't emit an empty enum value. So the accepted
values are `copy`, `snapshot`, `csi-clone` and `""`. A null still works
as before. The key has a chart default, so Helm drops a null for it
before validation, and the template omits the field. `uploadProxyURL`
only has to be a string.

The new test suite refuses `csi_clone` and values of the wrong type, and
accepts each of the allowed values and null.

Closes #4141

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(cdi): kubevirt-cdi now validates its values against a schema, so an invalid `cloneStrategyOverride` such as `csi_clone` fails at render time instead of when the CDI object is applied.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Configuration**
* Chart values are now validated to ensure the upload proxy URL is a
string, importer resources are provided as an object, and clone strategy
uses a supported value: `copy`, `snapshot`, `csi-clone`, or an empty
value.
  * Existing defaults for these settings remain unchanged.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 20:07:06 +04:00
Aleksei Sviridkin 5e460605c2 fix(cdi): drop the ExpandDisks gate from the CDI object (#4415)
## What this PR does

This drops a feature gate from the CDI object that CDI never reads.
`ExpandDisks` sits in `spec.config.featureGates` in
`packages/system/kubevirt-cdi/templates/cdi-cr.yaml`, but it is a
KubeVirt gate.

CDI v1.66.1 has no gate with that name. It looks a gate up by exact
name, so an extra entry is read and ignored. The CRD types the list as
plain strings, so nothing rejects it either. Disk expansion comes from
KubeVirt, where `ExpandDisks` is GA at v1.9.0, the release the platform
ships. I left the KubeVirt CR as is; #4029 tracks the GA gates in that
list.

The gate list now has its own test case, pinned whole to what upstream's
`cdi-cr.yaml` ships. Adding `ExpandDisks` back turns it red. `make
update` only checks the upstream file against its pinned hash and writes
nothing under `templates/`, so it can't bring the line back.

Closes #4112

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(cdi): stop listing the KubeVirt `ExpandDisks` feature gate on the CDI object, where CDI ignored it. Disk expansion is unaffected.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Configuration**
* CDI now enables only the `HonorWaitForFirstConsumer` feature gate. The
`ExpandDisks` gate is no longer included in the configuration.
* **Tests**
* Updated configuration checks to verify the exact feature-gate list and
ensure it matches the gate CDI supports.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 20:06:31 +04:00
Aleksei Sviridkin 25d4a8c951 fix(kubevirt)!: evict non-migratable VMs instead of blocking the drain
The cluster-wide evictionStrategy was LiveMigrate. For a VMI that is
not live-migratable (a ReadWriteOnce disk, a passthrough GPU, a host
device), virt-api's eviction webhook answers every eviction with 429,
so a node drain retries forever and never completes.

LiveMigrateIfPossible keeps live migration for every VMI that can
migrate and lets the eviction through for the rest. The VM is shut
down and started again according to its runStrategy. A VM whose disk
is bound to the drained node stays down until the node returns.
KubeVirt writes the cluster strategy into a VMI's spec when the VMI
is created, so a VMI started before the upgrade keeps LiveMigrate and
still blocks the drain until it is restarted once.

A drain and a launcher update after a KubeVirt upgrade go through the
same eviction, so the workload updater's Evict fallback shuts down
non-migratable VMs started under the new strategy on every later
KubeVirt upgrade, where it used to get the same 429. Evict stays
anyway: for kubernetes-nodes workers, which use evictionStrategy
External, that eviction marks the VMI for evacuation, CAPK drains the
tenant node and deletes the VMI, and the VM starts it again on the new
launcher. The test pins the method list so the fallback is not dropped
to avoid the shutdowns.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 17:36:56 +03:00
Aleksei Sviridkin 9c97a7a7ac fix(cdi): validate kubevirt-cdi values against a schema
Without a schema, a cloneStrategyOverride outside the CDI enum, such as
csi_clone, renders a CDI object the API server refuses on apply, so the
mistake shows up as a failed HelmRelease rather than at the value.

The field is constrained by a pattern, not by the CRD's enum. The empty
string is the chart's way to omit the field and let CDI pick a strategy
per StorageProfile, the CRD enum has no member for it, and
cozyvalues-gen cannot emit an empty enum value. The key has a chart
default, so Helm drops a null for it before validating, and a null
keeps rendering as an omitted field.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 17:06:56 +03:00
Aleksei Sviridkin 0c358addfb fix(cdi): drop the ExpandDisks gate from the CDI object
ExpandDisks is a KubeVirt feature gate. CDI declares no gate by that
name and matches gate names exactly, so the entry was accepted and
ignored. Disk expansion is a KubeVirt feature, GA there at the release
the platform ships, so listing the gate on the CDI object points a
reader tracing where expansion comes from at the wrong component.

The feature-gate list is pinned whole in its own case, so a gate added
beside the one upstream ships turns the suite red.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 16:58:56 +03:00
Aleksei Sviridkin 2b5770cb4a fix(vm-instance)!: refuse resources that cannot size the VM (#4405)
## What this PR does

A vm-instance could ask for half a core and get one, with no error
anywhere. `resources.cpu` and `resources.sockets` are typed as
quantities, but they end up in `domain.cpu.cores` and
`domain.cpu.sockets`, which KubeVirt declares as unsigned integers. The
chart cast them with `int64`. That turns `500m` or `1.5` into `0`
without an error, and KubeVirt boots a zero core count as one core.

The chart now reads these two fields the way an integer field reads
them. It decodes the raw value as YAML and stops the render unless the
result is a positive whole number that fits in uint32. So `2`, `"2.0"`,
`"+2"` and `"1e1"` render as the integer they decode to. `500m`,
`2000m`, `1k`, `"1.5"`, `0`, `"0"`, `"-2"`, `"010"` and `"4294967296"`
stop the render with an error that names the field. A leading zero is
refused because YAML reads `010` as octal 8. The check looks at
presence, not truthiness, so an unquoted `0` is refused instead of being
read as unset. It is the same rule #4386 uses for kubernetes-nodes.

Memory gets a presence check too, which covers the case from the issue
thread. A quoted `"0"` was truthy and rendered `requests.memory: "0"`,
while an unquoted `0` was falsy and got dropped. Both spellings of zero,
and any negative amount, now stop the render. Any other value passes
through unchanged, because KubeVirt takes memory as a quantity. That
includes an amount below one byte, such as `500m` or `1e-3`, which
renders as it does on `main` today; this change does not refuse it.

The check lives in `virtual-machine.domainResources`. The VM spec, the
instance type decision and the update hook's patch all take their values
from that helper. The tests that pinned `cores: 0` as the old behaviour
now expect the refusal, and the helper comments about the truncation are
trimmed.

This is a breaking change. A release that carries one of these values
fails its next upgrade until the value is fixed: a cpu or sockets that
is not a positive whole number within uint32 or has a leading zero, and
a zero or negative memory. This includes a cpu or sockets set on its own
next to an `instanceType`, which used to be ignored whatever it held.

Some values change meaning instead of failing. A value like `"1e1"` or
`"2."` was truncated to 0 by the `int64` cast, so the VM booted with one
core. Now it gets the count the value spells, so that VM changes size on
upgrade.

I refuse these values instead of rounding them, because silent coercion
is how `500m` became one core in the first place. The issue asks for a
deprecation path, and I did not add a migration that rewrites stored
values. It would have to guess what `500m` meant, and both options (one
core, or dropping the field) hide the same mistake again. The
HelmRelease upgrade retries without uninstalling, so the running VM
stays up until someone fixes the value. #4386 makes the same trade-off.

The check runs at render time, not at admission. The schema still
accepts `500m`, so the API takes the object and the error shows up in
the HelmRelease status. Narrowing the schema to integers, as the issue
also proposes, would move the error to admission, but it needs a
migration of stored values. I left it for a separate change.

The field descriptions now say which values are accepted, and `make
generate` carried them into the README, the schema, the Go types and the
ApplicationDefinition.

The new `tests/resources_values_test.yaml` covers the accepted and
refused spellings for cpu, sockets and memory.
`tests/instancetype_resources_test.yaml` gets a case for a lone cpu next
to `cpuModel`. The whole vm-instance suite passes. Every refusal case in
the new file, and the `"1e1"` case, fails against the current `main`
template.

Closes #4104

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

I walked the trigger map against the diff. No field is added, removed or
renamed, and no default or enum changes. Only three field descriptions
change, and the website reference page is regenerated from the README at
release time.

### Release note

```release-note
fix(vm-instance)!: resources.cpu and resources.sockets must now be positive whole numbers, and resources.memory a positive quantity. A cpu or sockets that is not a plain whole number within uint32 (for example 500m, 1k or 1.5), or a zero or negative memory, now fails at render time with an error naming the field, instead of silently becoming a one-core VM or a zero memory request. A release carrying such a value fails its next upgrade until the value is fixed. A value such as 1e1 or 2., which used to be truncated to one core, now sizes the VM with the count it spells.
```



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* VM sizing now accepts only positive whole-number CPU and socket values
within the supported range. Fractional, millicore, zero, negative, and
out-of-range values are rejected during rendering.
* Memory values must be positive quantities. A CPU or socket value
specified alone does not size the VM and is ignored; incomplete sizing
is rejected where required.

* **Documentation**
* Updated VM instance resource guidance to clarify CPU and socket
requirements, memory validation, and how socket values define vCPU
topology.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 17:58:36 +04:00
Aleksei Sviridkin 99de79ff52 fix(cdi): pin the CDI release kubevirt-cdi is reconciled against and stop update overwriting it (#4107)
## What this PR does

`packages/system/kubevirt-cdi` resolved its CDI version at run time from
GitHub's `releases/latest` redirect and wrote the result straight over
`templates/cdi-cr.yaml`. That file is not vendored. Upstream ships a
bare CDI object; ours adds Helm templating for `cloneStrategyOverride`,
`uploadProxyURL` and the worker-pod `podResourceRequirements`, the
`ExpandDisks` feature gate, a `CriticalAddonsOnly` toleration, and the
two RBAC documents that let a tenant clone from the golden-image
namespace. I diffed the two at v1.66.1 to write that list, so it is
complete as of that release rather than only hedged: upstream ships 283
bytes with one feature gate, two nodeSelectors and an `imagePullPolicy`,
and we are a strict superset of it. Every entry arrived by hand, so
`wget -O templates/cdi-cr.yaml` was not refreshing a vendored file, it
was overwriting a first-party one. Same loss as the `rm -rf templates`
that #4097 removes, arriving through the fetch instead of the wipe.

`update` now pins the release the object is reconciled against, and
fetches it only to confirm the bytes still hash to a digest I measured.
Nothing is written under `templates/`. At the next CDI bump whose asset
differs, that check fails, which is the point: upstream moving its own
defaults is when the local copy needs a human to reconcile it, not an
overwrite. The download is kept on a mismatch, since reconciling one
starts by reading it.

`RELEASE` is `v1.66.1`, which is the release `kubevirt-cdi-operator`
vendors, so the pin and the deployed version agree. What the platform
deploys is decided by that package, and it can move to another release
without this file changing, so this line is pointed at it by hand.
`update` then checks the two agree: it compares `RELEASE` with the
`cdi-operator` image tag that package vendors and fails naming both
versions when they differ, because otherwise its closing line would
report a release the platform may no longer run. That is a check and not
a derivation, so `RELEASE` stays a pin a human moves.

Upstream publishes no checksum for the asset, so the pin comes from a
download rather than from a published file:
`a497f90de608c1df26f9ee4095289373f17132d74a2042ca03e00c17c964f8a7`. It
catches an asset that moved under a tag that should be immutable. It is
not authenticity verification, and the comment beside it says so.

The digest command is chosen when `update` expands it, `sha256sum` when
it is on PATH and `shasum -a 256` otherwise. Which one ran here is worth
stating, because the usual claim about macOS is out of date: this
machine picked `sha256sum`, and that binary is `/sbin/sha256sum`, a
native Darwin one rather than GNU coreutils. It reports `sha256sum
(Darwin) 1.0` and prints the same `<digest> <path>` format.

The package already had a `test:` target, from `0418577f5`, so what it
gains here is the `.PHONY` line that commit did not add.
`hack/package.mk` declares only the nine targets it defines itself, so
without it a file or directory named `test` or `update` beside the
Makefile makes make consider that target up to date and skip the recipe
in silence. The new suite pins the `ExpandDisks` gate, the toleration,
`podResourceRequirements`, the `uploadProxyURL` override and the clone
RBAC, and renders the upload-proxy Ingress and TLSRoute, each under the
`gateway-enabled` value that selects it. The clone-strategy override is
pinned by `tests/clone_strategy_test.yaml`, which is already on `main`.

One consequence of the pin move is a claim rather than a number, so it
gets its own line. Upstream carried `WebhookPvcRendering` from v1.63.0
through v1.65.0 and dropped it by v1.66.0; the local object never
carried it, so the divergence ran both ways across those three releases
and one way since. The comment says to re-derive the delta at each bump
instead of trusting it, because that was a fact about a span of releases
and not about CDI.

`packages/system/kubevirt-cdi/Makefile` carried two identical `test:`
targets, one from `0418577f5` and one appended by a later merge, so make
printed `overriding commands for target 'test'` on every invocation in
the package. The second sat inside the span this recipe rewrites, so it
goes with the rewrite rather than being re-authored into it. The comment
kept above it is the reason a reader should not add another. Closes
#4157.

### Scope

An earlier revision also pinned and tested `kubevirt-cdi-operator`.
#3920 has since landed the two real bugs in that recipe, the patch strip
level and `patches/tolerations.diff`, so that half is dropped here
rather than merged twice. Pinning and testing the operator package is
#4190.

Named improvements left alone, so they read as choices.

- The two upload-proxy tests each render their route under the
`gateway-enabled` value that selects it and neither asserts the other is
absent; the templates already settle that with `ne "true"` and `eq
"true"` on the same variable, and `tests/standalone_render_test.yaml`
already asserts each route absent under the values that select the
other. So the package does cover it; these two cases just do not, and a
third copy would not add anything.
- The CDI object carries no version string for a rendered-output
assertion, so `RELEASE` is checked by `make update` rather than by `make
test`.
- Neither route case pins the backend service port, 443 in both
templates; that is outside what the suite claims.
- The template-loss guard still counts `cdi-cr.yaml` as named, because
the download line in `update` mentions the stem, so re-adding a wipe
here would have it report two files instead of three; the guard's header
already says a mention is not proof of a write, and the case needs a
wipe this recipe no longer has.
- The suite pins the local additions by design, and most of the
upstream-derived half is pinned by nothing: `imagePullPolicy` and the
two `nodeSelector` blocks have no assertion anywhere under `tests/`.
`HonorWaitForFirstConsumer` is the exception and only as a side effect:
`featureGates` is pinned whole, so that gate rides along with
`ExpandDisks`. The rest are maintained by the same hand reconciliation
the pin forces whenever the upstream asset changes, so a carry-over that
drops one of them is green. Three asserts would close what is left.
- One more, remote enough to name rather than code around: if the digest
tool is missing entirely, `$$actual` comes out empty, the comparison
against `CR_SHA256` fails, and the error text diagnoses a moved asset
when the real cause is a machine with neither `sha256sum` nor `shasum`.
Busybox ships the first and macOS ships the second, so nobody has hit
it. Closing it means an executable line, which would then owe mutation
coverage of its own, and that is disproportionate to a case with no
known instance. The mechanism is written here so the next reader does
not have to re-derive it from an error message that points the wrong
way.
- Separately, the per-package `.PHONY` gap this Makefile's own line
argues against is not specific to this package: most package Makefiles
do not declare their own `test:` or `update:` phony, and #4214 tracks
that.
- A key that does not exist yet is not held either: every item in the
delta is pinned whole, but a NEW key appearing at `spec` or
`spec.config` level passes unnoticed, and such a key can change
behaviour rather than sit inert. I confirmed it on a copy extracted from
this commit, appending `spec.config.filesystemOverhead.global: "0.99"`
and getting a green suite. That is the shape to recognise at the next
bump: not a changed value, which is caught, but a new sibling key beside
the ones we pin.
- And `spec.config.podResourceRequirements` is now pinned whole here and
by four leaves in `tests/importer-resources_test.yaml`: the whole-map
form is the stronger one and the leaves are a subset of it, so changing
the default takes two edits and the second failure reads as a surprise
until you know why both exist.
- If the operator's image reference ever gains a digest
(`:v1.66.1@sha256:...`), `${deployed##*:}` takes the digest's hex
instead of the tag, and `update` fails with a loud false mismatch rather
than passing silently.
- Deriving `RELEASE` with `$(shell ...)` from the sibling manifest was
suggested and declined on merits: a derived value is not a pin, the
point is that a human chooses which CDI release this object is
reconciled against, and computing it from whatever the neighbour last
vendored hides that choice from the diff.

The operator comparison lives in `update` and not in `test:`, even
though `kubevirt-operator` ties its own manifest to the tree from
`test:`. A mismatch calls for a hand reconciliation, and `update` is the
recipe that asks for one; it is also where the success line that would
otherwise name the wrong release is printed. I checked it on a copy
extracted from this commit. With the operator's image tag moved to
v1.67.0, `make update` exits 2 and names both versions. With the tags
equal it passes. With the check's `exit 1` removed, the moved tag passes
again, so the red comes from the check.

Worth knowing when reading the mismatch path: `kubevirt-operator`
deletes its download on a hash mismatch and this package keeps it. That
is deliberate here, since reconciling a moved asset starts by reading
it, but the two packages do read oppositely and neither file says so.

### Base and ordering

This stacks on #4097 and should land after it. Both edit
`packages/system/kubevirt-cdi/Makefile`, and this change builds directly
on that one's edit to the same recipe, so it is based on the head of
`fix/cdi-update-keeps-local-templates` rather than on `main`. GitHub
computes mergeability against the PR's base branch, which here is that
stack branch and not `main`. Both branches are rebased onto current
`main`, which has not touched these paths since the previous merge-base.


### Screenshots

No UI changes.

### Downstream repositories

Walked the trigger map against the diff. Two entries come close and
neither fires.

The website triggers on developer tooling and package Makefiles, but
`content/en/docs/next/development.md` documents `make update` only as
"Update Helm chart and versions from the upstream source", which still
holds, and its worked example is cilium, which this does not touch. Its
`make test` line refers to the e2e sandbox in `packages/core/testing`,
not per-package unit tests.

`ccp` triggers on `hack/` layout and on what a make target does, and
nothing under `hack/` is touched. Its `package-bump` skill already
assumes a package's `update` recipe hardcodes the upstream version
instead of resolving it at run time, and names cilium, kafka-operator,
opensearch-operator and postgres-operator as examples. `kubevirt-cdi`
was one of those counterexamples and now is not. Pinning
`kubevirt-cdi-operator` the same way is #4190 and out of scope here. The
skill handles vendored Helm charts, in-repo image builds and version
enums, and a raw manifest download is none of those. Everything else in
the map needs an app, a schema, an installer value, a variant or a
release asset.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(cdi): `make update` in kubevirt-cdi no longer overwrites the hand-maintained CDI object. It pins the CDI release that object is reconciled against instead of resolving GitHub's releases/latest at run time, and verifies the fetched asset against a pinned sha256 without writing under templates/.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Reliability**
* Updates now use verified, pinned CDI releases to prevent unintended
manifest changes.
* Failed or checksum-mismatched downloads no longer overwrite existing
configuration.

* **Bug Fixes**
  * Preserved local CDI customizations during release updates.
* Added validation for CDI resources, access controls, upload-proxy
routing, and TLS configuration.

* **Tests**
* Added automated checks for incomplete template refreshes and generated
Kubernetes resources.
* Added chart tests covering CDI behavior, clone permissions, and
upload-proxy networking.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 17:53:23 +04:00
Myasnikov Daniil bc8c54d6e6 feat(kubernetes): boot tenant worker disks from a shared golden Talos image
Every tenant worker VM imports the same Talos raw image from the Image Factory
over HTTP, once per worker: a 20-worker cluster fetches the identical ~4.15 GiB
artifact 20 times, and each fetch is a live dependency on the Factory at exactly
the moment a node joins.

packages/system/kubernetes-worker-image is a new opt-in catalog that holds one
golden DataVolume per (schematicID, version) in cozy-public, named
talos-worker-<schematicID>-<version>. A worker pool opts in with osImage.builtin
on its own KubernetesNodes resource, and its disk-system DataVolume then carries
source.pvc pointing at that golden, so CDI performs a storage-layer CSI clone
instead of an HTTP import -- one import per flavor, shared by every worker of
every tenant. Confirmed on a LINSTOR-backed cluster: the worker disk is a CSI
clone plus a resize, with no host-assisted upload pod.

Both halves are off unless asked for. The catalog is registered through the
platform's optional-package helper and disabled by default, so a cluster that
does not enable it creates no golden and keeps importing over HTTP. The disk
source is derived entirely from osImage, which is absent by default, so a pool
that does not set it renders byte-identical to the pre-feature chart and
adopting the feature never rolls existing workers. Editing osImage on an
existing pool re-images that pool, as changing any other image reference does.

What the render refuses, rather than degrading into a silent fallback:

- A pool whose golden has no catalog entry, or whose golden terminally failed to
  import. A golden that is merely mid-import is not an error; CDI holds the
  worker disk until the source populates.
- A pool whose StorageClass differs from its golden's. CDI cannot CSI-clone
  across classes and falls back to a host-assisted copy over the pod network,
  which is the transfer this feature exists to remove. An empty class on either
  side means the cluster default and is resolved before the comparison, and a
  pool that has already cloned reads its class from its own bound PVC rather
  than from today's default -- the golden's side is frozen the same way, and an
  admin moving storageclass.kubernetes.io/is-default-class must not fail the
  release of a pool whose disks are all on the right class. That resolved class
  is then written onto the worker disk, so the disk binds where the guard
  checked instead of on whatever the default has become by then -- reading one
  class and binding another is how a pool at an empty storageClass passed the
  comparison and still took the host-assisted copy. The HTTP path leaves an
  empty class to the cluster as before, which is what keeps its render, and the
  KubevirtMachineTemplate hash named after it, byte-identical.
- A diskSize below the golden's storage size, which a clone target cannot be.
- A schematicID that is not a plain lowercase alphanumeric identifier, or a
  version that is not a vMAJOR.MINOR.PATCH release. Both are interpolated into
  a DataVolume name and an image URL, and values.schema.json types them as bare
  strings, so a malformed one otherwise reaches the render. The catalog applies
  the same patterns, since both sides build the same name from the same fields.
- An images[] entry edited after its golden exists. A DataVolume spec cannot be
  patched in place, so the message names the entry and the supported move -- a
  new (schematicID, version) -- instead of leaving the API server to reject the
  patch on the next upgrade.

Goldens carry helm.sh/resource-policy: keep, so dropping an entry stops
managing a golden rather than deleting it: a missing golden aborts the whole
Kubernetes HelmRelease of every tenant pinned to it, not just the disk. Each
cloned worker disk annotates the golden it came from and the pool that owns it,
so "does anything still clone this?" is answerable before a manual delete
instead of after.

A builtin pool needs a StorageClass whose volumes are reachable from any node,
in practice a replicated/DRBD one. The golden in cozy-public has no consumer and
binds as soon as it imports, wherever CDI's importer landed, while a worker disk
binds where its VM is scheduled, and nothing here arranges for those to be the
same node. The platform ships cloneStrategyOverride: csi-clone, which rules out
the host-assisted strategy that would cross nodes. What a node-pinned class does
in that situation is not measured, so values.yaml states the requirement and not
a symptom.

ComputePlane pools forward only the per-pool shape fields, and osImage is
excluded there by name: the worker OS image, its Talos installer and its Image
Factory are operator decisions about the platform's own images.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 15:35:44 +03:00
Aleksei Sviridkin 9e0e1239da fix(kubevirt-cdi-operator): pin the vendored CDI release (#4383)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

This pins the CDI release the platform deploys, so it moves only when
someone edits a version line. Until now `make update` in
`packages/system/kubevirt-cdi-operator` took the release from GitHub's
`releases/latest` redirect. Any refresh could move CDI and its image
tags, and the diff had no version line to review. The fix follows the
shape `packages/system/kubevirt-operator` already uses.

- `RELEASE = v1.66.1` names the release. It is the one already vendored,
so `templates/` does not change, and `make update` reproduces it byte
for byte.
- `MANIFEST_SHA256` pins the downloaded asset and matches the digest
GitHub publishes for it. On a mismatch the recipe writes nothing.
- The download is transformed and patched outside `templates/` and moved
in at the end. A refresh that fails halfway leaves the vendored file as
it was.
- A new `test` target runs a helm-unittest suite over the vendored
manifest, then greps it for the operator image at `$(RELEASE)`. The grep
is there because helm-unittest cannot see a Makefile variable. Moving
the pin without re-running `update` fails here.

One thing is left for later. The suite asserts three of the eight image
references directly, and the digest keeps the rest in step on `update`,
but a hand edit to one of the other image env entries would pass.

Closes #4190

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

Nothing on the trigger map applies. The diff touches one package's own
Makefile, `.gitignore` and tests, and no shared tooling, values, CRD,
namespace or release asset.

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(kubevirt-cdi-operator): pin the vendored CDI release to v1.66.1 and verify the downloaded manifest against a digest, so refreshing the package no longer moves the deployed CDI. The deployed version stays the same.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Updates**
* The KubeVirt CDI Operator is now pinned to version 1.66.1, providing a
consistent operator version across deployments.
* **Reliability**
* Operator manifest updates are checked before they’re applied, helping
prevent incomplete or mismatched manifests from replacing the current
version.
* Deployment configuration is validated to ensure it matches the pinned
release.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 16:20:04 +04:00
Aleksei Sviridkin eec069b94f docs(kubevirt): name every payload line the tail checks do not cover (#4340)
## What this PR does

Follow-up to a review comment on #4254 that was right and landed after
the merge.

The sentence that PR left in `packages/system/kubevirt/Makefile` says
the device and CPU payload lines stay outside the marker checks, which
reads as if the rest are handled. The `- {{ . }}` entries inside the
`extraFeatureGates` and `disabledFeatureGates` ranges are neither device
nor CPU and are not covered either. Deleting either of them from a copy
of the CR and running `make update CR_FILE=<copy>` exits 0 and leaves
the range wrapped around nothing, with no check firing; dropping `-
Template` from a clean copy fails the target as a control.

The wording now names every uncovered payload line: the device tables,
the `cpuAllocationRatio` value and both feature-gate entries, and keeps
`migrations` as the one block validated whole. Covering those payloads
in code is a wider change; happy to do it separately if that is wanted.

### Screenshots

Not a UI change.

### Downstream repositories

Comment-only change in one Makefile.

- [x] No downstream repository is affected by this change

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Clarified update validation guidance for disabled-feature markers.
* Documented partial-block recovery requirements and the workflow for
restoring missing markers.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 12:37:10 +04:00
Aleksei Sviridkin c290dee945 fix(vm-instance)!: refuse resources that cannot size the VM
resources.cpu and resources.sockets are typed as quantities, but they
land in domain.cpu.cores and domain.cpu.sockets, which KubeVirt
declares as uint32. The chart cast them with int64, which turns 500m,
1.5 or 1e1 into 0 without an error, and KubeVirt boots a zero core
count as one core. The user asked for half a core and got a one-core
VM with nothing reporting it.

The raw spelling is now decoded as YAML, the way an integer field
reads it, and the render stops unless the result is a positive whole
number that fits in uint32. A leading zero is refused, because YAML
reads 010 as octal 8. The check keys on presence rather than
truthiness, so an unquoted 0 is refused instead of being read as
unset.

Memory gets the same presence check. A quoted "0" was truthy and
rendered requests.memory: "0", while an unquoted 0 was falsy and
dropped, so one quantity took two paths. A zero or negative amount
now stops the render in both spellings.

BREAKING CHANGE: values that used to render now either fail or
render differently. A cpu or sockets that does not decode to a plain
whole number within uint32 (a millicore, suffixed such as 1k,
fractional, zero, negative or leading-zero value), and a zero or
negative memory, fail the next upgrade of the release until the
value is fixed. This includes cpu or sockets set on its own beside an
instanceType, which used to be ignored whatever it held. A spelling
that the int64 cast truncated to 0 but that YAML decodes to a whole
number, such as the exponent 1e1 or the trailing dot 2., used to boot
one core and now sizes the VM with the count it spells, so such a VM
changes size on upgrade.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 11:31:07 +03:00
Aleksei Sviridkin 89da4a121c fix(cdi): stop make update from deleting first-party templates (#4097)
## What this PR does

`make update` in `packages/system/kubevirt-cdi` began with `rm -rf
templates` and then refetched one file, `cdi-cr.yaml`. Nothing in that
directory is purely vendored. Two of the three manifests, the
upload-proxy Ingress and TLSRoute from the Gateway API work, are ours
and no fetch recreates them, so the next CDI bump would have dropped
upload-proxy exposure from the chart and still exited 0. The third is
`cdi-cr.yaml` itself, which despite its name carries Helm templating,
the `ExpandDisks` gate, a toleration and the two RBAC documents that
preauthorize cloning from `cozy-public`; the fetch overwrites it, and
removing the wipe does not change that.

That recipe now opens with `mkdir -p templates` and deletes nothing,
which is where `packages/system/kubevirt-operator` landed at the last
KubeVirt bump. The wipe bought nothing there anyway: `wget -O` truncates
the file it writes.

`kubevirt-cdi-operator` has the same recipe shape, and the same edit
there would be a regression rather than a fix. It applies two patches,
and since #3920 moved them to `-p4` they resolve, so a hunk upstream has
moved out from under leaves `templates/cdi-operator.yaml.rej`; its `awk
-i inplace` step leaves `templates/cdi-operator.yaml.gawk.XXXXXX` if
gawk dies abnormally. Helm renders every file under `templates/`, and I
reproduced both: the reject fails the package with a YAML parse error,
and the gawk temp renders a duplicate object at exit 0. The wipe is what
takes both, so removing it for symmetry would trade a latent loss for a
live one, and the quieter of the two is the one that survives.

Deleting the wipe there wants the recipe restructured to transform
outside `templates/` and move the result in once, which is what
`kubevirt-operator` does today and what its own comment gives this exact
reason for. That is a larger change than this one and belongs on its
own; #4383 makes it. Meanwhile the scan below covers the package: I
added a first-party file to its `templates/` and the guard reported it,
so the day that package grows one, this fails rather than the file being
lost.

`hack/package-update-templates.bats` guards this across the tree rather
than in these two packages. It scans every package Makefile and fails
when a recipe that clears its own `templates/` leaves behind a tracked
file it never names. Put the removal back into `kubevirt-cdi` and it
goes red naming both upload-proxy files. A file counts as named only
outside the Makefile's own `NAME` and `NAMESPACE`, however the
assignment is written. The match is on the variable name rather than on
a list of spellings: the tree already carries three of them, `export
NAME=`, the bare `NAME=`, and `NAME := ` in `packages/core/testing`, and
the hole is identical in each. Without that, a package called `foo-bar`
satisfies the check for its own `templates/bar.yaml` through its
identity alone. Nothing the scan reaches is exempted that way as the
tree stands, so the exclusion is there for the first package that would
be: rename the asset in `kubevirt-cdi-operator`'s recipe and put the
removal back, and the guard reports the orphaned template only with the
exclusion in place. The two variables are matched by name rather than by
a `[A-Z_]+=` shape, because a shape that broad is one unspaced
assignment away from swallowing a variable that does name templates,
which is how `etcd-operator-crds` names all five of its CRDs. A control
test runs the same helpers over a fixture broken the same way, over one
whose name swallows a template stem, then over that fixture rewritten to
spell the removal with a trailing slash, to take one file instead of the
directory, and to drop the removal entirely. Both cases pass under
`hack/cozytest.sh hack/package-update-templates.bats`.

Two limits, both written into the file's header. The check fires on the
consequence rather than the shape, so `rm -rf templates` in a package
that does refetch everything stays green, which is what keeps
`kubevirt-cdi-operator` green for as long as its recipe wipes. And it
only looks at a recipe clearing its own directory: `cert-manager` clears
`../cert-manager-crds/templates/*` across the package boundary, which
needs the removal's target resolved rather than the wiping package's
own, so that directory is unguarded. The other packages that clear their
own `templates/` refetch all of it and are untouched here.

Two more limits the header names. The scan reads the text of each
package's own `update:` recipe, so a removal made elsewhere is out of
range: in a prerequisite target, behind an include, or behind a
recursive make. `capi-operator`, `clustersecret-operator`, `openbao` and
`reloader` write `update: clean <name>-update` and take both recipes
from `hack/package.mk`, which removes nothing under `templates/` today.
And the glob stops at `packages/*/*/Makefile`. The package Makefiles off
that depth are the group aggregators such as `packages/system/Makefile`
and the vendored kamaji subcharts, and none of them declares `update`.

One behaviour of the matcher worth naming, and deliberately not changed
here. `wipe_command` evaluates `/^update:/` ahead of its terminator, so
a second `update:` in the same Makefile re-arms the scan instead of
ending it, and what that means depends on what sits between the two:
adjacent or separated by a blank line, both recipes are read; separated
by another target, only the first is. The direction that matters is a
false positive rather than a miss. A Makefile whose first `update:`
clears `templates/` and whose second does not would be reported by this
scan, while make runs only the second, so the package fails over a
recipe that never executes. No package in the tree defines `update:`
twice today.

Three more notes, all non-blocking and none acted on. The header says
what the scan leaves uncovered is a recipe clearing someone else's
`templates/`; that is the uncovered set for *deletion*, and
`kubevirt-cdi` is green under the scan while still losing
`cdi-cr.yaml`'s content to the in-place `wget -O` on every run, which
the Makefile comment records but the header does not. The identity strip
has a false-positive direction the header does not argue: a wiping
package writing `templates/$(NAME).yaml` would be reported even though
the recipe writes it, which no package does today. And the fixture
stages `.gitkeep` with `git add --all`, so a developer whose global
excludes list that name would get a red for environment rather than
defect; `--force` would remove the dependency.

The stem match is generous, and the helper's comment now names the two
live shapes. A sibling name: `templates/crds.yaml` next to the existing
`templates/crds-experimental.yaml` on `gateway-api-crds`. A mention from
an unrelated context: `templates/sidecar.yaml` on
`objectstorage-controller`, whose Makefile says `sidecar` across its
image targets while its recipe clears `templates/` and writes only
`templates/controller.yaml`. I fed both to the helper and both come back
exempted; a stem the Makefile never mentions is still reported.

The duplicate `test:` target in this Makefile is #4157 and stays there.
It sits outside the hunks this diff touches, so removing it here would
be reaching past what this change is about.

One more note, not a defect. Where the comment points at
`kubevirt-operator` as the shape that closes the transient-file class,
`packages/system/kubevirt/Makefile` is the closer analogue for the
`cdi-cr.yaml` half specifically: its `update` downloads nothing and
re-applies the parameterization onto a hand-maintained CR with a
per-marker self-check. And the failure message offers a third option,
staging into a tmpdir the way `etcd-operator-crds` does, in a
parenthetical that reads as a gloss on the option before it.

What this PR does not fix, and what does. `make update` here still
overwrites `templates/cdi-cr.yaml`, which the Makefile comment says
plainly. A wholesale overwrite is loud: upstream's object carries
neither `podResourceRequirements` nor `cloneStrategyOverride`, and
`tests/importer-resources_test.yaml` and
`tests/clone_strategy_test.yaml` pin both, so `helm unittest` goes red.
The silent case is a partial hand reconciliation that keeps those two
and drops something else, because nothing here pins the `ExpandDisks`
gate, the `CriticalAddonsOnly` toleration, the `uploadProxyURL` override
or the two RBAC documents that preauthorize tenant cloning from
`cozy-public`. #4107, stacked on this branch, stops `update` writing the
file at all and adds `tests/local_templates_test.yaml`, which pins
those.

One limit the header does not name, left alone here. The comment strip
is anchored to the start of a line, so a trailing comment on a recipe
line survives into the searched text and counts as naming the file it
mentions. I confirmed it by feeding `wget ... -O templates/cdi-cr.yaml #
local-extra.yaml is written by hand` through the filter. Nothing in the
tree is written that way, and the header only claims comment lines are
dropped, which is what it does. `sed 's/#.*//'` in place of the `grep
-v` closes it if it ever matters.

Closes #4028.

### Screenshots

No UI change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff. Two entries come close: the website's "developer tooling (the
package `Makefile`s)" line, and ccp's "move or rename anything under
`hack/`". Neither fires. `make update` still behaves as the website's
development page describes, nothing under `hack/` moved or was renamed,
and the two files ccp's skills key on (`hack/package.mk`,
`hack/common-envs.mk`) are untouched.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(cdi): `make update` in kubevirt-cdi no longer deletes its first-party templates
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Updated Kubernetes CDI package updates to preserve existing template
files while refreshing downloaded manifests.
- Prevented update operations from clearing first-party and vendored
template content.

- **Tests**
- Added safeguards to detect package update recipes that could remove
tracked templates without restoring them.
  - Added Helm unit test support for the Kubernetes CDI package.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 12:23:58 +04:00
Timofei Larkin a3911811c7 feat(api): take Application name limits from the application's schema
The API server capped Kubernetes and Kafka names by knowing what those
two charts render: worker pool releases under a 17-character prefix,
controller pods with a 13-character suffix. That is the charts'
knowledge sitting in the platform, and it does not generalise. The next
application with a sub-resource budget needs another branch on Kind,
and an application served from outside this repository cannot get one
at all.

An application now declares the limits on its own name in its schema,
under x-cozystack-name, and the server applies whatever it finds there
for whatever kind the ApplicationDefinition describes. A declaration
only tightens: prefix + name must still fit the 53-character Helm
release name limit. A rejected name is answered with the chart's own
explanation of the limit.

The schema is untrusted input, and definitions from other repositories
are written by hand. A declaration that cannot be enforced as written,
whether an unknown keyword, a wrong type, a limit no name satisfies or
a pattern Go cannot compile, refuses creates of that kind with a
configuration error. Dropping it would admit names the chart cannot
render; applying it would reject every name and blame the user.

An upgrade brings cozystack-api up before the definitions it serves are
re-rendered, so for a while the new server can read a Kubernetes or
Kafka definition that predates its declaration. The two old caps stay
as a floor for exactly that case, a definition that declares nothing,
until no supported upgrade starts from one.

Assisted-by: LLM
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-09-23 10:59:51 +03:00
Timofei Larkin 06b41da030 feat(apps): declare the Kubernetes and Kafka name caps in their charts
Both caps are facts about what each chart renders, and nothing outside
the chart can derive them. Every Kubernetes worker pool is a separate
KubernetesNodes release named kubernetes-nodes-<name>-<pool>, so a
cluster name has to leave room for the default md0 pool inside the
53-character Helm release name limit: 32 characters. A Kafka KRaft
controller pod is named kafka-<name>-c-<hash>-<id> with an 8-character
hash, and that hostname has to fit the 63-character DNS-1123 label
limit: 44 characters.

Each chart now states its cap with a @name annotation, which
cozyvalues-gen v1.7.0 publishes as x-cozystack-name in
values.schema.json and hack/update-crd.sh carries into the
ApplicationDefinition. The pre-commit job moves to that release, the
first that understands the annotation.

Assisted-by: LLM
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-09-23 10:59:51 +03:00
Aleksei Sviridkin ce35da451f docs(api): separate the healthCheckExprs failure modes (#4387)
## What this PR does

The `healthCheckExprs` field doc said any bad expression makes the
HelmRelease hang until its timeout. That is true for some mistakes and
wrong for others, and the text ships in the CRDs, so `kubectl explain`
sends people to the wrong place.

The doc now lists the cases separately, checked against the controller
versions this repo installs (helm-controller v1.5.0, fluxcd/pkg runtime
v0.100.2, Helm v4.1.1). An expression that does not parse, an empty
`current`, or a duplicate group and kind stall the release right away
with reason `InvalidCELExpression`, and no Helm action runs. An
expression that errors on the object keeps the wait polling until the
timeout with the CEL error in the message. A `current` expression that
evaluates but never turns true also runs the timeout out, with no error
to point at it. A wrong group or kind leaves the expressions unused.

Two details go against the issue text. Reading `status` before the CR's
controller writes one errors even inside `has()`, as the issue says, but
the object is only Unknown on that poll, and the wait still succeeds
once a status shows up. A CRD that is not installed does not hang the
wait: Helm fails to build the manifest and the action fails before any
wait starts.

Doc comments and the regenerated CRD YAML only, no behaviour change.

Closes #4154

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
docs(api): describe each healthCheckExprs failure mode separately in the ApplicationDefinition and PackageSource field docs.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Expanded the health-check expression guidance to explain how invalid
or duplicate expressions, evaluation errors, expressions that remain
false, unmatched resource types, and missing CRDs affect Helm releases.
* Clarified that health-check expressions are not evaluated when Helm
action waiting is disabled.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 11:52:49 +04:00
Aleksei Sviridkin d48d3ecc91 test(console): add coverage for enrichSchemaWithEnums (#4186)
## What this PR does

backup-utils.ts (in the in-tree console dashboard) had no test file at
all, despite sibling files in the same directory
(dynamic-options.test.ts, crd-option-sources.test.ts,
immutable-paths.test.ts) establishing that convention. Adds
backup-utils.test.ts covering enrichSchemaWithEnums: the non-object
passthrough, a path with no match in the enum map, a top-level enum
attach, a nested one, and that the caller's path array is not mutated
(verified by reading the function, it builds a new array via spread on
every recursive call rather than mutating the one it receives).

Test-only change, no production code touched. Verified with vitest run
src/lib/backup-utils.test.ts (5 tests pass) and tsc --noEmit (no new
errors).

### Screenshots

N/A, test-only change, no UI impact.

### Downstream repositories

Walked the trigger map in docs/agents/contributing.md against the diff.
This is a single new test file for a pure function, no production code,
schema, API, CRD, annotation, or naming changed.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Tests**
- Added coverage for schema enum enrichment, including top-level and
nested paths.
  - Verified that unrelated schemas remain unchanged.
- Confirmed unmatched paths and caller-provided path arrays are handled
safely.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 11:42:16 +04:00
Yan Bondarenko b6ed32c441 docs(kubevirt): name the disabledFeatureGates entry among unchecked lines
The disabledFeatureGates range renders each gate a user adds through a
`- {{ . }}` line with the same shape as the extraFeatureGates entry, and
no check looks at it either: deleting it from a copy of the CR leaves
`make update` exiting 0 and the configured gate silently stops taking
effect. The comment listed the extraFeatureGates entry alone, so it read
as complete while one uncovered line was missing from it.

Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-09-23 11:17:25 +04:00
Yan Bondarenko 63c9271568 docs(kubevirt): name every payload line the tail checks do not cover
The sentence #4254 left behind says the device and CPU payload lines stay
outside the marker checks, which reads as if the rest are handled. The
extraFeatureGates entry is neither device nor CPU and is not covered either:
deleting `      - {{ . }}` under its range on a copy of the CR and running
`make update CR_FILE=<copy>` exits 0 and leaves the range wrapped around
nothing, with no check firing. The sentence it replaced covered that case.

Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-09-23 11:17:25 +04:00
Aleksei Sviridkin 90a1a947c5 feat(kube-ovn): bump kube-ovn to v1.15.26 (#4331)
## What this PR does

Bumps kube-ovn from `v1.15.10` to `v1.15.26` (package `cozy-kubeovn`
`0.38.0` -> `0.39.0`).

The motivation is a cold-start failure mode in the OVN NB/SB raft leader
checker. In `v1.15.10`, `checkDBClusterIntegrity`
(`pkg/ovn_leader_checker/ovn.go:336`) called `os.Remove` on a healthy
local raft DB (`ovn.go:367`) whenever a leader observed fewer members
than expected. That count drops below the target transiently during
cluster growth, a rolling restart, or recovery after a node address
change, so the check could delete a valid DB and leave `ovn-central`
unable to elect a leader. `v1.15.26` removes that function entirely, so
the DB is no longer deleted. It is a removal of the faulty check, not a
grace-period around it.

The chart is re-vendored with `make update`, not hand-edited, so the
templates match the image:

- `charts/kube-ovn` is re-pulled from the `v1.15.26` tag and
`patches/mtu.diff` is re-applied; the image build args are regenerated.
- This brings the whole v1.15.26 chart: the reworked master-node
affinity helpers, the widened kube-ovn CRD schema
(`bandwidth.ingress`/`egress` from `integer` to int-or-string,
`u2oFeatures` added), and the new `OVN_CENTRAL_MODE`
(`cluster`/`single`) support with its `central-svc`, `central-pvc` and
`central-hcp-rbac` templates. `OVN_CENTRAL_MODE` defaults to `cluster`
(raft on hostPath), so an in-place upgrade keeps the existing topology.
- Two vendored patches now included upstream in `v1.15.26` are dropped:
`fix-keepvmip-portgroup.diff` (`hasAliveSiblingVMPod`, `pod.go:2437`)
and `fix-vmim-scheduling-retry.diff` (scheduling requeue,
`kubevirt.go:195`). `fix-vmim-target-pod-retry.diff` is kept: `v1.15.26`
still returns nil at `kubevirt.go:215`.

The image also picks up `kube-ovn-bfdd-supervisor`, which v1.15.26 adds
(`makefiles/build.mk:33`) and upstream's `dist/images/Dockerfile:12`
copies into `/kube-ovn`. `pkg/controller/vpc_egress_gateway.go:47`
hardcodes that path for the `bfdd` container a `VpcEgressGateway` with
`spec.bfd` gets, so a copy that stops at `kube-ovn-controller` ships a
container that cannot exec. `hack/check-kubeovn-image-copies.sh`
compares what our Dockerfile and upstream's copy into the image, `make
update` runs it against the tag it pulls and
`hack/check-kubeovn-image-copies.bats` runs it on every CI run, because
`make update` otherwise rewrites only the ARG lines and the copy drifts
at each bump.

The `multus` commit moves the `kube-ovn-cni-plugins-checked-against`
marker to `v1.15.26`. `CNI_PLUGINS_VERSION` is `v1.9.1` in kube-ovn's
base image at both releases and in multus, so portmap and macvlan stay
in lockstep; the marker is the only change.

The image ref stays digest-pinned (`repository` + `tag@sha256`), which
the promote-retag release contract requires for a cozystack-built image.
The digest is carried over from `v1.15.10` as a placeholder and is
refreshed to the real `v1.15.26` image by the release-candidate
digest-vendoring step; no PR or main job rebuilds it.

Verification: `helm unittest .` passes (22 tests), `helm template .`
renders clean, the `cni-plugins-staging-contract` and `promote-retag`
unit contracts are green, and the image-build
`TestVMIMigrationSchedulingRetry` gate passes 6/6 on the patched
v1.15.26 tree.

> [!NOTE]
> `start-db.sh` was substantially reworked upstream between `v1.15.10`
and `v1.15.26`. The privileged fork E2E run installs into a sandbox and
passes, but the in-tree E2E job is skipped on a fork PR, and a fresh
install is not an upgrade. An in-place upgrade on a multi-node control
plane (image rebuild plus OVN raft re-convergence) is still untested
before merge.

### Screenshots

N/A: no UI changes.

### Downstream repositories

Walked the trigger map against the diff. The change is confined to
`packages/system/kubeovn` (re-vendored chart, image build args, version
pins) and a one-line marker in `packages/system/multus`. It does not
add, rename or remove a package, does not touch platform values,
variants, components, release assets, `ApplicationDefinition` semantics,
the Talos version, or any hand-typed CRD schema in a downstream
provider. No downstream trigger matches.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(kube-ovn): bump kube-ovn to v1.15.26, which removes the OVN leader-checker integrity check that could delete a healthy raft DB and prevent ovn-central from electing a leader after a node address change or rolling restart
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added hosted control-plane support for OVN Central with StatefulSet
deployment, persistent storage, service exposure, and dedicated RBAC.
- Added split-cluster and data-plane-only configuration with external
OVN endpoints.
- Added configurable TLS, leader election, service, and storage
settings.
- Enhanced CRDs with overlay-only routing and flexible bandwidth
quantities.

- **Bug Fixes**
- Improved VM migration retries during scheduling and preserved
connectivity during live migration.
  - Improved upgrade handling and service configuration validation.

- **Updates**
  - Upgraded Kube-OVN components and images to version 1.15.26.
- Documented required OVN IPsec directory preparation for read-only
filesystems.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 02:42:25 +04:00
Aleksei Sviridkin 0f21d06520 fix(dashboard): raise the console nginx header buffers for large OIDC tokens (#4376)
## What this PR does

A user whose OIDC access token carries many group claims cannot use the
console: nginx answers `400 request header or cookie too large` before
the request reaches the apiserver proxy. The config baked into the
console image kept the nginx defaults, 1k for
`client_header_buffer_size` and four buffers of 8k for
`large_client_header_buffers`, while the token arrives as one Cookie
header several kilobytes long. A header field has to fit into a single
buffer, so the per-buffer size is what decides this. Both directives are
now 16k, at the top level of that config rather than inside the server
block: a header field read before the `Host` header has selected the
virtual server is measured against the default server's buffers, so the
same lines inside a server block cover that server only while it stays
the default one.

Measured in the pinned nginx base image: a 12 KB cookie gets 400 on the
old config and 200 on the new one, a 20 KB one still gets 400. `nginx
-t` accepts the result.

This hop sees more than the browser sent. The gatekeeper in front of it
runs with `--pass-access-token=true` and
`--pass-authorization-header=true`, so the request arrives carrying the
`kc-access` cookie, an `Authorization: Bearer` header and
`X-Forwarded-Access-Token`, three separately bounded header fields where
the edge saw one.

This is the console image only. The platform ingress-nginx controller
also runs on the defaults, which is #2853. Worth reading that trace
again before fixing it there: the controller logged `upstream_status
400`, so it forwarded the request and the 400 came back from behind the
gatekeeper, which is this layer.

The check in `hack/console-nginx-header-buffers.bats` reads the config
out of the Containerfile heredoc and fails when either directive is
missing, below 16k, or no longer ahead of the first server block. That
config lives inside an image build, so no chart template and no
helm-unittest case can reach it. `make unit-tests` picks the file up on
its own.

No image reference changes here. CI stamps the console digest.

Closes #3113

### Screenshots

No UI change. This is the nginx config in front of the SPA.

### Downstream repositories

The diff is the console image's nginx config plus one new `hack/*.bats`
file. `cozystack-ui` is archived and the console lives in this repo, so
a console change belongs in this PR. Nothing under `hack/` moved or was
renamed, and no make target changed behaviour.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(dashboard): the console no longer answers `400 request header or cookie too large` to users whose OIDC access token carries many group claims
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Increased the console’s NGINX request-header buffer capacity to
support larger header fields and request headers.
* Improved handling of requests with headers that previously exceeded
the default buffer limits.

* **Tests**
* Added automated checks to verify the console’s NGINX header-buffer
settings remain correctly configured.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 01:31:45 +04:00
Aleksei Sviridkin 8c5126ccd9 fix(kubernetes): give System-mode tenant OIDC users a UID (#4379)
## What this PR does

Users authenticated through `spec.oidc.mode: System` on a tenant
Kubernetes cluster now get a UID. The AuthenticationConfiguration the
chart renders mapped `username` and `groups` and nothing else, and with
no `claimMappings.uid` entry the apiserver resolves such a user to an
empty UID. The `X-Remote-Uid` header the tenant control plane publishes
then reaches an aggregated API server carrying nothing.

The mapping takes the `sub` claim. The username claim is `email`, and
the platform's Keycloak realm turns on the UPDATE_EMAIL required action,
so that value is one a user changes; `sub` is the account UUID and stays
where it is.

Users on a cluster that is already running gain a UID where they had
none. Tenant RBAC does not notice, because a role binding matches a
username or a group. Audit records on the tenant apiserver do carry
`user.uid` now, `kubectl auth whoami` prints it, and an admission
webhook or extension server inside the tenant that read the empty value
reads the subject instead.

`CustomConfig` mode stays as it was. An operator-supplied configuration
is written verbatim, uid mapping or not, and a unit test pins that the
chart adds none of its own.

Closes #4166

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff, which touches the tenant
chart's OIDC template, its unit tests, `docs/oidc-tenant.md`, and the
shared e2e runner library with its bats twin.

`values.schema.json` does not change. No values key is added, renamed or
removed, no default moves and the version enum is untouched, so the
terraform-provider's hand-written schema and its expand/flatten pair
stay correct. No package is added or renamed and no
`ApplicationDefinition` changes, so the website's app lists and
generated reference pages are unaffected, and its tenant OIDC page
carries no claim about UIDs that this could make stale. The installer
values, the platform values and the variant list are untouched, so the
ansible role is unaffected.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(kubernetes): users authenticated through `spec.oidc.mode: System` on a tenant Kubernetes cluster now carry a UID, taken from the issuer's `sub` claim, where they previously had an empty one. Audit records, `kubectl auth whoami` and the `X-Remote-Uid` header that reaches an aggregated API server inside the tenant all show it. RBAC is unaffected: a role binding never matches on a UID.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* System-mode OIDC authentication now provides a stable user identifier
based on the account subject.
* Existing clusters now expose this identifier in audit records,
`kubectl auth whoami`, and to extension servers without changing tenant
RBAC behavior.
* Custom OIDC configurations continue using operator-defined UID
mappings.

* **Documentation**
* Updated OIDC documentation to explain stable identifiers and their
visibility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 01:31:37 +04:00
Aleksei Sviridkin 225b639c18 fix(cdi): stop make update from overwriting the first-party CDI object
templates/cdi-cr.yaml is ours. Against upstream's bare CDI object it
adds Helm templating for cloneStrategyOverride, uploadProxyURL and the
worker-pod podResourceRequirements, the ExpandDisks feature gate, a
CriticalAddonsOnly toleration, and the two RBAC documents that let a
tenant clone from the golden-image namespace. Five of those arrived by
hand. The toleration did not: it came in with the first commit, which
was byte-identical to upstream v1.58.0, and upstream dropped it at
v1.64.0 while this object kept it. That is the same divergence
`WebhookPvcRendering` records, running the other way. Either way
`wget -O templates/cdi-cr.yaml` was not refreshing a vendored file, it
was overwriting a first-party one -- the same loss as the
`rm -rf templates` this recipe stopped doing, arriving through the
fetch instead of the wipe.

What replaces it: the release the object is derived from is pinned
rather than resolved from GitHub's releases/latest redirect, and
`update` fetches it only to confirm the bytes still hash to a measured
sha256. Nothing is written under templates/. The next CDI bump that
changes the asset fails that check, which is the point -- upstream
moving its own defaults is when the local copy needs a human to
reconcile it, not an overwrite.
The download survives a mismatch, because reconciling one starts by
reading it.

RELEASE is v1.66.1, which is the release the operator manifest
vendors. ../kubevirt-cdi-operator decides what the platform deploys,
and it can move to another release without this file changing.
RELEASE is still set by hand rather than derived, because a
derived value is not a pin. What `update` adds is a comparison with the
cdi-operator image tag that package vendors, failing when the two
differ; without it the recipe's closing line would name a release the
platform may no longer run.

The difference from upstream is stated against v1.66.1 rather than as
a standing fact: upstream carried WebhookPvcRendering from v1.63.0
through v1.65.0 and dropped it by v1.66.0, and this object never
carried it, so the difference ran both ways for three releases and one
way after. The comment says to re-derive it at each bump for that
reason.

The `test:` target this package needed arrived separately, so what it
gains here is the `.PHONY` line that came without: hack/package.mk
declares only the targets it defines itself, so a file or directory
named test or update beside this Makefile would otherwise make that
target up to date and skip the recipe. The package also carried a
second, identical `test:` target, so make warned on every invocation
in it; that one is gone, and the comment above the remaining target is
what should stop another being added.

The suite is what makes the local delta checkable rather than
described, which is why the comment above `update` no longer lists
what the object adds: two copies of that list would diverge, and only
one of them can fail.

hack/package-update-templates.bats attributed `@#` recipe comments to
one named package. The tree writes them in more than one, and this
adds another, so the attribution goes; the case below it exercises
both spellings without naming who writes which. The header in that
file also lost its two-item example of first-party manifests. The
list named the two the recipe never wrote, and this change stops it
writing `cdi-cr.yaml` as well, so the example would have been short
by one the moment it landed.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 00:08:41 +03:00
Aleksei Sviridkin 65a038caa4 docs(api): separate the healthCheckExprs failure modes
The field doc filed every mistake under one outcome, a HelmRelease
that hangs until its timeout. helm-controller treats them differently.
An expression that does not parse is rejected before any Helm action
and stalls the release at once, with a named reason. One that parses
but errors on the object keeps the wait polling with the CEL error
attached, and a current expression that evaluates cleanly but never
turns true runs the wait out with no error at all. A read of status
errors until the CR's controller writes one, even inside has(), but
the wait survives it. A wrong group or kind silently falls back to
the built-in readiness rules, and a CR whose CRD is missing fails the
Helm action before the wait starts.

The text ships in the CRDs and is what kubectl explain shows, so a
reader who cannot check it against the controller source was sent
to the wrong place for the parse error and the missing CRD.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 00:05:35 +03:00
Aleksei Sviridkin 2f5625f0dc fix(cdi): stop make update from deleting first-party templates
The update recipe in kubevirt-cdi opened with `rm -rf templates` and
then refetched one upstream manifest. The directory keeps two more
files -- the upload-proxy Ingress and TLSRoute -- that are written by
hand and never refetched, so the next CDI bump would drop upload-proxy
exposure from the chart while `make update` still exited 0. The clean
slate bought nothing either way: wget -O truncates the file it writes.

The recipe now creates the directory with `mkdir -p` and leaves the
rest of it alone, which is what kubevirt-operator moved to when it was
last bumped.

kubevirt-cdi-operator keeps its wipe. The same change there would be
for symmetry rather than for a loss, and it is not free: both its
patch calls resolve, so a hunk upstream has moved out from under
leaves templates/cdi-operator.yaml.rej, and gawk's inplace edit leaves
templates/cdi-operator.yaml.gawk.XXXXXX if it dies abnormally. Helm
renders every file under templates/, so the first fails the package
loudly and the second renders a duplicate object at exit 0. The wipe
is what takes both. Removing it wants that recipe restructured to
transform outside templates/ the way kubevirt-operator now does, which
is its own change. Until then the scan below is what watches that
package, with the limit the scan already documents: it reports a
tracked template whose stem the Makefile never mentions, so a name
that collides with something already written there, templates/
tolerations.yaml against the patches/tolerations.diff line, is
exempted. It catches the loss it was written for, not every loss.

The bats case scans every package Makefile rather than a pinned list,
so a first-party template added under a recipe that clears templates/
fails the commit that adds it instead of vanishing at the next bump.
A package's own NAME and NAMESPACE do not count as naming a file,
because a package called foo-bar would otherwise vouch for its
templates/bar.yaml with nothing writing it. A removal reaching into
another package is outside the scan's range, and the header records
the case in the tree that this leaves uncovered.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-22 23:53:49 +03:00
Aleksei Sviridkin a75919087e fix(opensearch-operator): bound every call to a cluster's HTTP API (#4341)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

A call to a cluster's HTTP API has no bound of its own.
`NewOsClusterClient` builds its own `http.Transport`, and a fresh
transport carries no dial, TLS-handshake or response-header timeout; the
config sets no `http.Client`, so there is no timeout there either; the
calls the client makes on its own behalf pass `context.Background()`;
and the calls that take a context from the caller are handed one with no
deadline on it, since nothing on the reconcile path sets one. The
`Timeout` field the request types carry is the server-side query
parameter: it bounds how long the server waits before answering, which
helps only once something answers.

Each controller runs a single worker, so one call that never returns
stops every `OpenSearchCluster` from being reconciled, and the finalizer
on the next deletion waits out its whole termination timeout with
nothing in the log to say why. Reported in #4337, measured in #4332.

A fourth patch on the image bounds every request the operator's client
sends, at two levels.

- Every request goes out through a `RoundTripper` that puts a 30s
deadline on it. That is the only level that reaches the request
opensearch-go builds for itself: before a client's first request the
library runs a product check with a `context.Background()` of its own
and decodes the body of the answer, and a client is built per reconcile,
so that check runs on every one. The deadline cannot be released when
`RoundTrip` returns, because the body goes back to the caller still
open, so the cancel rides on the body and is released by `Close`, which
is how `http.Client` applies its own `Timeout`.
- That bounds one attempt. opensearch-go retries a request up to three
more times on 502, 503 and 504, and the transport's deadline starts over
on each of them, so the calls carry a 30s deadline a second time for the
series: the 15 calls in `os_client.go` that request and decode before
returning through `requestContext`, and the 28 that go through the
`doHTTP*` helpers in `performRequest`.
- The transport keeps a 10s dial and a 10s TLS handshake, so an
unreachable node fails in a third of the budget with an error that names
what went wrong.

`ResponseHeaderTimeout` is gone from the transport. It stops counting
the moment the headers arrive, so it is no bound on a body that starts
and then stops, and at 60s it sat above a request deadline that now
covers everything it covered.

This bounds the wait, it does not make an unresponsive cluster answer: a
call that passes its deadline fails the reconcile, which requeues.

The change belongs upstream, and there is no upstream issue for it yet.
The Dockerfile entry carries the drop condition the other entries carry.

### Tests

Seven, in the package the image's own `go test -race` step already runs:

- a call to a cluster that accepts the request and answers nothing
returns instead of running forever;
- a new client, against a cluster whose product check flushes its
headers and then stops, returns instead of running forever;
- a request through the operator's transport, to a server that accepts
the connection and then says nothing, returns instead of running
forever;
- a body that arrives with its headers and then stops is abandoned;
- a call retried on 503 is bounded as a series rather than per attempt;
- a body is still readable after the call that fetched it has returned;
- the transport carries the handshake bound by value, and the dial bound
through the dialer it is built from.

Every bound was checked against the mutation that removes it, and all
seven turn a test red: the transport wrapper dropped from the
constructor, the deadline dropped from `RoundTrip`, the cancel moved off
the body there, the deadline dropped from `performRequest`, one call
site reverted to `context.Background()`, and the dial and handshake
timeouts dropped in turn. Waits are measured from the test goroutine, so
an unbounded call fails the test rather than hanging the run until CI
kills it. The silent-server test holds its accepted connections and
forces a collection while the call is in flight, because an unreachable
`net.Conn` is closed by the collector (`net/fd_posix.go` puts a
finalizer on `netFD`) and that turns the silent server into one that
resets the connection.

The full four-patch sequence applies to a pristine v2.8.0 tree in the
glob order the build relies on, with no offsets and no fuzz; `gofmt` and
`go vet` are clean; and `docker buildx build images/opensearch-operator`
builds the image on linux/arm64 with the `go test -race` step green.

### Screenshots

Not a UI change.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

Walked the trigger map against the diff: one comment block in
`packages/system/opensearch-operator/images/opensearch-operator/Dockerfile`
and one new file in that image's `patches/`. No package added, renamed
or removed, no values key, no schema, no enum, no `hack/` file, no make
target, no namespace and no `ApplicationDefinition`.

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(opensearch-operator): time out calls to a cluster's HTTP API, so one unanswered request no longer stops the operator reconciling
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Added timeout handling for OpenSearch cluster connections, including
connection, TLS-handshake, response-header, and request deadlines.
* Prevented requests from hanging indefinitely when a cluster is
unavailable, unresponsive, or stops returning data.
* Ensured response bodies remain readable after the initial request
completes when applicable.
* Bounded the full request lifecycle, including response-body reads and
retried calls.

* **Tests**
* Added coverage for connection, response, retry, and body-read timeout
scenarios.
  * Enabled race detection for relevant package tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 00:47:46 +04:00
Aleksei Sviridkin 8dfbaa1d66 fix(kubevirt-cdi-operator): pin the vendored CDI release
This package decides which CDI the platform runs, but its update
recipe took the version from GitHub's releases/latest redirect. Any
refresh, including one done for an unrelated reason, could move the
deployed CDI and its seven image tags without anyone choosing a
release, and the diff carried no version line to review. That version
is becoming load-bearing for behaviour, not only for images: logic
that enumerates CDI's DataVolume phases is correct against one
release.

The release is now named in the Makefile, the asset is checked
against a digest before anything is written, and a refresh that fails
partway leaves the vendored manifest untouched. The digest guards
against an asset replaced under its tag; it is not an authenticity
check.

A test target ties the pin to the tree: helm-unittest reads only the
vendored file, so a pin moved without re-running update would
otherwise leave every assertion green against the old release. The
suite also holds the two local patches and the namespace rewrite that
every refresh has to reapply.

The pinned release is the one already vendored, and update reproduces
templates/ byte for byte.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-22 23:45:58 +03:00
IvanHunters ff0b851bbc test(vpn): add default render helm unit tests (#4347)
## What this PR does

vpn was one of the eight user-facing apps in `packages/apps/` called out
in #3631 as shipping with zero helm-unittest coverage. This adds a
`tests/` suite pinning the default render and wires the standard `test:
helm unittest .` target the sibling charts already use, following the
same pattern as #4081, #4074 and #4093.

Ran `helm unittest .` locally against the chart, it passes. `make show`
needs a live cluster connection through `cozyhr`, which was not
available in this environment, so I relied on `helm unittest` for local
verification, same as the referenced sibling PRs.

### Screenshots

Not applicable, test-only change.

### Downstream repositories

Walked the trigger map in docs/agents/contributing.md against the diff:
this only adds a test file and a Makefile target inside an existing
package, it does not add, rename or remove a package, and it does not
touch `values.yaml`, `values.schema.json`, `Chart.yaml` or `README.md`,
so none of the listed triggers apply.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
test(vpn): add Helm unit test coverage for the default render
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Tests**
- Added automated validation for the VPN deployment’s default
configuration, including replica count, container image, service port,
and generated resource naming.

- **Chores**
- Added a convenient command for running the VPN chart’s automated
tests.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-22 23:07:18 +03:00
Aleksei Sviridkin e2464ea788 fix(kubernetes): give System-mode tenant OIDC users a UID
The AuthenticationConfiguration the chart renders mapped username and
groups only, so upstream resolved every System-mode user to an empty
UID and the X-Remote-Uid header the tenant apiserver publishes reached
an aggregated API server carrying nothing.

Map uid to the sub claim. The platform realm enables Keycloak's
UPDATE_EMAIL required action, so the address the username claim
carries is mutable and cannot serve as a stable identifier, while sub
is the account UUID and survives a change of either. CustomConfig
mode is untouched: an operator-supplied configuration is still
written verbatim, uid mapping included or not.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-22 23:04:28 +03:00
Aleksei Sviridkin 4bf95815a8 fix(kubernetes): render the aggregation-layer UID header for tenant OIDC (#3539)
## What this PR does

Tenant clusters with `spec.oidc.mode: System` or `CustomConfig`
authenticate users fine, but a UID does not survive the hop to an
aggregated API server. The tenant kube-apiserver publishes
`requestheader-uid-headers` into the
`extension-apiserver-authentication` ConfigMap only when the flag is
set, and the chart never set it. The aggregation layer still sends
`X-Remote-Uid`, but the extension server trusts only the header names
published in that ConfigMap, so it ignores the one it gets and the call
arrives without a UID. The field documentation even told operators to
add the flag by hand via `controlPlane.apiServer.extraArgs`.

It forwards a UID rather than creating one. ServiceAccount tokens carry
one, and for those the aggregation hop starts working. Users
authenticated by `spec.oidc.mode: System` still have none, because the
generated AuthenticationConfiguration maps `username` and `groups` only;
giving them one means mapping a claim, which changes the identity of
users who have none today, and is tracked separately.

The chart now renders `--requestheader-uid-headers=X-Remote-Uid` on
v1.32 and newer, and adds `--feature-gates=RemoteRequestHeaderUID=true`
on v1.32, where the gate is still alpha and the apiserver rejects the
flag without it. From v1.33 the gate is beta and on by default, so only
the header flag is rendered. v1.31 gets nothing: the flag does not exist
there, and writing it by hand stops the apiserver from starting.
`--emulated-version` in the same passthrough moves that decision,
because the gate resolves through the versioned spec at the emulated
version.

Rendering the gate on every version from v1.32 rather than on v1.32
alone was the alternative, and it would remove the migration step
entirely, because an opt-out would then drop two entries instead of one
and move the entry count by itself. It loses on what it leaves behind: a
redundant gate term on every cluster, which stops the apiserver from
starting on the day the gate is retired from the known list. The design
also prefers rendering over failing for the same reason the fail
branches are narrow, since a cluster stranded on a HelmRelease that no
longer converges is a worse outcome than one that never gets the UID
fix.

The chart reads an operator entry the way the apiserver reads it: header
list elements matched whole and untrimmed and folded as `EqualFold`
folds them, gate names compared exactly, gate values resolved as
`strconv.ParseBool` resolves them, `AllAlpha` and `AllBeta` blankets
honoured, an empty value treated as no entry. On v1.32 an entry that
says nothing about `RemoteRequestHeaderUID` gets a second one carrying
the operator's own terms plus `RemoteRequestHeaderUID=true`. Carrying
those terms into the merged copy is what makes it land correctly whether
the tenant control plane collapses duplicate flags into the last one or
hands both to the apiserver, which merges them in the order it reads
them.

Tests cover the render matrix over mode, version and operator input:
every fail path with its exact message, the ownership and merge paths,
duplicate entries in both orders, blanket gates, empty values, and case
variants on the header name and on the gate value. The latest-version
tenant lane creates its tenant with `spec.oidc.mode: System` and asserts
the flag on the rendered KamajiControlPlane, so the tenant apiserver
starts with the flag in CI. Nothing asserts the UID arriving at an
aggregated apiserver; that needs an extension server inside the tenant
and is not in this change. The layer where the count-keyed reset
actually bites is the tenant Deployment's arguments, and asserting there
needs a live tenant control plane, so it is not in this change either.

The rule this follows is that a configuration which renders and boots
today keeps rendering. Both flags are new, so an `extraArgs` shape
reaching this code was rendering before it existed, and stranding a
running tenant on a HelmRelease that no longer converges is a worse
outcome than one that never gets the fix. A few shapes do stop
rendering, and reaching any of them takes writing
`--requestheader-uid-headers` into the passthrough by hand, attempting
exactly what the chart now does and getting it wrong; in each the
kube-apiserver fails option validation and never starts, so nothing that
was working is lost. `docs/oidc-tenant.md` covers these shapes. One
known limitation is in there and worth naming here: on v1.33 and newer,
adding the documented opt-out is count-neutral in `extraArgs`, and so is
removing it again, and on v1.33 and v1.34 adding or removing
`--emulated-version=1.31`. The pinned Kamaji build rewrites a running
apiserver's arguments only when that count changes and otherwise keeps
every flag that has left the list, so either the header flag stays
beside a now-disabled gate or the disabled gate stays beside the header
flag, and the apiserver refuses to start. Changing the entry count in
the same edit is the mitigation, and the fix belongs to the pin rather
than to this chart, tracked in #3541. The paths that need no operator
edit are clear: turning the feature on for a default tenant moves the
count, and a tenant already carrying the opt-out renders
byte-identically before and after.

Out of scope: `mode: None` still publishes no UID header even though
ServiceAccount tokens carry UIDs. That is pre-existing, and widening the
condition changes the render for every default tenant. Tracked
separately, along with the claim mapping above.

Closes #3377

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change

Checked the trigger map against the diff: no package added or removed,
no schema field, enum or default changed (description text only), no
ApplicationDefinition surface touched, no hack/ layout moved. The
managed-apps reference page picks up the new field description through
release-time README regeneration.

### Release note

```release-note
fix(kubernetes): tenant clusters with `spec.oidc.mode` set to `System` or `CustomConfig` now render `--requestheader-uid-headers=X-Remote-Uid` on the tenant kube-apiserver, plus the `RemoteRequestHeaderUID` feature gate on v1.32, so that an aggregated API server can read the caller's UID out of the request headers. Without that flag the tenant apiserver never publishes `requestheader-uid-headers`, and an aggregated API server ignores the UID header even for callers that have one, ServiceAccount tokens among them. It forwards a UID rather than creating one: users authenticated by `spec.oidc.mode: System` still have none, because the generated AuthenticationConfiguration maps `username` and `groups` only, and that is tracked separately. Adding the flag by hand through `controlPlane.apiServer.extraArgs` is no longer necessary, and an entry of your own is respected: the chart steps aside when yours already carries `X-Remote-Uid` or already enables the gate, merges `RemoteRequestHeaderUID=true` into a copy of a v1.32 `--feature-gates` entry that says nothing about the gate, and renders neither flag when you switch the gate off. The render fails where a `--requestheader-uid-headers` list of your own would stop the kube-apiserver from starting anyway: one that omits `X-Remote-Uid`, one with an empty element, and a list of your own while the gate is off. On v1.33 and newer there is a migration step, and it applies in both directions: the pinned Kamaji build rewrites a running apiserver's arguments only when the number of `extraArgs` entries changes, and otherwise keeps every flag that has left the list. Adding the documented opt-out keeps that number the same, so the header flag already on the Deployment stays beside the now-disabled gate. Removing the opt-out later keeps it the same too, so the disabled gate stays on the Deployment beside the header flag the chart renders again, although your spec no longer shows it. Adding or removing `--emulated-version=1.31` has the same shape both ways on v1.33 and v1.34. The apiserver refuses to start in each case, so change the entry count in the same edit.
```
2026-09-23 00:03:20 +04:00
IvanHunters 48058dcc74 build(opensearch-operator): run the patched tree's tests under -race
The timeout tests drive a listener from one goroutine and inspect what it
accepted from another. The plain run passes an unsynchronised access there, so
nothing in this repository would catch one. The builder image carries cgo and
gcc, and the package images are built natively on x86-64 runners, so the
detector is supported where this runs.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-22 22:50:24 +03:00
IvanHunters 1a2c87635a fix(opensearch-operator): bound every call to a cluster's HTTP API
Upstream gives the operator's calls to a cluster no deadline of any kind. The
transport it builds carries no dial, handshake or response-header timeout, no
http.Client is configured, the calls the client makes on its own behalf pass
context.Background(), and the ones that take a context from the caller are
handed one with no deadline on it. A cluster that accepts a connection and then
says nothing holds such a call for as long as the process lives, and with one
worker per controller that single call stops every OpenSearchCluster from being
reconciled. The deletion that then waits out its whole termination timeout is
the visible end of it.

The bound goes on the RoundTripper, the one level every request passes through,
the ones opensearch-go builds for itself included: before a client's first
request the library runs a product check that carries a context.Background() of
its own and decodes the body of the answer, and a client is built per reconcile,
so that check runs on every one. The deadline cannot be released when RoundTrip
returns, because the body goes back to the caller still open; it rides on the
body and is released by Close, the way http.Client applies its own Timeout.

That bounds one attempt. opensearch-go retries a request up to three more times
on 502, 503 and 504 and the transport's deadline starts over on each, so the
calls carry a deadline a second time for the series: through requestContext
where the call decodes the answer before it returns, and through performRequest
where the still-open body goes back to the caller. The dial and the TLS
handshake keep a shorter bound of their own, so an unreachable node fails in a
third of the budget with an error that names what went wrong.

Each of those bounds has a test that fails when it is removed.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-22 22:50:24 +03:00
Aleksei Sviridkin 08bbed3fb5 feat(talos-log-collector): add talos-log-collector package (#3260)
## What this PR does

Adds `talos-log-collector`, an optional system package: a node-local
Vector
DaemonSet that receives Talos machine and kernel logs and forwards them
to the
tenant VictoriaLogs (vlinsert), alongside the container/audit/event logs
already collected by monitoring-agents.

Talos keeps system and kernel (kmsg) logs inside the OS and does not
write them
to a host file the Fluent Bit `tail` input could read, so these logs are
currently never collected. Vector receives them over the node loopback
(Talos pushes to 127.0.0.1:5170). Talos exposes the two streams through
independent config paths, and both must point at the loopback socket:
`machine.logging.destinations` forwards the JSON-lines service logs, and
a
`KmsgLogConfig` document forwards the kernel (kmsg) stream (including
DRBD
kmsg). The node-side setup for both is in the README.

Design notes:
- The pod stays on the pod network so cluster DNS resolves vlinsert with
no
  extra config. Talos pushes to 127.0.0.1:5170; a `hostPort` bound to
`hostIP: 127.0.0.1` publishes that loopback socket into the pod via
portmap,
  keeping the receiver off the host network (no node-wide pod exposure).
- The receiver binds inside the pod netns, so a NetworkPolicy is
load-bearing:
a CiliumNetworkPolicy restricts ingress to the host entity when Cilium
is
present, otherwise a default-deny-ingress `networking.k8s.io/v1`
fallback
  keeps the socket protected on non-Cilium variants.
- Runs non-root, read-only root filesystem, all capabilities dropped.
- Optional package (opt-in via `bundles.enabledPackages`): inert until
each
Talos node is configured to push logs. Node-side setup is in the README.

Verified end-to-end on a Talos v1.12 dev cluster: service and kernel
logs
(including DRBD kmsg) reach VictoriaLogs; pods run non-root with 0
restarts.

### Release note

```release-note
feat(talos-log-collector): add talos-log-collector package to collect Talos machine and kernel logs into VictoriaLogs
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added an optional node-local Talos log collector (via system bundle
packages) that forwards Talos service and kernel logs over loopback to
the platform log ingestion endpoint.
* Introduced the collector Helm chart (Vector-based) with configurable
log level, loopback TCP receive port, digest-pinned image, default
resources, and routing target.
* Added conditional Cilium network policy support with a safe
deny-by-default fallback.
* **Security**
* Hardened the collector to run non-root with reduced privileges,
read-only filesystem, and dropped Linux capabilities (and disabled
service account token automount).
* **Documentation**
* Documented configuration requirements (including kernel log
enablement), operational behavior, and teardown steps.
* **Build & Release**
  * Extended the build to include building the collector’s image.
* **Tests**
* Added Helm rendering/contract tests covering daemonset, configmap,
service account, and network policy.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-22 23:49:33 +04:00
IvanHunters 47a611cf9e docs(kube-ovn): point the update comment at the test that asks for the fixture
The comment said the bats suite compares against "a fixture this step
tells you to refresh", but the update step never mentions the fixture:
its failure asks you to fix our Dockerfile. The test named "the fixture
tracks the vendored chart version" is the one that asks, and it fires
after a bump lands.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-22 22:47:57 +03:00
Aleksei Sviridkin 4fbef512b4 fix(opensearch-operator): name the component reconciler a stall sits in (#4333)
## What this PR does

The cluster controller runs nine component reconcilers in sequence, and
none of them logs unconditionally when it is entered. The TLS one has a
line above its work, but it sits on the early return for a cluster with
no security section, which
`packages/apps/opensearch/templates/opensearch.yaml` never renders, and
the rolling restart one opens at `V(1)`, off at the verbosity the
operator ships with. So a call that stops returning leaves nothing
behind at all: the operator log ends on whatever ran before it, the
object stops being reconciled, and from the outside that is
indistinguishable from an operator with nothing to do.

That is not hypothetical. Two e2e runs on 2026-09-18 ended that way, and
the evidence is the operator's own log, read out of the crust-gather
host snapshot each failed run uploads, at
`snapshots/opensearch/host/namespaces/cozy-opensearch-operator/v1/pod/opensearch-operator-controller-manager-*/operator-controller-manager/current.log`.

In run 35343282947 the operator reconciled `OpenSearchCluster` 27 times
on a steady cadence, gaps of 10.2s for almost all of them and 26.5s at
the widest. Its last line is at 13:51:33.225Z. helm-controller applied
the CR revision that moves the cluster onto the chart CA at 13:51:35Z.
Nothing was written for the remaining four minutes, the pod stayed
Running and Ready with 0 restarts, and the admin certificate the step
was waiting on was never re-minted. Run 35333289958, a different branch
and a different operator pod, has the same shape: last line
10:56:35.706Z, Apply at 10:56:38Z, widest gap before that 24.2s, then
silence. The securityconfig Job had completed successfully in both, and
the cluster itself was quiet. #4332 tracks it.

A patch on the image now wraps each component so a call still in flight
after a minute reports the component's own name and how long it has been
running, and keeps reporting on the same interval while it is stuck. A
pass through the components takes seconds on a healthy cluster and the
controller requeues itself every 30s, so a minute is well past anything
normal. Nothing at all is written while a component behaves, which is
why this is a watchdog rather than an entry log: logging every entry
would put nine lines per cluster into the log on every reconcile. The
finalizer path is deliberately left alone, because every
`DeleteResources` at this version returns immediately and a watchdog
there could only spawn goroutines for calls that cannot block.

The e2e lane already collects that log and nothing in it changes here.
#4279 put a `podLogs` leg on the operator's own namespace into the
opensearch catch, `namespace: cozy-opensearch-operator`, `selector:
control-plane=controller-manager`, `tail: 2000`, and this branch sits on
top of it. The other `podLogs` leg takes the cluster's own pods in the
test namespace, and cozyreport's per-pod walk keeps only pods that are
not Running or not Ready, which a stalled operator is neither, so before
#4279 these lines reached the job log only through the crust-gather
snapshot.

This is a diagnostic, not the fix. Where the reconcile stops is still
unknown, and the next occurrence should name it. The Dockerfile comment
says to drop the patch once the stall itself is understood and fixed.

The new code lives in `pkg/reconcilers`, already covered by the image's
`go test` step. Five specs hold it: the wrapped result and error pass
through untouched, a call that returns before the threshold is never
reported, a call that returns before the threshold also takes the
watchdog with it, the report names the component and a moving elapsed
while the call is in flight, and the report repeats rather than firing
once. Six mutations each fail the spec that owns them: dropping
`close(done)`, `time.NewTimer(0)`, an empty component name, a frozen
elapsed, a fire-once ticker, and a swallowed result. The full four-patch
sequence applies to a fresh v2.8.0 tree in the alphabetical order the
build relies on, and builds, vets and tests clean there, with five
`-race` runs of the new specs.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff: one Dockerfile comment block
and one new patch file under
`packages/system/opensearch-operator/images/`. No values key, schema,
make target or documented workflow changes.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Added stall monitoring for OpenSearch component reconciliation. When
reconciliation remains in progress beyond the configured threshold, the
operator logs the component name and elapsed time at regular intervals.
- Monitoring remains silent for promptly completed operations and
preserves existing results and errors.

- **Documentation**
- Clarified reconciliation watchdog behavior and documented why certain
resource deletion operations are not monitored.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-22 23:44:30 +04:00
IvanHunters f942a96aac fix(vm-instance-rd): disable helm readiness wait on VMInstance releases
The generated HelmRelease is health-assessed by helm-controller's kstatus
poller, which returns InProgress for any object whose
status.observedGeneration trails metadata.generation: a built-in pre-check
in fluxcd/pkg/runtime/cel that runs before, and cannot be overridden by,
healthCheckExprs. KubeVirt legitimately keeps a VirtualMachine's
observedGeneration behind its generation while a template change is pending
a restart, the VM is stopped, or a live-updatable change never converges
(kubevirt/kubevirt#18700), so a healthy Running VM is read as permanently
InProgress and the release never reaches Ready. Releases still carrying the
pre-1.5.0 remediation.retries:-1 + rollback spec then revert the previous
manifest, dropping just-applied changes such as opened ports.

Disable the readiness wait so the release settles once the VirtualMachine is
applied; VM health is observed on the VirtualMachine itself. This restores
the pre-1.5.0 behaviour under which the VM was never a helm readiness gate.
A helm-unittest suite guards that the annotation stays in place.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-22 22:30:40 +03:00
scooby87 95b8549238 feat(dashboard): brand the token-proxy login page from the console config (#4255)
## What this PR does

Part of #4217. Dashboard branding (title, logo) is only reachable
through the authenticated dashboard/kube-api path, so the pre-auth
surfaces render unbranded. This PR fixes the first of them: the
token-proxy login page — the first screen an unauthenticated user sees
on a non-OIDC cluster — hardcoded `<title>Login</title>` and had no
branding hookup at all.

The branding config lives in the `cozy-dashboard-console-config`
ConfigMap, which the SPA reads only through the authenticated kube-api.
This mounts that same ConfigMap read-only into the token-proxy container
and renders its `titleText`, `logoText` and `logoSvg` on the login page,
falling back to the historical unbranded page when the file is absent or
unreadable so a cluster without branding is unaffected. The SVG is
encoded into an `<img>` data URI rather than inlined, which keeps any
embedded script inert. The OIDC branch is untouched: its login is served
by Keycloak, already branded through the realm login theme.

The unauthorized/access-denied screens rendered by the SPA itself (for a
logged-in user without kube-api RBAC) are handled in a separate PR.

Verified with `go test`, `go vet`, `gofmt`, and `helm unittest`
(branding mount asserted present in the non-OIDC branch and absent in
the OIDC branch).

### Downstream repositories

The diff touches only `packages/system/dashboard` (the in-repo
token-proxy image and its chart). No app is added, renamed or removed,
no `values.schema.json` / enum / default changes, no CRD or namespace
changes, and the `config.json` branding keys are unchanged. Nothing in
the trigger map applies.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(dashboard): the token-proxy login page now renders the deployment's branding (title and logo from `branding.titleText` / `branding.logoText` / `branding.logoSvg`) instead of the hardcoded default, so unauthenticated users on non-OIDC clusters see a branded sign-in screen.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added deployment branding to the login page, including a custom title,
logo, and heading.
* Branding is automatically loaded from the dashboard configuration when
available.
* Login pages continue to display a default unbranded experience when
branding configuration is unavailable or invalid.

* **Tests**
  * Added coverage for branded and unbranded login-page rendering.
* Added validation for branding volume mounts across authentication
modes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-22 22:07:32 +03:00