Commit Graph
1655 Commits
Author SHA1 Message Date
Aleksei Sviridkin d9cbbec5ff fix(build): cross-compile image builders on the build platform
The remaining Go builder stages, plus the kube-ovn, console and
piraeus-server builders, ran on the target platform. A multi-platform
build therefore ran the whole compiler under QEMU: slow, and for Go
unreliable, as the migration-controller Dockerfile already records a
"found bad pointer in Go heap" crash from exactly that. The builders
now run on $BUILDPLATFORM and produce the target architecture from
TARGETARCH.

Two builders only produced the right architecture because they were
emulated. token-proxy ran `go build` with no GOOS/GOARCH, and kube-ovn
calls upstream `make build-go`, which hardcodes GOARCH=amd64 even on an
arm64 build. On a native builder both would have shipped host binaries
inside the other architecture's image, and nothing fails until a node
of that architecture starts the container. token-proxy now passes the
target explicitly; kube-ovn picks build-go or its arm64 twin
build-go-arm from TARGETARCH.

The console builder emits static JS and linstor-server's debian/control
declares every package Architecture: all, so both run natively without
a target.

A bats test requires every stage that compiles to sit on a BUILDPLATFORM
FROM and to read TARGETARCH or TARGETPLATFORM on a RUN or ENV line. A
bare ARG declaration does not count, which is what let token-proxy
through.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-26 15:05:06 +03:00
Aleksei Sviridkin 2c2c0301f7 chore(deps): roll up pending Renovate updates and move to Go 1.27.1 (#4447)
## What this PR does

This replaces the open Renovate PRs with one rollup. Go modules move to
their newest releases that keep the Kubernetes libraries and
controller-runtime where they are, the toolchain and builder images move
to Go 1.27.1, and the first-party code gets the Go 1.27 modernizers.
Tests for the code the bumps touch come first, in their own commits, and
pass on both the old and the new dependencies.

Before the bumps:

- The e2e sandbox image builds again: dl.min.io answers 410 for mc since
MinIO archived it, so mc now comes from its last GitHub release with a
pinned checksum.
- Renovate stops offering tags of the apimachinery fork. They mirror
upstream, so #2999 is a downgrade that drops our patch.
- `make unit-tests` now runs the tests of the nested Go modules
(kubevirt-csi-driver, token-proxy, kubeovn-webhook, api/apps/v1alpha1).
They never ran in CI before.
- New tests: JWT verification in the dashboard token proxy, the kube-ovn
plunger metrics, cert-manager object idempotency in the tenant gateway,
values-file layering in the operator, and the kubevirt CSI driver
wrapper.
- One real fix. The lineage webhook decoded admitted objects with
encoding/json, so an int64 above 2^53 came back rounded. jsonpatch v2.5
compares numbers exactly and would emit a replace that rewrites such a
field in the pod. The webhook now decodes with apimachinery's json.

Held back on purpose, with the reasons in the commit messages:

- Kubernetes libraries stay on v0.35. #3001 needs the apimachinery fork
rebased onto release-1.37 first.
- cel-go v0.29.2 and etcd v3.6.15, the lines k8s.io/apiserver v0.35 is
built and tested with.
- grpc v1.83.2. v1.84.0 is affected by GO-2026-6443 and no v1.84 patch
release carries the fix yet.
- kubevirt.io/api and the CDI API stay on what upstream csi-driver
builds against. From CDI v1.60 it no longer compiles.
- api/apps/v1alpha1 keeps go 1.26.4, so consumers of the published types
are not forced onto Go 1.27.
- go fix runs without embedlit (Go 1.27-only syntax, breaks backports to
release branches on Go 1.26) and omitzero (would make metadata, spec and
status required in the published OpenAPI schemas), and one errorsastype
rewrite that did not compile is left out.

govulncheck reports nothing reachable in any of the five modules,
standard library included.

Supersedes #4311, #4310, #4309, #3452, #3993, #3927, #3902, #3805,
#3451, #3415, #3414, #3413, #3302, #4050 and #3363. #3001 stays open.

Not checked locally: the images that build upstream projects (kube-ovn,
kamaji, cilium, linstor-csi and others) with the Go 1.27 builder. The
image build jobs of this PR cover them.

Closes #4487.

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
chore(deps): move the toolchain and builder images to Go 1.27.1 and raise Go dependencies to their latest compatible releases, closing the published advisories; the lineage webhook no longer rewrites int64 fields above 2^53 in admitted objects.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added a restore resource policy option for Velero strategies.
  * Added a test target to run Go tests across tracked modules.
* **Bug Fixes**
* Updated the end-to-end test environment to download and verify the
MinIO client from its current release location.
* Added coverage for token validation, webhook patching, controller
behavior, and reconciliation.
* **Maintenance**
* Updated the Go toolchain, dependencies, and container image versions.
  * Refreshed pinned workflow actions and base image digests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-26 10:17:29 +04:00
Aleksei Sviridkin a285a5ef36 feat(kubernetes): add a proxmox substrate for managed clusters and worker pools (#4088)
## What this PR does

**Series behind #3481** — three separable pull requests:

1. #4087 — the Cluster API providers (capmox + in-cluster IPAM)
2. **#4088 — the `substrate` switch** in `kubernetes` and
`kubernetes-nodes`
3. #4089 — the Proxmox CSI driver and cloud-controller-manager

#4087 and #4088 share no files and can be reviewed in parallel. #4089
builds on this one.

---

#4087 adds the Cluster API providers; this PR teaches the two
managed-cluster
charts to point at them.

### The shape

A single value, `substrate: kubevirt | proxmox`, on both charts — not a
parallel
set of charts. The control plane is Kamaji either way; what changes is
the
infrastructure half:

| | `kubevirt` (default) | `proxmox` |
|---|---|---|
| `Cluster.infrastructureRef` | `KubevirtCluster` | `ProxmoxCluster` |
| worker template | `KubevirtMachineTemplate` | `ProxmoxMachineTemplate`
|
| apiserver Service | `ClusterIP` | `LoadBalancer` |
| tenant CCM/CSI | in-cluster (`kccm`, kubevirt-csi) | rendered off; the
Proxmox pair is #4089 |

The KubeVirt path renders byte-for-byte what it rendered before. Checked
by
rendering `kubernetes-nodes` with default values at `main` and at this
head and
diffing the output, not only by the snapshots — the snapshots cover
`nodegroup.yaml` alone, and an earlier revision of this branch changed
the
reconcile Job while they stayed green. The parent chart's default render
differs
only in the random fields of `talos-secrets`, which differ between two
renders of
`main` as well.

`osImage` (#4171) is refused on the proxmox substrate. Only half of it
could be
honoured there — `factory.version` reaches the machineconfig's installer
image
while the worker disk keeps coming from the Proxmox template — and a
field that
half-works is worse than one that says no. `talos.version` and
`talos.schematicID` still set the installer target.

### Three things that only show up once workers are off-cluster

**The apiserver endpoint.** A KubeVirt worker runs inside this cluster
and
reaches the tenant apiserver by ClusterIP. A Proxmox worker does not: it
lives on
the hypervisor's L2 and cannot route to `10.96.0.0/16`. The machine
provisions
fine, `VMProvisioned=True`, `providerID` set — and then never joins,
with no CSR
to show for it. On the proxmox substrate the Kamaji Service is therefore
a
`LoadBalancer`, and the reconcile Job reads
`.status.loadBalancer.ingress[0].ip` instead of `.spec.clusterIP`.

**And DNS, for the same reason.** The worker machineconfig carried
`nameservers: ${COREDNS_IP}` — the management cluster's CoreDNS
ClusterIP,
equally unreachable. The proxmox path takes its own resolvers
(`proxmox.dnsServers`) and the chart refuses to render without them,
rather than
writing an address that cannot answer.

**`externalManagedControlPlane` and the managed-by annotation.**
capmox's
webhook nil-derefs on a `ProxmoxCluster` whose control-plane endpoint is
empty
unless `spec.externalManagedControlPlane` is set, which is correct here
— Kamaji
owns the endpoint. Separately, the `cluster.x-k8s.io/managed-by: kamaji`
annotation that the KubeVirt path carries must **not** be set on the
Proxmox
object: with it, CAPI never sets an ownerRef and capmox stops at
`Waiting for Cluster Controller to set OwnerRef`, leaving the address
pool
uncreated and every worker waiting on infrastructure.

### Linked clones by default

`proxmox.full: false` — worker disks are linked clones of the template
rather
than full copies. Measured on a ZFS-backed store: **8 KiB per worker
instead of
10.2 GiB**. Operators who want independent disks set `full: true`; the
value is
plumbed straight through to `ProxmoxMachineTemplate`.

`proxmox.storage` is a full-clone parameter and is refused with a linked
clone:
a linked clone always lives on the template's storage, and both Proxmox
(`parameter 'storage' not allowed for linked clones`) and capmox's own
CRD
(`Must set full=true when specifying storage`) reject the pair. The
chart says
so at render time, naming both values, instead of failing at apply time
with an
error about a `ProxmoxMachineTemplate` nobody wrote.

### Sizing

`ProxmoxMachineTemplate` takes `numCores` and `memoryMiB`, not
Kubernetes
quantities, so the pool's `resources.cpu`/`memory` are converted
(millicores →
cores, bytes → MiB, both rounded up). The rounding is pinned in
`tests/substrate_branches_test.yaml` with `1500m` and `8G`, where
rounding up and
down disagree; `tests/substrate_proxmox_sizing_test.yaml` covers the
refusal to
size a pool that gives neither.

### Known upstream behaviour this surfaces

`TalosConfigTemplate.spec` is immutable by schema, so the reconcile Job
cannot
rewrite an endpoint it has already published — `kubectl apply` fails
with
`TalosConfigTemplate.Spec is immutable`, and the template has to be
deleted for
the Job to recreate it. A fresh install never meets it; it appears when
the
apiserver address changes on an existing cluster. Handling it changes
the
KubeVirt path too, so it is not in this PR: it is a separate change on
`proxmox/tct-replace`, with its own upgrade note.

### Evidence

```
$ KUBECONFIG=<tenant> kubectl get nodes -o wide
kubernetes-px-md0-jmzds-ndnl6  Ready  <none>  75s  v1.35.6  10.0.0.160  Talos (v1.13.6)

$ KUBECONFIG=<tenant> kubectl get pods -A
cozy-cilium  cilium-…            Running
kube-system  konnectivity-agent  Running
kube-system  coredns-…           Running
```

No management-cluster address remains in the rendered
`TalosConfigTemplate`.

### Testing

`make unit-tests` — exit 0. `apps/kubernetes` 26 suites / 314 tests;
`apps/kubernetes-nodes` 29 suites / 160 tests / 12 snapshots, all
unchanged on
the default path. Each of the ten commits passes both suites on its own.

Every substrate branch is asserted on both sides, and every assert goes
red on
the mutation it exists for: inverting the `SVC_IP` condition, inverting
the
`nameservers` condition, `ceil`→`floor` in the sizing, removing the
proxmox
gate from each of the kccm and csi templates (the two RBAC bindings
included),
pinning the retention lookup back to `KubevirtMachineTemplate`, widening
the
IPv6 class of the `dnsServers` validation, dropping the parent chart's
`dnsServers` validation, the storage-with-linked-clone refusal, and the
`osImage` refusal. Negated `containsDocument` asserts carry `any: true`.

`hack/talos-reconcile-heredoc_test.bats` additionally runs the rendered
heredoc
through a real shell: a `dnsServers` entry with a substitution is
refused and
named, an IPv6-shaped one with a substitution is refused too, and a real
address survives the shell unchanged.

### Screenshots

Not applicable — no UI change.

### Downstream repositories

I walked the trigger map in `docs/agents/contributing.md` against this
diff, file by file. What it matches is below. **No box is ticked**,
because every follow-up it points at depends on a decision that is not
mine to make: #3481 asks whether Proxmox becomes a supported platform at
all, and writing its documentation or its Terraform schema before that
is answered would be work built on an assumption. I have not filed
speculative PRs or issues in those repositories for the same reason. If
the direction is accepted, I open them; a human should decide which.

**cozystack/terraform-provider-cozystack — matches, no follow-up yet.**

- `packages/apps/kubernetes/values.schema.json` and
`packages/apps/kubernetes-nodes/values.schema.json` both gain fields:
`substrate`, and a `proxmox` object (`allowedNodes`, `dnsServers`,
`ipv4Config`, and on the pool side `templateTags`, `network`, `storage`,
`full`). The provider hand-maintains a schema, a model and an
expand/flatten pair per Kind, with no codegen from these files.
- `values.yaml` gains defaults for those fields (`substrate: kubevirt`,
`proxmox.full: false`, `prefix: 24`). The provider restates defaults it
models and sends its own, so a stale one wins silently rather than
showing as a diff.

**Everything else: no match.** No app added, renamed or removed (website
app lists); no version enum, `kind`, `plural`, `release.prefix` or
output Secret/Service name changed; `packages/core/platform/values.yaml`
untouched; the `*-rd` files are regenerated, not renamed (ccp); no
`hack/` change; no node prerequisites (talm, ansible); no telemetry
metric; no commit-convention change (community).

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(kubernetes): add a `substrate` value to the `kubernetes` and `kubernetes-nodes` applications, selecting whether worker VMs run on KubeVirt in this cluster (`kubevirt`, the default and unchanged) or on an external Proxmox VE cluster through capmox (`proxmox`). On the Proxmox substrate the tenant apiserver is published through a LoadBalancer and worker disks default to linked clones.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for choosing KubeVirt or Proxmox for Kubernetes clusters
and node pools.
* Added Proxmox configuration for nodes, networking, storage, DNS,
templates, cloning, and IPv4 allocation.
* Proxmox deployments now use the appropriate cluster, machine,
networking, and control-plane resources.

* **Validation**
* Added checks for supported substrates, required Proxmox settings,
sizing, and incompatible options.

* **Documentation & Tests**
* Documented the new settings and expanded coverage for rendering,
validation, cloning, sizing, and template replacement.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-25 22:09:23 +04:00
Aleksei Sviridkin 09d02fdc6c docs(apps): correct the shared resources and preset paragraphs (#4394)
## What this PR does

Corrects two sentences of the boilerplate shared by the app READMEs,
both raised in the review of #3936.

- "When left empty, the preset defined in `resourcesPreset` is applied"
describes an all-or-nothing rule, but
`cozy-lib.resources.defaultingSanitize` merges per key: every resource
`resources` leaves unset is taken from the preset. Twelve READMEs
carried the sentence.
- The legacy flat names were called aliases of their "1:1 instance-type
equivalents", which reads as the `c1` series listed just before it.
`nano` through `small` equal the `t1` sizes, and `medium` equals
`c1.small` rather than `c1.medium`, the trap
`docs/operations/resource-presets.md` already warns about. Ten READMEs
carried the sentence.

Only the hand-written section changes. `make generate` for all twelve
packages leaves the diff at exactly those 22 lines.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff: README prose only, no values,
schema, default or package change. The website regenerates the app
reference pages from these READMEs on release, so nothing to open there.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Clarified how resource settings work across several app READMEs: any
unset resource field now inherits its value from the preset, rather than
requiring the entire resource block to be empty.
* Updated legacy preset name guidance to explain that deprecated flat
names keep their original sizes, including `nano`–`small` matching `t1`
sizes and `medium` matching `c1.small`.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-25 20:17:41 +04:00
Aleksei Sviridkin b16edec3f3 fix(tcp-balancer): rename backends and pin HAProxy so the chart starts on 3.3+ (#4366)
## What this PR does

`haproxy:latest` now resolves to 3.3 and newer. Since 3.3 a backend
named like a frontend is a fatal config error, so every tcp-balancer pod
fails at config parse (#1966). We hit it on Cozystack 1.6.1 with
`haproxy:3.4.4` behind the tag.

- backends get a `_backend` suffix and every `default_backend` follows;
the rename is the one from #2321 by @officialasishkumar
- the image is `haproxy:{{ .Chart.AppVersion }}` with `appVersion:
3.4.4`, the current 3.4 LTS tag
- the whitelist `acl`, `accept` and `reject` lines of each frontend
render inside one guard
- `tests/` holds helm-unittest suites, and the package `Makefile` gets
the `test` target the sweep looks for
- the `whitelistHTTP` description says it needs a non-empty `whitelist`

Two inputs change behaviour on upgrade. On `main` both fail at config
parse with `no such ACL : 'whitelist'` on every HAProxy version, because
the `acl` line sits inside the `with` and `accept if whitelist` does
not:

- `whitelistHTTP: true` with an empty `whitelist` now fails the render
with `whitelistHTTP is enabled but whitelist is empty`. Its HelmRelease
upgrade fails and the release stays on its old revision, which is
already crash-looping.
- `kubernetesAndTalos` with an empty `whitelist` now starts, and ports
6443 and 50000 accept any source. With `external: true` the Service is a
LoadBalancer and carries both ports. `whitelist` is the only switch
those frontends have, so an empty list means no filtering.

Every other input renders the same config apart from the backend names.
The image and the `checksum/config` annotation change, so the Deployment
rolls, and for releases already crash-looping on `latest` the rollout is
the fix.

Validation: renders of `main` and of this branch for the default,
whitelist and empty-whitelist inputs were checked with `haproxy -c` on
2.9.7 and 3.4.4. `helm unittest .` passes 9/9, and removing the `reject`
or `accept` line from any single frontend turns a test red.

Fixes #1966

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff: chart templates, `Chart.yaml`,
the package `Makefile`, `tests/`, and a description-only change in
`values.yaml` with its generated carriers. No field is added, removed or
renamed and no default changes, so nothing in the map fires.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(tcp-balancer): rename HAProxy backends and pin the image to the chart appVersion so the balancer starts on HAProxy 3.3+. whitelistHTTP with an empty whitelist now fails the render; kubernetesAndTalos with an empty whitelist leaves ports 6443 and 50000 unfiltered.
```
2026-09-25 19:29:39 +04:00
Marian Koreniuk 8722462066 fix(kubernetes-nodes): refuse osImage on proxmox instead of honouring half of it
#4171 gave a worker pool an osImage, choosing where its system disk comes from:
a CDI clone of a shared golden image, or an HTTP import from an Image Factory.
A Proxmox worker's disk comes from neither. capmox clones a VM template that
already carries Talos, and nothing in this chart can change what that template
holds.

Half of the field would still appear to work, which is why it is refused rather
than ignored. osImage.factory.version reaches the machineconfig's installer
image and changes the reconcile Job, so a pool could set it, watch the Job
roll, and end up running one Talos while booting another: v1.12.6 moves the
installer to .../installer/<schematic>:v1.12.6 and leaves the
ProxmoxMachineTemplate untouched. osImage.builtin has no meaning here at all.

Nothing is lost by refusing it: the installer target on this substrate is
settable through talos.version and talos.schematicID, which do reach it.

The branches suite gains the refusal case. The substrate parameter's description said changing the value "rolls the
pool". The chart refuses the change on a live pool, and the description says
so; it reaches the README, the values schema and the CRD.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:15 +02:00
Marian Koreniuk 4bc6621a5c test(kubernetes,kubernetes-nodes): assert both sides of every substrate branch
Every `if substrate` in both charts is asserted on both sides, so an inverted
condition cannot pass:

- the reconcile Job reads SVC_IP from the LoadBalancer status on proxmox and
  from the ClusterIP on kubevirt;
- the machineconfig carries the pool's nameservers on proxmox and none on
  kubevirt;
- the proxmox gate on each kubevirt-only template in the parent chart, the two
  RBAC bindings included, so a proxmox cluster renders no binding to a Role,
  ClusterRole or ServiceAccount it does not render;
- the sizing conversion at 1500m and 8G, where ceil and floor disagree
  (2 cores and 7630 MiB against 1 and 7629);
- the old-template retention loop keeps a ProxmoxMachineTemplate a MachineSet
  still references;
- the proxmox worker monitor carries no Pod threshold while the kubevirt one
  does;
- a live cluster and a live pool refuse a substrate switch while a first install
  still renders;
- both charts refuse a dnsServers entry that is not an address, the parent
  chart included, where the value reaches ProxmoxCluster.spec.dnsServers.

The negations carry `any: true`: without it a negated containsDocument passes
whether or not the document is in the render.

hack/talos-reconcile-heredoc_test.bats gains the dnsServers cases: the render
refuses a command substitution and names the entry; an entry with a colon in it
is refused too, which pins the IPv6 branch of the validation and not only the
IPv4 one; and a real address survives the Job's shell unchanged. The heredoc is
extracted by shape — `cat <<EOF` at the start of a line or right after `=$(` /
`="$(` — so the same case keeps working whichever way the Job writes it.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:15 +02:00
Marian Koreniuk 517baf945d fix(kubernetes,kubernetes-nodes): validate dnsServers, refuse a substrate switch, drop the Pod threshold on proxmox
Three faults in the proxmox substrate, none of which the render refused.

1. proxmox.dnsServers reached the reconcile Job's shell unescaped.

The Job applies the worker TalosConfigTemplate through an unquoted heredoc, so
every byte of its body is subject to parameter expansion and command
substitution at Job runtime. The INVARIANT comment above that block says each
tenant-controlled value must be shell-escaped or pattern-validated; the new
nameserver list was neither. `quote` there is YAML quoting.

Rendered with `$(id -u)` and a backtick pair in dnsServers, extracted the way
hack/talos-reconcile-heredoc_test.bats does and run under sh, the shell executed
both and put their output in the worker's resolver list. The Job's pod holds the
parent cluster's Talos token, cluster id, cluster secret and bootstrap token in
its environment, and its Role can read the Talos CA and patch the parent
KamajiControlPlane.

Each entry is now validated as an IPv4 or IPv6 address at render time, in both
charts. Validation rather than escaping because Talos takes addresses here: an
address cannot carry a metacharacter, so the property holds by construction, and
the ordinary mistake of writing a hostname is caught in the same move. IPv6 is
matched on its character set rather than its full grammar — what matters is which
bytes may reach the shell, and the apiserver rejects a malformed address anyway.

2. Nothing stopped a live cluster or pool from being switched between substrates.

`substrate` had no immutability of any kind, and the `## @immutable` marker this
adds is dashboard-enforced only, so both charts now read the live object.

Switching a cluster removes the KubevirtCluster, the cloud-controller-manager,
the kubevirt-csi controller and the in-tenant -csi HelmRelease with all their
RBAC from the render. Flux then uninstalls the CSI driver behind every PVC the
tenant has. CAPI does not prevent it — its Cluster webhook forbids removing
spec.infrastructureRef but admits a change of kind.

Switching a pool rolls every worker to the other hypervisor and drops the
machine template the draining MachineSet still references, because the retention
loop looks up only the kind the new substrate uses. A pool whose substrate
differs from its parent's was not rejected either, though the README says the
two must match.

Both guards are inert offline, like assertNoForeignPool and assertParentVersion
beside them: with no apiserver the lookup is empty and a first install renders.

3. The worker WorkloadMonitor reported a proxmox pool as broken.

The controller lists Pods matching the monitor's selector and sets
Operational=false while fewer than minReplicas are ready. On kubevirt those Pods
are the virt-launcher Pods. A proxmox worker is a VM on another machine with no
Pod here, so the count is always zero and any threshold pinned the pool — and
through the lineage labels its whole application — to WorkloadsReady=False while
every node was healthy. The monitor still renders for the dashboard and the
lineage; what it no longer carries on proxmox is a threshold counted in Pods.

Two smaller things. The wait-loop log line is written per substrate, so a
default kubevirt render of kubernetes-nodes is byte-identical to main and no
existing pool gets a new reconcile Job on upgrade. And the comment claiming a
rename of $kubevirtmachinetemplate would roll every live worker VM is replaced:
the hash is taken over the rendered template, not the variable name, so renaming
it leaves the render byte-identical.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:15 +02:00
Marian Koreniuk 33e1de52d2 test(kubernetes-nodes): pin the linked-clone default and three silent guards
Nothing failed when `full` was wrong: the pool came up, the workers joined, and
each one quietly cost the whole template disk instead of the increment — 10.2
GiB against 8 KiB on a ZFS store. A default whose only symptom is disk usage is
exactly the kind that drifts, so both it and its override are now asserted.

The bridge and dnsServers guards were two of the guards on this path with no
coverage. Both fire on absence, so the suite omits each and the tests supply the
other one back; declaring either at suite level would make it impossible to take
away, which is why the sizing guard already lives alone.

The storage guard is the third: `storage` with a linked clone is refused and the
message names both values, while `storage` with `full: true` reaches the
ProxmoxMachineTemplate.

Each assert goes red on the edit it exists for: flipping the default reds the
clone suite, deleting a guard reds the guard suite.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:15 +02:00
Marian Koreniuk c5996c1d0f fix(kubernetes,kubernetes-nodes): give Proxmox workers an endpoint they can reach
A worker on the proxmox substrate is a VM on the hypervisor network, not a Pod
in this cluster. It was handed two addresses only an in-cluster consumer can
route to: the apiserver as a ClusterIP, and the management cluster's CoreDNS
ClusterIP as its resolver. The VM booted, the Machine reported Ready with a
providerID, the disk was where it should be — and no CSR ever arrived. Nothing
in that picture points at an address.

The control-plane chart now publishes the tenant apiserver as a LoadBalancer on
the proxmox path, putting the endpoint on the network the workers share, and
the pool's reconcile Job reads the Service's LoadBalancer address instead of its
ClusterIP for the host entry the workers dial. The nameservers follow the same
reasoning: proxmox pools carry their own resolvers rather than pointing at a
CoreDNS they cannot reach, and the chart refuses to render a pool that has not
named any.

KubeVirt keeps the ClusterIP on both counts. Its workers are in this cluster,
where a ClusterIP is the right answer and a LoadBalancer address would be an
unnecessary one.

Verified end to end on a real tenant cluster: the node joins, its CSR is
approved, and cilium, konnectivity and coredns come up in the tenant.

One operational note this surfaced, not caused by it: TalosConfigTemplate.spec
is immutable, so the reconcile Job cannot rewrite an endpoint that has already
been published — the template has to be deleted for the Job to recreate it.
That matters only when an existing cluster's apiserver address changes.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:15 +02:00
Marian Koreniuk d78af928fc feat(kubernetes-nodes): default Proxmox worker disks to linked clones
A full clone on a ZFS pool configured with `sparse 0` reserves the whole
volsize the moment the VM is created: 10.2G for a worker whose Talos image
occupies 337M. A linked clone writes only its own increment — measured at 8K
for the same worker on the same host — because it shares the template's base
snapshot until it diverges.

The cost is that the clone holds that snapshot, so the template cannot be
rotated while any linked clone still references it. That is bounded rather
than permanent: consolidation detaches each worker 24 hours after creation, so
template rotation is blocked for at most a day.

Consolidation is not a chart concern, but two of its constraints are worth
recording where the flag is set, because both were discovered by testing on a
live host rather than from the docs. A linked clone can only be detached by
moving its disks to a *different* storage — Proxmox refuses a same-storage
move outright. And both disks must move: leaving efidisk0 behind keeps the
template pinned just as effectively as the system disk would.

Set full: true for a pool that must be independent of the template from the
first second.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:14 +02:00
Marian Koreniuk 135eb6bac4 fix(kubernetes): make the Proxmox cluster object reconcilable
Two corrections found by building a real tenant cluster on Proxmox rather than
by rendering the chart. Each produced a symptom that pointed away from its
cause.

capmox skips control-plane endpoint validation only when
spec.externalManagedControlPlane is set. Kamaji owns the endpoint here, so the
field was semantically true all along and merely absent — and on an absent
controlPlaneEndpoint the webhook does not return a validation error, it panics
(internal/webhook/proxmoxcluster_webhook.go:105). The install failed with a
nil-pointer stack trace out of an admission webhook, which reads like a broken
provider rather than a missing field.

The cluster.x-k8s.io/managed-by annotation now renders only on the KubeVirt
path. It tells Cluster API the infrastructure object is owned by something
outside CAPI, so the Cluster controller never sets an ownerRef on it — and
capmox's reconciler exits at "Waiting for Cluster Controller to set OwnerRef on
ProxmoxCluster" without ever creating the InClusterIPPool its workers draw
addresses from. Harmless on KubevirtCluster, which nothing reconciles;
load-bearing on ProxmoxCluster, which capmox must.

With both in place the object reaches Ready=True, the pool is created, and the
Cluster reaches Provisioned.

The proxmox suite's ProxmoxCluster case asserts both: the field is set and
the annotation is absent.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:46:14 +02:00
Marian Koreniuk db839d1198 feat(kubernetes): add a proxmox substrate for managed clusters
Companion to the same switch in kubernetes-nodes. The control plane is Kamaji
on either substrate, so this changes exactly one object — the one
Cluster.infrastructureRef points at — plus the two tenant addons that exist
only because the workers are KubeVirt VMs living in this cluster.

A KubevirtCluster needs no spec: its workers are in the same cluster and reach
the apiserver over the pod network. A ProxmoxCluster addresses an external
hypervisor, so it carries what capmox cannot infer — which Proxmox nodes may
host VMs, which addresses they may take, and who resolves names for them.
controlPlaneEndpoint stays absent on both: the cluster.x-k8s.io/managed-by
annotation hands that to Kamaji.

The address pool is required rather than defaulted. capmox has no DHCP mode; it
allocates from an InClusterIPPool built out of ProxmoxCluster.spec.ipv4Config,
so an unset pool is not a smaller configuration but a ProxmoxCluster that never
reaches Ready and worker Machines that wait on it forever. Refusing at render
time turns that into a message instead of a silence.

The KubeVirt cloud-controller-manager, its cloud-config and the kubevirt-csi
driver are gated off on the proxmox path. They reach the workers'
VirtualMachines and DataVolumes in this cluster; on Proxmox those objects do
not exist, and leaving them rendered would put controllers in the tenant
reconciling nothing. Their Proxmox equivalents are separate packages and land
separately.

The default path is unchanged: rendering it before and after this commit
differs only in the per-render random bootstrap tokens, and all 19 existing
suites pass untouched.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:29:09 +02:00
Marian Koreniuk 85f576756b feat(kubernetes-nodes): add a proxmox substrate for worker pools
Worker pools are already Talos machines created by Cluster API; only the
substrate underneath them is fixed to KubeVirt. This adds `substrate:
kubevirt|proxmox` and swaps exactly two things: the machine template object
and every reference to it. Autoscaler capacity annotations, kubelet
reservations, MachineDeployment, MachineHealthCheck, WorkloadMonitor and the
talos-reconcile Job are substrate-agnostic and rendered once, which is why
this is a branch in the existing chart rather than a second copy of it.

The default path is unchanged down to the bytes: the render snapshot and all
15 existing suites pass untouched, and the hash-named template keeps its
variable name so no live worker VM rolls on upgrade.

Five things the Proxmox path must do differently, each learned from a
failure that pointed somewhere else:

- Skip capmox's guest-agent and cloud-init-status probes. Talos ships
  neither, and left on they leave the VM booted and the Machine stuck on
  "error waiting for agent: the operation has timed out". This is the same
  class of problem `virtualMachineBootstrapCheck.checkStrategy: none` solves
  for CAPK's SSH probe; capmox's CRD names TalOS in the skipCloudInitStatus
  description.

- Clone a template carrying the nocloud image, not openstack. capmox injects
  bootstrap data as a nocloud ISO, and an openstack image ignores it exactly
  as silently as a nocloud image ignores the configdrive the KubeVirt path
  depends on.

- Pass templateTags through verbatim. capmox compares the template's tag set
  with slices.Equal after sorting, so a selector listing a subset matches
  nothing and reports "found 0 VM templates with tags ..." while the tags are
  plainly present.

- Require resources.cpu and resources.memory. Proxmox has no
  VirtualMachineClusterInstancetype, so the lookup that normally supplies
  effective CPU and memory has nothing to read — and those two numbers feed
  the kubelet reservations and the autoscaler capacity annotations. A pool
  that cannot state them would be scheduled against invented capacity, so it
  is refused at render time. KubeVirt-only knobs (gpus, logSerialConsole,
  podCpuLimit/Request) are refused too rather than accepted and ignored.

- Refuse proxmox.storage unless full is true. A target storage is a full-clone
  parameter: a linked clone always lives on the template's storage, and both
  Proxmox ("parameter 'storage' not allowed for linked clones") and capmox's
  own CRD ("Must set full=true when specifying storage") reject the pair.
  Left to them, a pool with storage set and a linked clone fails at apply time
  with an error about a ProxmoxMachineTemplate the operator never wrote;
  failing the render names the two values instead.

The Talos image is not a chart input on this path: capmox clones a template
that already exists on the hypervisor, so building and rotating that template
is an operator responsibility with no equivalent on the KubeVirt side.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-09-25 16:29:08 +02:00
scooby87 03e2da5805 feat(postgres,mariadb)!: drop plaintext user passwords (#4078)
## What this PR does

Removes the plaintext `users[].password` field from the `apps/postgres`
and `apps/mariadb` charts (and the mariadb `root` account), so
managed-database passwords can no longer be pinned in tenant values or
committed to git. Passwords are always chart-generated into the
`<release>-credentials` Secret and preserved across reconciles via
`lookup`, matching the model `redis`/`valkey` already use. This closes
the "User passwords" item of the portal upstream epic
(aenix-org/cozyportal#1060).

The generated password never lands in the init-script Secret or the SQL
it runs: the postgres init-job reads each user's password from the
mounted `<release>-credentials` Secret at run time and applies it
through a psql variable (`ALTER ROLE … WITH PASSWORD :'pw'`), so it does
not linger in the init-script manifest a tenant can read. It does live
in cleartext `stringData` in the `<release>-credentials` Secret itself,
which is a chart manifest and therefore kept in Helm release history for
`MaxHistory` revisions like any chart-generated secret — that residue is
exactly why rotation cannot be built on the chart (see below). A
`password` or a `passwordRotation` key left over in a tenant's values is
accepted-but-inert and draws an admission warning, so an operator
editing it is told it has no effect rather than getting a silent 200.

Backup/restore into a copy keeps the target's chart-managed credentials
working: the CNPG driver clears `bootstrap.enabled` once recovery is
healthy so the init-job re-applies the generated passwords onto the
recovered roles, and the MariaDB driver excludes `mysql.global_priv`
from the dump (`ignoreGlobalPriv`) for backups captured from this
release onward, so restoring one of those into a copy does not clobber
the target's chart-managed users or `root`. Excluding the grant table
has a flip side, and it applies to every MariaDB restore, not only a
to-copy one: only accounts declared through `User` CRs (the chart's
`users` map) are captured, so an account created out of band directly in
MySQL is in no backup and a restore brings back its data and `mysql.db`
grants but not the account itself. This is also a backup-side fix only:
a restore from a backup taken **before** this change still replays the
source's `mysql.global_priv` and overwrites the target's users and
`root`, and `root` then needs an out-of-band reset (the operator applies
`rootPasswordSecretKeyRef` only at datadir bootstrap, so there is no
in-band path). See `docs/operations/backup-classes.md` for the
convergence window, the out-of-band-account limitation, and this
old-backup caveat.

Password **rotation** is deliberately **not** in this PR. A
chart-rendered credentials Secret keeps its cleartext in Helm release
history for `MaxHistory` revisions, so a rotation built on it cannot
revoke a leaked password; rotation is being reworked as a controller
that owns the Secret — design proposal in cozystack/community#72.

Migration is seamless: existing releases keep their current passwords on
upgrade (the value already lives in `<release>-credentials`, which
`lookup` re-reads and preserves); a `password` still present in a
tenant's values is ignored, so no existing HelmRelease fails validation.

Verification: `helm unittest` passes for both charts (postgres 87,
mariadb 126, including render-time username guards that reject any name
k8s would later reject as an invalid Secret key or User resource name);
`go build ./...`, `go vet`, the backup-controller tests and the
apps-apiserver tests are green; `make generate` was run with the
CI-pinned `cozyvalues-gen` v1.6.0 so `values.schema.json`, the READMEs,
the API types, and the `cozyrds` ApplicationDefinitions are all in sync
(the branch was rebased onto `main`, which reconciled cleanly with the
read-replica autoscaling feature that landed there).

### Screenshots

N/A — no UI change.

### Downstream repositories

- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up: affected — `content/en/docs/next/cozystack-api/go-types.md`
is hand-written and still shows `Password:` in the Go examples, and the
managed-app reference pages regenerate from these chart READMEs, so both
document `users[name].password` until refreshed (not yet filed).
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: affected — the provider hand-maintains the postgres/mariadb
schemas and per-field defaults; this PR removes `users[].password`. A
follow-up issue/PR is needed there (not yet filed).

`terraform-provider-cozystack` and `website` are reached by this diff;
both boxes are left unticked because the follow-ups are not yet filed.
Same-repo future work this change suggests: the controller-owned
password rotation (design in cozystack/community#72), and extending the
"passwords only in the Secret" model to the other engines that still
accept plaintext (`clickhouse`, `mongodb`, `nats`, `opensearch`,
`rabbitmq`, `vpn`).

### Release note

```release-note
feat(postgres,mariadb)!: passwords for managed PostgreSQL and MariaDB users can no longer be set via `users[].password` — they are always auto-generated and stored only in the `<release>-credentials` Secret, never rendered into the init-script or the SQL it runs. Existing releases keep their current passwords on upgrade; a `password` (or a `passwordRotation`) left in values is ignored with an admission warning. Password rotation is deferred to a follow-up controller design (cozystack/community#72). MariaDB backups now exclude the `mysql.global_priv` grant table (`ignoreGlobalPriv`), so a restore recreates only accounts declared through the chart's `users` map — an account created out of band directly in MySQL is not captured and must be recreated after a restore.
```
2026-09-25 15:49:53 +03:00
Alexey Artamonov 237f2e2bbd fix(postgres): say how to get past a refused extension clause
Before the charset guard, an extension value was spliced into CREATE
EXTENSION whole, so a release can carry a clause after the name, such
as earthdistance CASCADE, and that is the usual way to pull in a
dependency. The guard refuses such a value on upgrade too, and keeping
it that way is right, but the tenant needs to know what to write
instead. The reject message now says: drop the clause, which changes
nothing for an extension that is already installed because the script
never drops one and CREATE EXTENSION IF NOT EXISTS leaves it alone, and
list the dependencies first, since extensions are created in list
order.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-25 00:03:22 +03:00
Alexey Artamonov 70dcd9daed fix(backups): let a restored Postgres app converge its credentials
A restore sets bootstrap.enabled on the target app, which skips the
init-job, so the recovered roles keep the source's password hashes while
the target's credentials Secret holds generated ones and every
application login fails. Once recovery converges, the driver records
RecoveryConverged and then clears bootstrap.enabled, so the next
HelmRelease reconcile runs the init-job and applies the passwords. Ready
carries RestoreCompletedCredentialsPending and an Event says the
convergence is still pending. A clear that keeps failing is retried
under BootstrapDisablePending within a fixed grace window after
convergence and then fails the restore. bootstrap.enabled is serialized
without omitempty so the patch sends false instead of dropping the key.

Running the init-job on a restored cluster exposes its delete pass to a
catalog the spec does not describe: the driver sets spec.databases to
the Backup's snapshot, but recovery replays WAL past it, so a database
created after the Backup comes back still marked as managed and would
be dropped with its data. The first run after a restore, told apart by
the per-RestoreJob bootstrap.newServerName that the cluster has not yet
recorded, releases such a database and its roles from management
instead, and the run records the value once every pass has succeeded.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-24 18:11:07 +03:00
Alexey Artamonov 9f857f64e1 feat(mariadb)!: generate every user password, root included
users[].password is gone from the chart, its schema and the API type.
Every password, root included, is generated into <release>-credentials
and kept across reconciles by lookup, and a password left in values is
ignored. An upgrade that finds no credentials Secret fails instead of
minting new passwords, because mariadb-operator applies root only at
datadir bootstrap and would advertise a root it never set. The Secret
carries k8s.mariadb.com/watch so the operator applies a changed password
without waiting for its requeue interval.

A username or database name becomes a Secret key and a resource name, so
render-time guards reject a name outside the DNS-1123 rule or past its
limit: 80 for a username, as the User CRD allows, and 64 for a database,
as the server allows. Names that differ only in "_" and "-" collide on
one resource; that is rejected on install only, since an existing
release already applied the overwrite. A user in both admin and readonly
of a database keeps a single ALL grant under the unchanged Grant name,
because renaming a Grant makes the operator revoke the live privilege. A
role member past 128 characters, the server's account-name limit, and a
database name of 65-80 characters, which the Database CRD accepts but
the server does not, are rejected on install and on upgrade only when
their Grant or Database does not exist yet, so a release that already
applied one keeps upgrading. User CR names and keys are quoted so a
username that reads as a YAML scalar stays a string.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-24 18:10:57 +03:00
Alexey Artamonov 2b5877eff9 feat(postgres)!: generate user passwords and read them at runtime
users[].password is gone from the chart, its schema and the API type.
Every application-user password is generated into <release>-credentials
and kept across reconciles by lookup, so an upgraded release keeps the
password it already has and a password left in values is ignored. The
init-job mounts that Secret and applies each password with ALTER ROLE
and a psql variable, so no password is rendered into the init-script
Secret that Helm keeps in release history. While bootstrap is enabled
the Secret is marked credentials-pending, and NOTES say the generated
passwords apply once the init-job runs.

Names reach superuser SQL and the init-script shell, so render-time
guards reject a username, database or extension name outside
[-._a-zA-Z0-9] or past the length PostgreSQL silently truncates: 63
bytes for a role or extension, 54 for a database, whose <db>_readonly
role would otherwise truncate and fail the next run. A username over 63
bytes that the credentials Secret already carries still upgrades.
users.postgres, the CNPG superuser, is rejected in every bootstrap
state. Changing a database's key in values drops the database, so the
reject message and the README give the rename that keeps the data: a
platform administrator renames it and its roles as the CNPG superuser
before the key changes.

The init-job cannot hang its Helm hook: the readiness wait ends on
elapsed time below activeDeadlineSeconds and the Job never retries, so a
purged restore target fails with a diagnostic. The delete-users pass
reads managed users from pg_shdescription instead of parsing the
version-dependent \du+ layout, which on PostgreSQL 16 and later matched
nothing and left a removed user able to log in; it compares names as
CREATE ROLE stores them and reassigns a removed user's objects in every
database before dropping it. A database without roles renders, and
extension names are quoted for hyphenated names and folded to lowercase
so they resolve as they did unquoted. A value that carries its own
surrounding quotes, the only way a hyphenated extension rendered before,
has that pair removed and keeps its case, so such a release still
upgrades. The README reads a password with jq, since jsonpath splits a
dotted key.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-24 18:10:57 +03:00
Aleksei Sviridkin 7a0aab2bdc refactor: apply the Go 1.27 modernizers
Run go fix over every first-party module: interface{} becomes any,
hand-written loops become slices, maps, min/max and range-over-int,
errors.As becomes errors.AsType, pointer helpers become new(expr) and
are inlined at their call sites, and so on. Every rewrite keeps
behaviour.

Three rewrites stay out. embedlit flattens embedded-field literals
such as ObjectMeta, syntax only Go 1.27 accepts, which would break
backports to release branches still on Go 1.26. omitzero drops
omitempty from struct fields, and openapi-gen reads omitempty as
optional, so metadata, spec and status would turn required in the
published schemas. And errorsastype rewrote an errors.As whose
target, apierrors.APIStatus, does not implement error, which
errors.AsType rejects at compile time.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 17:23:18 +03:00
Aleksei Sviridkin db6c20f841 chore(deps): raise Go modules to their latest compatible releases
Several pinned modules carry published advisories: golang.org/x/crypto,
net, text and mod, grpc, the opentelemetry SDK and exporters, velero,
the etcd client and cel-go. Rather than take only the minimum fixed
versions, every module outside the Kubernetes lockstep set moves to
its newest release whose own requirements keep k8s.io/* and
controller-runtime where they are and fit Go 1.27.1. Where the newest
release asks for more, the module stops at the last one that does not:
cert-manager v1.19.6, source-controller/api v1.8.5, source-watcher/api
v2.1.1, fluxcd meta v1.26.0; helm-controller/api, fluxcd kustomize,
gateway-api and external-snapshotter stay put.

A few modules stop below their newest release on purpose. cel-go and
the etcd modules stay on the lines k8s.io/apiserver v0.35 was built
and tested with (cel-go v0.29.2, the version Kubernetes itself ships,
and etcd v3.6.15), since the apiserver is their only consumer here.
grpc stays on v1.83.2: v1.84.0 predates the fix for GO-2026-6443, a
server panic the CSI driver's gRPC server can reach, and no v1.84
patch release carries it yet. kubevirt.io/api and the CDI API stay on
the versions upstream csi-driver builds against: from CDI v1.60
StorageSpec.Resources changes type and upstream no longer compiles.

velero v1.18 adds RestoreSpec.ResourcePolicy, which the strategy CRD
embeds, so the CRD is regenerated. The backup controller only consumes
the velero v1 types; the new Queued and ReadyToStart backup phases
fall through to its requeue path like any other in-progress phase.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 17:23:18 +03:00
Aleksei Sviridkin ec7387109e build(go): move the toolchain and builder images to Go 1.27.1
Go 1.26.4 carries standard-library advisories in net/http, net/url,
crypto/tls, encoding/asn1 and html/template that govulncheck reports
as reachable from the first-party binaries. The shipped stdlib comes
from the builder image, so the builder tags move together with the
go directive; the digests are the multi-arch indexes that golang:1.27,
1.27-alpine and 1.27-bookworm resolve to at 1.27.1.

api/apps/v1alpha1 keeps go 1.26.4. It is a published types-only
module, no binary here links its stdlib, and raising its directive
would force every consumer onto Go 1.27.

linstor-csi v1.11.2 pins grpc v1.81.0, which no longer builds on
Go 1.27: golang.org/x/net drops http2.TrailerPrefix there. Its image
now raises grpc to v1.83.2 before building, as the kamaji image does
for its advisories; the upstream unit tests pass on it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 17:23:18 +03:00
renovate[bot] a7dfbe6f8c chore(deps): update container image & action digests
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:04 +03:00
Aleksei Sviridkin 73ac282562 chore(deps): advance kubevirt csi-driver to upstream f5de10d
The previous pin was the pre-merge head of the "check VMI status when
detaching" change; upstream main carries the same change with
identical file contents, so moving onto main keeps it. On top of it
come NodeStage rescanning udev, Unpublish using CleanupMountPoint as
other CSI drivers do, and a runc security bump.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:03 +03:00
Aleksei Sviridkin 2be5d4625c test(kubernetes): cover the kubevirt CSI driver wrapper
Only the rate-limit flags were tested. Pin what a CSI spec or client
bump could change under the wrapper: RPCs the driver does not serve
answer Unimplemented, the RWX DataVolume carries the requested size
under spec.storage.resources.requests, and the NFS export, access-mode
and endpoint-selector helpers.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:03 +03:00
Aleksei Sviridkin a92e89e969 docs(kubernetes): describe the OIDC UID header from v1.33
The tenant OIDC guide, the version field description and the e2e
helper comment still described a --feature-gates term rendered on
v1.32. Left as they were, they would tell an operator that the chart
switches that alpha gate on, which it no longer does. The guide now
says what a v1.32 tenant has to write itself to get the UID, and which
--emulated-version values fail the render, so an operator who meets
that refusal finds it explained. The breaking-changes note in the
README said the chart refuses the mixed gate spelling while OIDC is
on; with that refusal gone it would promise a render error that no
longer comes.

The version description returns to its earlier wording in values.yaml
and in the update-versions.sh template, because that template rewrites
the line on every version bump and would otherwise bring the old text
back. make generate carries it into the schema, README, API types and
ApplicationDefinition.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 15:10:53 +03:00
Aleksei Sviridkin 0078727620 fix(kubernetes): render the OIDC UID header only from v1.33
RemoteRequestHeaderUID is alpha and off on v1.32, so the chart had to
enable an alpha gate on v1.32 tenants with OIDC on to make
--requestheader-uid-headers start. From v1.33 the gate is beta and on
by default. No release has shipped the v1.32 path, so moving the floor
to an effective v1.33 now removes a behaviour no released chart had:
the chart renders no --feature-gates term of its own, and below the
floor it renders no UID flag and runs none of the UID checks, as it
already did on v1.31. The refusal of a list that ends in a bare flag
still applies there, because it guards the --authentication-config the
chart appends on every version.

The refusal of a kube gate spelling mixed between bare and
kube:-prefixed terms goes too. Once the chart writes no gate term, the
spelling of the operator's terms no longer concerns anything it
renders, and the chart already passes other gate values the apiserver
refuses, unknown names and unparseable values, through to it.

--emulated-version is still read, because emulation moves the
effective version below the floor. Its kube version is now read from
the numbers at the start of the value, with the element split at its
first equals sign as toVersionMap splits it, so v1.32, 1.32.0 and
1.32-rc all read as 1.32. With OIDC on, an --emulated-version entry
without quotes whose kube version has a non-zero patch component fails
the render and names the entry: the apiserver refuses such a value and
would otherwise only crash-loop.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 15:10:53 +03:00
Aleksei Sviridkin 7d7f363cf2 docs(kubernetes): list the Kamaji update's extraArgs breaks
The Kamaji update changes what an existing tenant's extraArgs does, and
the package README is where an operator reads upgrade breaks. Record
there that an entry naming a flag the control plane owns stops taking
effect and, except for the two flags the chart moves into a field,
stops the release from rendering; which plugin names the admission
field accepts; and that repeated feature-gate spellings and kube
emulated versions, which used to collapse to the last entry, now reach
the apiserver and stop it from starting.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 14:59:13 +03:00
Aleksei Sviridkin 4e9f83ac0d fix(kubernetes)!: move or refuse extraArgs flags the control plane owns
From 26.9.4-edge the tenant control plane sets a list of kube-apiserver
flags itself and drops any extraArgs entry naming one before the
apiserver sees the list; 26.3.6-edge let such an entry win. An existing
tenant that set, for example, its own admission plugins through
extraArgs would lose them on the first reconcile after the upgrade with
nothing to say so, and a flag written with its value in a separate
entry leaves that value behind as a positional argument the apiserver
refuses.

Two of those flags have a KamajiControlPlane field the control plane
renders them from. An --enable-admission-plugins=value or
--kubelet-preferred-address-types=value entry is moved into that field
while the field is empty, and left out of the rendered extraArgs, so
the tenant keeps its setting across the upgrade without an edit. The
move is computed from the values on every render; the last entry wins,
as it did while the control plane kept one entry per flag name, and a
repeated address type is dropped because that field is a set. The
post-upgrade Job that patches extraArgs onto the KamajiControlPlane
reads the same list.

Every other entry naming an owned flag is refused at render time in
every OIDC mode, since the chart cannot stop the drop and the refusal
is where the operator learns of it. So are the two moved flags beside a
non-empty field, with a value that names nothing or is quoted, and in
the two-entry spelling. The list mirrors the Kamaji builder for the
shape of control plane this chart renders, and the extraArgs
description names the same flags.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 14:59:12 +03:00
Aleksei Sviridkin 0f9b8dc16c feat(kubernetes): expose admission controllers and kubelet address types
The KamajiControlPlane carries two lists the tenant control plane
renders into kube-apiserver flags: spec.admissionControllers into
--enable-admission-plugins and spec.kubelet.preferredAddressTypes into
--kubelet-preferred-address-types. The chart rendered neither from the
tenant values; the address types were a fixed list. Pass both through.

Admission controllers are rendered only when the list is non-empty, so
a tenant that sets none keeps the control plane's own default set, and
a list replaces that default rather than adding to it. An empty or
absent list of address types renders the list the chart rendered
before, so a tenant that sets nothing renders exactly as it did.

The names are declared as enums copied from the KamajiControlPlane
CRD at the provider pin, so the application API refuses a misspelled
name where the tenant edits it instead of at apply time. Both fields
are optional.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 14:59:11 +03:00
Aleksei Sviridkin 88264216b7 docs(kubernetes): restate the OIDC UID flags for the merged reading
The migration note told operators to move the extraArgs entry count in
the same edit so the running control plane would rewrite its apiserver
arguments; the control plane compares a hash of the list now, so any
change already forces the rebuild and no second edit is owed in either
direction. The paragraphs describing a value that survives only as the
last entry of its flag, and an empty value reaching the apiserver as a
bare flag, describe a build that is no longer shipped.

What replaces them is the accumulation each flag actually performs: a
uid-headers list the apiserver validates as the union of every entry,
feature gates settled across entries with the last term winning and one
spelling of the kube component throughout, and an emulated version that
two entries cannot both name.

One contract the earlier text sold is narrowed to what the chart
keeps. The render checks run only where the chart renders something, so
the three readings it refuses to guess at switch them off as well, and a
shape that would otherwise be refused passes through to a control plane
that does not start. A flag and value written as two entries is a
narrower case, and the page says which checks hold next to one.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 14:59:01 +03:00
Aleksei Sviridkin 5b370d533f fix(kubernetes): read every extraArgs entry for the OIDC UID flags
The tenant control plane hands the kube-apiserver every entry of a flag,
so reading only the last one misses an opinion the apiserver acts on. An
earlier --feature-gates entry turning RemoteRequestHeaderUID off, or an
earlier --emulated-version naming kube, was previously dropped before the
apiserver saw it; now it decides, and rendering the uid header flag
against the later entry alone stops the control plane from starting.

Repeats of --requestheader-uid-headers append, so the chart adds its own
element beside a list that omits X-Remote-Uid instead of refusing to
render: the union satisfies the check the refusal existed for.
--feature-gates terms accumulate into one component bucket, so the chart
carries only its own term rather than a copy of the operator's, and
naming the kube component both bare and kube:-prefixed is refused
outright, which nothing the chart renders can repair.

A flag and its value written as two entries is read the same way the
apiserver reads it. pflag pairs them, so the value belongs to an entry
this code reads separately: the uid header flag counts as present, which
is what the gate-disabled check needs, while --feature-gates and
--emulated-version decide what is rendered and are not guessed at. What
holds whatever such a value hides still applies next to one: an empty
header element is refused, an explicit opt-out beside a header entry is
refused unless the hidden value is a gate term that could undo it, and
X-Remote-Uid is added beside an operator's own header list, which it can
only help. The chart's other additions wait for a reading.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 14:58:12 +03:00
myasnikovdaniil 9bec21461d feat(kubernetes): boot tenant worker disks from a shared golden Talos image (CDI clone) (#4171)
## What this PR does

Tenant worker boots from talos OS disk image that CDI streams over HTTP
from image factory, and it does that once per worker. Same bytes N
times, and N chances for a fetch to fail. In closed or rate-limited
environment that is wrong shape.

This PR adds opt-in catalog and per-pool selector that reads from it.

`packages/system/kubernetes-worker-image` imports one golden DataVolume
into `cozy-public` per `(schematicID, version)`, named
`talos-worker-<schematicID>-<version>`. Pool opts in with
`osImage.builtin` on its own `KubernetesNodes`, and its `disk-system`
DataVolume then carries `source.pvc` pointing at that golden, so CDI
does storage layer CSI clone instead of HTTP import. One import per
flavor, shared by every worker of every tenant.

Catalog is disabled by default, enable by adding
`cozystack.kubernetes-worker-image` to `bundles.enabledPackages`. Its
knobs (`imageFactoryURL`, `storageClass`, `storage`, `images`) are set
through `spec.components.platform.values.kubernetesWorkerImage` on
cozystack-platform Package. That needed optional package helper to learn
components argument, which `cozystack.platform.package.optional.default`
did not have, so every optional package registered through that wrapper
could name variant and nothing else. Thirteen other packages get same
path for free.

`osImage` has two mutually exclusive forms:

- `osImage.builtin` boots pool workers by CDI-cloning golden from
catalog. Clone is storage layer CSI copy, so OS image never crosses pod
network and factory answers once for whole fleet instead of once per
worker.
- `osImage.factory` keeps HTTP import and lets pool pin own
`imageFactoryURL`, `schematicID` and `version`. This is bring-your-own
path for operator running own factory or caching mirror, and it is what
pool gets when `osImage` is omitted.

In-guest talos installer written by reconcile job follows same
resolution, so pool that pins release boots it and upgrades into it
instead of being flipped back to cluster default halfway.

This is pool half of what #3950 asks for. That issue wants platform
level defaults for two channels, worker OS image and in-guest registry
mirrors. Here image channel gets per-pool source and shared cache,
platform default plumbing is left to the issue.

## Existing workers do not roll

With `osImage` omitted rendered `KubevirtMachineTemplate` is byte
identical to what chart produced before. KMT is named by hash of its own
content, so any drift would re-image every worker VM in fleet on next
upgrade. Stored render snapshot is untouched by this PR, that is the
evidence and not the claim. Editing `osImage` on existing pool re-images
that pool, like changing any other image reference.

## What fails at render

Clone that would silently degrade into something slower or wrong is
refused instead of rendered.

Golden absent, or its import in terminal `Failed` phase. Pool
`storageClass` different from golden one, because CDI cant CSI-clone
across storage classes and falls back to host assisted copy over pod
network, which is exactly the transfer `osImage.builtin` removes. Empty
class on either side resolves against cluster default class before
comparing, so pool at `storageClass: ""` is checked too. `diskSize`
below golden storage, because clone target cannot be smaller than
source. Talos release the matrix lists as unsupported, whether pool
reached it through `talos.version` or through `osImage.*` override.
Matrix passes talos minor it does not list, so what it promises is that
known-bad pairing is refused, not that every pairing is known.
`schematicID` that is not plain lowercase alphanumeric, or version that
is not `vMAJOR.MINOR.PATCH`, both are interpolated into DataVolume name
and image URL while `values.schema.json` types them as bare strings.

Golden that is only mid-import is not an error. CDI holds worker disk
until source populates, and failing there would abort whole pool
HelmRelease for length of a download. Same reasoning covers pool that
already cloned: absent or `Failed` golden fails render only for pool
still waiting for its first disk, otherwise reconcile of healthy pool
would abort over source it no longer reads.

Golden that declares no StorageClass is read from its PVC, not from
whatever default is in force today, because those differ once cluster
default moves. Cluster with two default classes resolves same way
Kubernetes does, newest wins. Pool that already cloned is identified by
annotation naming exactly that pool, not by disk name prefix, because
pool `md0-gpu` produces disks carrying `md0` prefix, and it reads its
own class from its own bound PVC, so admin moving
`storageclass.kubernetes.io/is-default-class` does not fail release of
pool whose disks are all on right class.

That resolved class is then written onto worker disk. Reading one class
and binding another is how pool at empty `storageClass` passed the
comparison and still took host assisted copy on its next worker, so
guard and disk now come from one value. HTTP path leaves empty class to
cluster as before, which is what keeps its render byte identical.

Catalog side refuses duplicate `(schematicID, version)` entries, entries
whose schematic or version cannot produce name pool can reconstruct, and
edit to `storage`, `storageClass` or source URL of entry whose golden
already exists. Last one is not patchable in place, so without guard it
lands as rejected patch on next upgrade.

Goldens carry `helm.sh/resource-policy: keep`, so dropping entry stops
managing golden instead of deleting it. Missing golden aborts whole
Kubernetes HelmRelease of every tenant pinned to it, not just the disk,
and each cloned worker disk annotates golden it came from, so "does
anything still clone this" is answerable before manual delete.

Three drift guards live in
`hack/check-worker-image-catalog-defaults.bats`, because values that
have to agree sit in files nobody edits together. One-sided talos bump
trips all three.

## Upgrade

Three pool values that never produced working image rendered before and
hard-fail now, so pool carrying any of them flips its HelmRelease to
`InstallFailed` on upgrade and not on next edit: `talos.version` in
`vMAJOR.MINOR` form, `schematicID` that is not plain lowercase
alphanumeric, and empty `talos.imageFactoryURL`, which used to render
schemeless `/image/...` source URL. None of the three could boot worker,
so this is failing earlier and not a regression, but it surfaces on pool
nobody touched and fix in each case is to correct the value. CR carrying
such value is admitted by API with value intact, and later unrelated
edit reaches `.spec.values` of HelmRelease while it stays
`InstallFailed`, so edit is taken everywhere and applied nowhere.

## Why e2e does not exercise the clone path

Clone is storage layer copy, so it needs StorageClass whose volumes are
reachable from any node. Merge-gating e2e has none: those lanes run on
talos containers, which share runner kernel and cannot load DRBD, so
node-local `local` is only class there. Nothing would arrange the
co-location such a class requires either, golden in `cozy-public` has no
consumer and binds where CDI importer landed, while worker disk binds
where its VM is scheduled.

So the suites keep importing over HTTP with `osImage` omitted, which is
also the render the no-roll guarantee rests on. This PR touches no e2e
harness file: `hack/` differs from main only by the two guards this
feature owns, one tying catalog default golden to pool chart talos and
StorageClass defaults, one on in-guest installer heredoc.

Switching suites over does not work on that substrate. Golden asks for
`replicated`, lane does not have it, CDI resolves no StorageProfile, and
catalog HelmRelease times out on DataVolume stuck in `ImportInProgress`.
Passing lane own class would fix the class and leave the co-location.

Nightly and tag build run same suites on QEMU with DRBD and
`replicated`, which is the substrate clone path was verified on. That is
where e2e coverage belongs, as its own change with that lane storage
headroom measured rather than assumed.

Worth being straight about what this leaves, because it bears on whether
the feature earns its place. The e2e argument was never the strong one:
golden fetches from image factory exactly once, and #3513 established
that node-join stall tracks host `kvm_amd` `vgif` flag rather than
registry egress. What clone genuinely removes is the fanout, N workers
streaming over pod network becomes N storage layer copies. That is
property of storage layer, so it is worth having where DRBD is and worth
nothing where it is not. This ships as production capability with the
constraint stated on the field, not as CI improvement.

## Testing

On this tree:

- full `make unit-tests` plus `make test-controllers`: 95 charts, 2520
helm-unittest cases, 1710 bats cases, `go test ./pkg/...` and
`./internal/...`, exit 0
- `pre-commit run --all-files`: exit 0, root `make generate` and per-app
generate both clean, so no generated-file drift
- render snapshot unchanged, which is what pins byte identical render
for pool that does not adopt `osImage`
- mutation pass over render guards, each term deleted or inverted in
turn and suite required to notice it by name. Restoring presence gate on
catalog StorageClass comparison reddens exactly one case, inverting its
`ne` to `eq` reddens five. Reverting worker disk to omit its resolved
class reddens three cases that pin where builtin disk binds, and forcing
that class non-empty on every path reddens HTTP-path case and all three
render snapshots, with snapshot diff naming the changed KMT hash

`E2E Tests` passed on run 35831830568. That run was on the previous
head, 4fd8e7ba, and this head differs from it only by one test comment
and the commit message.

## Verified on a live cluster

Throwaway three-node talos stand (OCI, talos v1.13.6, LINSTOR/ZFS,
`replicated` StorageClass) installed from this branch `packages/` tree.
This covers what neither `helm template` nor unit suites can answer. It
predates the change that writes resolved StorageClass onto worker disk,
and that stand ran with explicit `replicated` on both sides, which is
the value that change now renders.

- The catalog wires up from `bundles.enabledPackages` alone:
`PackageSource cozystack.kubernetes-worker-image` reconciled, its
HelmRelease installed, and the golden `talos-worker-<schematic>-v1.13.6`
imported to `Succeeded` in 108s (6Gi, RWX, `replicated`).
- A pool on `osImage.builtin` clones it on the storage layer. Both
worker DataVolumes went `CSICloneInProgress` then `Succeeded` in 43s
with `Running=False reason=Completed message="Clone Complete"`, carrying
`source.pvc` and no `source.http`, and no importer or upload Pod
appeared in the tenant namespace.
- The clone yields a bootable talos root disk. Both worker VMs reached
`Running`/`Ready`, joined the tenant cluster and went `Ready` about 66s
after creation, and the tenant cilium, coredns, csi, ingress-nginx and
metrics-server addons all installed on them.
- Access modes match across the hop: golden `ReadWriteMany` 6Gi to
worker `ReadWriteMany` 20Gi on the same `replicated` class, grown by
CDI.
- The written annotations are correct in practice: each worker disk
carries `kubernetes-worker-image.cozystack.io/golden` naming the golden
and `.../pool` naming its pool.
- helm-controller converges on a builtin pool: forcing a reconcile left
the content-hashed KMT name unchanged and the VMI and DataVolume UIDs
identical, so a reconcile does not roll the pool.
- Cross-namespace clone RBAC is in place as shipped: `create datavolumes
--subresource=source -n cozy-public` resolves to `yes` for a tenant
ServiceAccount through the `cdi-clone-dv` RoleBinding, and CDI runs with
`cloneStrategyOverride: csi-clone`. This PR adds no RBAC of its own.

### Screenshots

Not applicable, no UI change.

### Downstream repositories

Trigger map points at two repositories.

Provider models every pool field by hand, so `osImage` needs schema,
model and expand/flatten pair there. Already tracked as
cozystack/terraform-provider-cozystack#39, filed when the field was
still called `image`.

`content/en/docs/next/operations/configuration/platform-package.md` is a
hand-written table of `spec.components.platform.values.*` keys and
nothing generates it, so `kubernetesWorkerImage` needs a row there.
Filed as cozystack/website#707.

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/issues/707
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
https://github.com/cozystack/terraform-provider-cozystack/issues/39
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(kubernetes): tenant worker pools can boot their system disk from a shared golden talos image instead of importing it from the Image Factory once per worker. Enable the opt-in `cozystack.kubernetes-worker-image` package via `bundles.enabledPackages` to publish one golden DataVolume per `(schematicID, version)` in `cozy-public`, then set `osImage.builtin` on a `KubernetesNodes` pool to have CDI clone it on the storage layer. A pool that omits `osImage` keeps importing over HTTP and renders exactly as before. The clone path needs the pool and the golden on the same StorageClass, and one whose volumes are reachable from any node, in practice a replicated/DRBD class.
```
2026-09-24 12:10:07 +05:00
Timofei Larkin 146e06e75e feat(api): take Application name limits from the application's own schema (#4367)
## What this PR does

The aggregated API server capped two kinds' names by knowing what their
charts render. A `Kubernetes` cluster name was held to 32 characters
because every worker pool is a `KubernetesNodes` release named
`kubernetes-nodes-<name>-<pool>`, and the default `md0` pool has to fit
the 53-character Helm release name limit. A `Kafka` name was held to 44
because the KRaft controller pod `kafka-<name>-c-<hash>-<id>` has to fit
the 63-character DNS-1123 label limit. The arithmetic is right, but it
is the charts' knowledge living in the platform: the next application
with a sub-resource budget needs another `if r.kindName == ...` branch,
and an application served from outside this repository — the whole point
of `ApplicationDefinition` — cannot get one at all.

An application now declares the limits on its own name in its own
schema, and the server applies whatever it finds there for whatever kind
the definition describes. The chart's `values.yaml` carries a `## @name`
annotation, `cozyvalues-gen` v1.7.0 publishes it at the root of
`values.schema.json` as `x-cozystack-name`, and `hack/update-crd.sh`
carries it into the `ApplicationDefinition`:

```yaml
## @name {string} - Cluster name. Every worker pool is a separate KubernetesNodes release named `kubernetes-nodes-<name>-<pool>`, so the name must leave room for the default `md0` pool inside the 53-character Helm release name limit.
## @maxLength 32
```

```json
"x-cozystack-name": {
  "type": "string",
  "description": "Cluster name. Every worker pool is a separate KubernetesNodes release named `kubernetes-nodes-<name>-<pool>`, so the name must leave room for the default `md0` pool inside the 53-character Helm release name limit.",
  "maxLength": 32
}
```

`maxLength`, `minLength` and `pattern` are honoured; `pattern` is
unanchored, as JSON Schema specifies. A declaration only tightens:
`prefix + name` must still fit the 53-character Helm limit. A rejected
name is answered with the chart's own description, or with "limit
declared by the <Kind> application" when the chart gives none.

The schema is untrusted input — definitions from other repositories are
written by hand — so a declaration that cannot be enforced as written
(not an object, a type error, an unknown keyword such as a misspelled
`maxLenght`, `maxLength` below 1, a negative `minLength`, `minLength`
above `maxLength` or above the Helm budget, a pattern that is not RE2)
refuses every create of that kind with `configuration error: the <Kind>
ApplicationDefinition declares an invalid x-cozystack-name: …`, and logs
the same at start-up. Dropping the declaration would admit names the
chart cannot render; applying it would reject every name and blame the
user. Other kinds are unaffected, and existing objects of the affected
kind can still be read, updated and deleted.

An upgrade brings `cozystack-api` up before `kubernetes-rd` and
`kafka-rd` re-render (`kubernetes-rd` depends on `cozystack-engine`), so
the new server can briefly read a definition that predates its
declaration. For exactly that case — a definition that declares nothing
— the two previous caps stay in `legacyNameCaps` as a floor, to be
removed once no supported upgrade starts from such a definition. That is
the only per-kind knowledge left in the name checks.

Also in this PR:

- The `pre-commit` job pins `cozyvalues-gen` v1.7.0, the first release
that understands `@name`
([cozystack/cozyvalues-gen#33](https://github.com/cozystack/cozyvalues-gen/pull/33)).
- The API review gate now diffs the root `x-cozystack-name` of every
definition as `metadata.name`, so a chart that lowers its cap or adds a
pattern is reported like any other tightened field. On this PR it
reports the Kubernetes and Kafka caps as added; their effective limits
are unchanged.
- The migration 56 comment that pointed at the removed
`maxKubernetesClusterName` now points at the chart's declaration.

Not in scope, because none of it is a static limit a chart can declare:
the `Tenant` name pattern in `pkg/apis/apps/validation`, the computed
tenant-namespace length check (it depends on the parent namespace at
request time), the tenant quota hierarchy, and the Phase 2 removed-field
warning for `Kubernetes`.

### How it is tested

`TestNameCapComesFromTheShippedSchema` reads the shipped `kubernetes`
and `kafka` ApplicationDefinitions off disk, builds storage through
`NewREST` as the server does, and checks the cap and the chart's
explanation; it then rewrites `maxLength` inside the shipped schema and
checks the limit moves with it, which a cap held in code would not.
`TestInvalidNameSchemaFailsClosed` covers each malformed declaration
through the same path, `TestLegacyNameCapsCoverUndeclaredDefinitions`
pins the upgrade floor and that a declaration replaces it, and
`TestClassifyNameDeclaration` covers the gate.

### Downstream repositories

Two repositories are affected and neither follow-up is filed yet, so no
box is ticked.

- `cozystack/website` — `content/en/docs/next/development.md` documents
the `values.yaml` annotations and needs `@name`;
`content/en/docs/next/cozystack-api/application-definitions.md`
describes `openAPISchema` and needs `x-cozystack-name`.
- `cozystack/external-apps-example` — it pins no `cozyvalues-gen`
version and regenerates against whatever is current; nothing in it
declares a name constraint, so it is unaffected until it chooses to.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(api): Application name limits now come from the application's own schema instead of being hardcoded per kind in the aggregated API server. A chart declares them with a `## @name` annotation in `values.yaml`, which cozyvalues-gen v1.7.0 publishes as `x-cozystack-name`; the API server enforces `maxLength`, `minLength` and `pattern` on `metadata.name` for any kind, answers a rejected name with the chart's own explanation, and refuses creates of a kind whose declaration it cannot enforce with a configuration error. The Kubernetes (32) and Kafka (44) name caps are unchanged in effect and are now declared by their charts.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Applications can declare minimum and maximum name lengths and
formatting patterns.
* Kubernetes cluster names are limited to 32 characters, and Kafka
cluster names to 44 characters, to meet generated resource naming
limits.
* Names that violate declared constraints are rejected with an
explanatory message. Tighter name constraints are also reported as
breaking changes.
* Declared maximum lengths cannot exceed the platform’s existing Helm
release-name budget.

* **Documentation**
* Added guidance on declaring and enforcing chart-specific resource-name
limits.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-24 10:29:27 +04:00
Myasnikov Daniil bc8c54d6e6 feat(kubernetes): boot tenant worker disks from a shared golden Talos image
Every tenant worker VM imports the same Talos raw image from the Image Factory
over HTTP, once per worker: a 20-worker cluster fetches the identical ~4.15 GiB
artifact 20 times, and each fetch is a live dependency on the Factory at exactly
the moment a node joins.

packages/system/kubernetes-worker-image is a new opt-in catalog that holds one
golden DataVolume per (schematicID, version) in cozy-public, named
talos-worker-<schematicID>-<version>. A worker pool opts in with osImage.builtin
on its own KubernetesNodes resource, and its disk-system DataVolume then carries
source.pvc pointing at that golden, so CDI performs a storage-layer CSI clone
instead of an HTTP import -- one import per flavor, shared by every worker of
every tenant. Confirmed on a LINSTOR-backed cluster: the worker disk is a CSI
clone plus a resize, with no host-assisted upload pod.

Both halves are off unless asked for. The catalog is registered through the
platform's optional-package helper and disabled by default, so a cluster that
does not enable it creates no golden and keeps importing over HTTP. The disk
source is derived entirely from osImage, which is absent by default, so a pool
that does not set it renders byte-identical to the pre-feature chart and
adopting the feature never rolls existing workers. Editing osImage on an
existing pool re-images that pool, as changing any other image reference does.

What the render refuses, rather than degrading into a silent fallback:

- A pool whose golden has no catalog entry, or whose golden terminally failed to
  import. A golden that is merely mid-import is not an error; CDI holds the
  worker disk until the source populates.
- A pool whose StorageClass differs from its golden's. CDI cannot CSI-clone
  across classes and falls back to a host-assisted copy over the pod network,
  which is the transfer this feature exists to remove. An empty class on either
  side means the cluster default and is resolved before the comparison, and a
  pool that has already cloned reads its class from its own bound PVC rather
  than from today's default -- the golden's side is frozen the same way, and an
  admin moving storageclass.kubernetes.io/is-default-class must not fail the
  release of a pool whose disks are all on the right class. That resolved class
  is then written onto the worker disk, so the disk binds where the guard
  checked instead of on whatever the default has become by then -- reading one
  class and binding another is how a pool at an empty storageClass passed the
  comparison and still took the host-assisted copy. The HTTP path leaves an
  empty class to the cluster as before, which is what keeps its render, and the
  KubevirtMachineTemplate hash named after it, byte-identical.
- A diskSize below the golden's storage size, which a clone target cannot be.
- A schematicID that is not a plain lowercase alphanumeric identifier, or a
  version that is not a vMAJOR.MINOR.PATCH release. Both are interpolated into
  a DataVolume name and an image URL, and values.schema.json types them as bare
  strings, so a malformed one otherwise reaches the render. The catalog applies
  the same patterns, since both sides build the same name from the same fields.
- An images[] entry edited after its golden exists. A DataVolume spec cannot be
  patched in place, so the message names the entry and the supported move -- a
  new (schematicID, version) -- instead of leaving the API server to reject the
  patch on the next upgrade.

Goldens carry helm.sh/resource-policy: keep, so dropping an entry stops
managing a golden rather than deleting it: a missing golden aborts the whole
Kubernetes HelmRelease of every tenant pinned to it, not just the disk. Each
cloned worker disk annotates the golden it came from and the pool that owns it,
so "does anything still clone this?" is answerable before a manual delete
instead of after.

A builtin pool needs a StorageClass whose volumes are reachable from any node,
in practice a replicated/DRBD one. The golden in cozy-public has no consumer and
binds as soon as it imports, wherever CDI's importer landed, while a worker disk
binds where its VM is scheduled, and nothing here arranges for those to be the
same node. The platform ships cloneStrategyOverride: csi-clone, which rules out
the host-assisted strategy that would cross nodes. What a node-pinned class does
in that situation is not measured, so values.yaml states the requirement and not
a symptom.

ComputePlane pools forward only the per-pool shape fields, and osImage is
excluded there by name: the worker OS image, its Talos installer and its Image
Factory are operator decisions about the platform's own images.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 15:35:44 +03:00
Aleksei Sviridkin c290dee945 fix(vm-instance)!: refuse resources that cannot size the VM
resources.cpu and resources.sockets are typed as quantities, but they
land in domain.cpu.cores and domain.cpu.sockets, which KubeVirt
declares as uint32. The chart cast them with int64, which turns 500m,
1.5 or 1e1 into 0 without an error, and KubeVirt boots a zero core
count as one core. The user asked for half a core and got a one-core
VM with nothing reporting it.

The raw spelling is now decoded as YAML, the way an integer field
reads it, and the render stops unless the result is a positive whole
number that fits in uint32. A leading zero is refused, because YAML
reads 010 as octal 8. The check keys on presence rather than
truthiness, so an unquoted 0 is refused instead of being read as
unset.

Memory gets the same presence check. A quoted "0" was truthy and
rendered requests.memory: "0", while an unquoted 0 was falsy and
dropped, so one quantity took two paths. A zero or negative amount
now stops the render in both spellings.

BREAKING CHANGE: values that used to render now either fail or
render differently. A cpu or sockets that does not decode to a plain
whole number within uint32 (a millicore, suffixed such as 1k,
fractional, zero, negative or leading-zero value), and a zero or
negative memory, fail the next upgrade of the release until the
value is fixed. This includes cpu or sockets set on its own beside an
instanceType, which used to be ignored whatever it held. A spelling
that the int64 cast truncated to 0 but that YAML decodes to a whole
number, such as the exponent 1e1 or the trailing dot 2., used to boot
one core and now sizes the VM with the count it spells, so such a VM
changes size on upgrade.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-23 11:31:07 +03:00
Timofei Larkin 06b41da030 feat(apps): declare the Kubernetes and Kafka name caps in their charts
Both caps are facts about what each chart renders, and nothing outside
the chart can derive them. Every Kubernetes worker pool is a separate
KubernetesNodes release named kubernetes-nodes-<name>-<pool>, so a
cluster name has to leave room for the default md0 pool inside the
53-character Helm release name limit: 32 characters. A Kafka KRaft
controller pod is named kafka-<name>-c-<hash>-<id> with an 8-character
hash, and that hostname has to fit the 63-character DNS-1123 label
limit: 44 characters.

Each chart now states its cap with a @name annotation, which
cozyvalues-gen v1.7.0 publishes as x-cozystack-name in
values.schema.json and hack/update-crd.sh carries into the
ApplicationDefinition. The pre-commit job moves to that release, the
first that understands the annotation.

Assisted-by: LLM
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-09-23 10:59:51 +03:00
Yan Bondarenko eee0596661 docs(tcp-balancer): say that whitelistHTTP needs a non-empty whitelist
The render now fails when whitelistHTTP is set with an empty whitelist,
so the values description tells users that before they hit the error.

Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-09-23 11:18:19 +04:00
Yan Bondarenko 6855bfb465 test(tcp-balancer): check each frontend keeps its own whitelist rules
The accept and reject assertions matched anywhere in haproxy.cfg, so a
frontend that lost its reject rule stayed green as long as another
frontend still carried one. Without reject a non-matching connection
falls through to HAProxy's default accept, so that frontend is open to
everyone. Each assertion now anchors the acl, accept and reject lines to
the frontend they belong to.

Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-09-23 11:18:19 +04:00
Yan Bondarenko 12cc48d465 fix(tcp-balancer): rename backends and pin HAProxy so the chart starts on 3.3+
haproxy:latest now resolves to 3.3 and newer, where a backend named like
a frontend is a fatal config error, so every tcp-balancer pod fails at
config parse. Backends get a _backend suffix and the image follows the
chart appVersion on the 3.4 LTS line instead of a floating tag.

The whitelist rules render inside one guard. On the kubernetes and talos
frontends an empty whitelist means no filtering; whitelistHTTP with an
empty whitelist fails the render, because a security opt-in that quietly
became a no-op would not be noticed the way a crash is.

The unittest suite pins the rendered config for the default, whitelist
and failure cases and the image pin, so a template refactor cannot bring
the collision or the floating tag back.

Fixes #1966

Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-09-23 11:18:19 +04:00
Yan Bondarenko cdcb2710cd docs(apps): correct the shared resources and preset paragraphs
Two sentences of the boilerplate shared by the app READMEs were wrong.
"When left empty, the preset is applied" described an all-or-nothing
rule, but cozy-lib.resources.defaultingSanitize merges per key: every
resource that resources leaves unset comes from the preset. And the
legacy flat names were called aliases of their "1:1 instance-type
equivalents", which reads as the c1 series; nano through small match t1,
and medium matches c1.small rather than c1.medium, the trap
docs/operations/resource-presets.md already warns about.

Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-09-23 01:20:50 +03:00
Aleksei Sviridkin 8c5126ccd9 fix(kubernetes): give System-mode tenant OIDC users a UID (#4379)
## What this PR does

Users authenticated through `spec.oidc.mode: System` on a tenant
Kubernetes cluster now get a UID. The AuthenticationConfiguration the
chart renders mapped `username` and `groups` and nothing else, and with
no `claimMappings.uid` entry the apiserver resolves such a user to an
empty UID. The `X-Remote-Uid` header the tenant control plane publishes
then reaches an aggregated API server carrying nothing.

The mapping takes the `sub` claim. The username claim is `email`, and
the platform's Keycloak realm turns on the UPDATE_EMAIL required action,
so that value is one a user changes; `sub` is the account UUID and stays
where it is.

Users on a cluster that is already running gain a UID where they had
none. Tenant RBAC does not notice, because a role binding matches a
username or a group. Audit records on the tenant apiserver do carry
`user.uid` now, `kubectl auth whoami` prints it, and an admission
webhook or extension server inside the tenant that read the empty value
reads the subject instead.

`CustomConfig` mode stays as it was. An operator-supplied configuration
is written verbatim, uid mapping or not, and a unit test pins that the
chart adds none of its own.

Closes #4166

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff, which touches the tenant
chart's OIDC template, its unit tests, `docs/oidc-tenant.md`, and the
shared e2e runner library with its bats twin.

`values.schema.json` does not change. No values key is added, renamed or
removed, no default moves and the version enum is untouched, so the
terraform-provider's hand-written schema and its expand/flatten pair
stay correct. No package is added or renamed and no
`ApplicationDefinition` changes, so the website's app lists and
generated reference pages are unaffected, and its tenant OIDC page
carries no claim about UIDs that this could make stale. The installer
values, the platform values and the variant list are untouched, so the
ansible role is unaffected.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(kubernetes): users authenticated through `spec.oidc.mode: System` on a tenant Kubernetes cluster now carry a UID, taken from the issuer's `sub` claim, where they previously had an empty one. Audit records, `kubectl auth whoami` and the `X-Remote-Uid` header that reaches an aggregated API server inside the tenant all show it. RBAC is unaffected: a role binding never matches on a UID.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* System-mode OIDC authentication now provides a stable user identifier
based on the account subject.
* Existing clusters now expose this identifier in audit records,
`kubectl auth whoami`, and to extension servers without changing tenant
RBAC behavior.
* Custom OIDC configurations continue using operator-defined UID
mappings.

* **Documentation**
* Updated OIDC documentation to explain stable identifiers and their
visibility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 01:31:37 +04:00
IvanHunters ff0b851bbc test(vpn): add default render helm unit tests (#4347)
## What this PR does

vpn was one of the eight user-facing apps in `packages/apps/` called out
in #3631 as shipping with zero helm-unittest coverage. This adds a
`tests/` suite pinning the default render and wires the standard `test:
helm unittest .` target the sibling charts already use, following the
same pattern as #4081, #4074 and #4093.

Ran `helm unittest .` locally against the chart, it passes. `make show`
needs a live cluster connection through `cozyhr`, which was not
available in this environment, so I relied on `helm unittest` for local
verification, same as the referenced sibling PRs.

### Screenshots

Not applicable, test-only change.

### Downstream repositories

Walked the trigger map in docs/agents/contributing.md against the diff:
this only adds a test file and a Makefile target inside an existing
package, it does not add, rename or remove a package, and it does not
touch `values.yaml`, `values.schema.json`, `Chart.yaml` or `README.md`,
so none of the listed triggers apply.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
test(vpn): add Helm unit test coverage for the default render
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Tests**
- Added automated validation for the VPN deployment’s default
configuration, including replica count, container image, service port,
and generated resource naming.

- **Chores**
- Added a convenient command for running the VPN chart’s automated
tests.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-22 23:07:18 +03:00
Aleksei Sviridkin e2464ea788 fix(kubernetes): give System-mode tenant OIDC users a UID
The AuthenticationConfiguration the chart renders mapped username and
groups only, so upstream resolved every System-mode user to an empty
UID and the X-Remote-Uid header the tenant apiserver publishes reached
an aggregated API server carrying nothing.

Map uid to the sub claim. The platform realm enables Keycloak's
UPDATE_EMAIL required action, so the address the username claim
carries is mutable and cannot serve as a stable identifier, while sub
is the account UUID and survives a change of either. CustomConfig
mode is untouched: an operator-supplied configuration is still
written verbatim, uid mapping included or not.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-22 23:04:28 +03:00
Aleksei Sviridkin 4bf95815a8 fix(kubernetes): render the aggregation-layer UID header for tenant OIDC (#3539)
## What this PR does

Tenant clusters with `spec.oidc.mode: System` or `CustomConfig`
authenticate users fine, but a UID does not survive the hop to an
aggregated API server. The tenant kube-apiserver publishes
`requestheader-uid-headers` into the
`extension-apiserver-authentication` ConfigMap only when the flag is
set, and the chart never set it. The aggregation layer still sends
`X-Remote-Uid`, but the extension server trusts only the header names
published in that ConfigMap, so it ignores the one it gets and the call
arrives without a UID. The field documentation even told operators to
add the flag by hand via `controlPlane.apiServer.extraArgs`.

It forwards a UID rather than creating one. ServiceAccount tokens carry
one, and for those the aggregation hop starts working. Users
authenticated by `spec.oidc.mode: System` still have none, because the
generated AuthenticationConfiguration maps `username` and `groups` only;
giving them one means mapping a claim, which changes the identity of
users who have none today, and is tracked separately.

The chart now renders `--requestheader-uid-headers=X-Remote-Uid` on
v1.32 and newer, and adds `--feature-gates=RemoteRequestHeaderUID=true`
on v1.32, where the gate is still alpha and the apiserver rejects the
flag without it. From v1.33 the gate is beta and on by default, so only
the header flag is rendered. v1.31 gets nothing: the flag does not exist
there, and writing it by hand stops the apiserver from starting.
`--emulated-version` in the same passthrough moves that decision,
because the gate resolves through the versioned spec at the emulated
version.

Rendering the gate on every version from v1.32 rather than on v1.32
alone was the alternative, and it would remove the migration step
entirely, because an opt-out would then drop two entries instead of one
and move the entry count by itself. It loses on what it leaves behind: a
redundant gate term on every cluster, which stops the apiserver from
starting on the day the gate is retired from the known list. The design
also prefers rendering over failing for the same reason the fail
branches are narrow, since a cluster stranded on a HelmRelease that no
longer converges is a worse outcome than one that never gets the UID
fix.

The chart reads an operator entry the way the apiserver reads it: header
list elements matched whole and untrimmed and folded as `EqualFold`
folds them, gate names compared exactly, gate values resolved as
`strconv.ParseBool` resolves them, `AllAlpha` and `AllBeta` blankets
honoured, an empty value treated as no entry. On v1.32 an entry that
says nothing about `RemoteRequestHeaderUID` gets a second one carrying
the operator's own terms plus `RemoteRequestHeaderUID=true`. Carrying
those terms into the merged copy is what makes it land correctly whether
the tenant control plane collapses duplicate flags into the last one or
hands both to the apiserver, which merges them in the order it reads
them.

Tests cover the render matrix over mode, version and operator input:
every fail path with its exact message, the ownership and merge paths,
duplicate entries in both orders, blanket gates, empty values, and case
variants on the header name and on the gate value. The latest-version
tenant lane creates its tenant with `spec.oidc.mode: System` and asserts
the flag on the rendered KamajiControlPlane, so the tenant apiserver
starts with the flag in CI. Nothing asserts the UID arriving at an
aggregated apiserver; that needs an extension server inside the tenant
and is not in this change. The layer where the count-keyed reset
actually bites is the tenant Deployment's arguments, and asserting there
needs a live tenant control plane, so it is not in this change either.

The rule this follows is that a configuration which renders and boots
today keeps rendering. Both flags are new, so an `extraArgs` shape
reaching this code was rendering before it existed, and stranding a
running tenant on a HelmRelease that no longer converges is a worse
outcome than one that never gets the fix. A few shapes do stop
rendering, and reaching any of them takes writing
`--requestheader-uid-headers` into the passthrough by hand, attempting
exactly what the chart now does and getting it wrong; in each the
kube-apiserver fails option validation and never starts, so nothing that
was working is lost. `docs/oidc-tenant.md` covers these shapes. One
known limitation is in there and worth naming here: on v1.33 and newer,
adding the documented opt-out is count-neutral in `extraArgs`, and so is
removing it again, and on v1.33 and v1.34 adding or removing
`--emulated-version=1.31`. The pinned Kamaji build rewrites a running
apiserver's arguments only when that count changes and otherwise keeps
every flag that has left the list, so either the header flag stays
beside a now-disabled gate or the disabled gate stays beside the header
flag, and the apiserver refuses to start. Changing the entry count in
the same edit is the mitigation, and the fix belongs to the pin rather
than to this chart, tracked in #3541. The paths that need no operator
edit are clear: turning the feature on for a default tenant moves the
count, and a tenant already carrying the opt-out renders
byte-identically before and after.

Out of scope: `mode: None` still publishes no UID header even though
ServiceAccount tokens carry UIDs. That is pre-existing, and widening the
condition changes the render for every default tenant. Tracked
separately, along with the claim mapping above.

Closes #3377

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change

Checked the trigger map against the diff: no package added or removed,
no schema field, enum or default changed (description text only), no
ApplicationDefinition surface touched, no hack/ layout moved. The
managed-apps reference page picks up the new field description through
release-time README regeneration.

### Release note

```release-note
fix(kubernetes): tenant clusters with `spec.oidc.mode` set to `System` or `CustomConfig` now render `--requestheader-uid-headers=X-Remote-Uid` on the tenant kube-apiserver, plus the `RemoteRequestHeaderUID` feature gate on v1.32, so that an aggregated API server can read the caller's UID out of the request headers. Without that flag the tenant apiserver never publishes `requestheader-uid-headers`, and an aggregated API server ignores the UID header even for callers that have one, ServiceAccount tokens among them. It forwards a UID rather than creating one: users authenticated by `spec.oidc.mode: System` still have none, because the generated AuthenticationConfiguration maps `username` and `groups` only, and that is tracked separately. Adding the flag by hand through `controlPlane.apiServer.extraArgs` is no longer necessary, and an entry of your own is respected: the chart steps aside when yours already carries `X-Remote-Uid` or already enables the gate, merges `RemoteRequestHeaderUID=true` into a copy of a v1.32 `--feature-gates` entry that says nothing about the gate, and renders neither flag when you switch the gate off. The render fails where a `--requestheader-uid-headers` list of your own would stop the kube-apiserver from starting anyway: one that omits `X-Remote-Uid`, one with an empty element, and a list of your own while the gate is off. On v1.33 and newer there is a migration step, and it applies in both directions: the pinned Kamaji build rewrites a running apiserver's arguments only when the number of `extraArgs` entries changes, and otherwise keeps every flag that has left the list. Adding the documented opt-out keeps that number the same, so the header flag already on the Deployment stays beside the now-disabled gate. Removing the opt-out later keeps it the same too, so the disabled gate stays on the Deployment beside the header flag the chart renders again, although your spec no longer shows it. Adding or removing `--emulated-version=1.31` has the same shape both ways on v1.33 and v1.34. The apiserver refuses to start in each case, so change the entry count in the same edit.
```
2026-09-23 00:03:20 +04:00
Aleksei Sviridkin 12e64a99cd fix(rabbitmq): right-size the default resources preset to s1.nano (#3936)
## What this PR does

Raises the RabbitMQ default resource preset from `t1.nano` (128Mi
memory) to `s1.nano` (250m CPU / 512Mi memory).

A default-sized RabbitMQ 4.2 broker on Cozystack 1.6.1 was OOMKilled
with exit 137, reached 10 restarts, and never became Ready. A canary on
a larger preset reached 3/3 Ready with zero restarts; its working set
peaked at 125.8Mi. Against the 128Mi `t1.nano` ceiling that is 98% of
the limit in steady state, and cgroup accounting (working set plus page
cache) crosses it under load. The canary and its storage were removed
after validation.

### Sizing

`s1.nano` keeps the CPU of `t1.nano` (250m) and quadruples memory to
512Mi — a 4x margin over the observed 125.8Mi peak.

An earlier revision of this PR proposed `u1.nano` (1Gi). Review
correctly objected that 1Gi reserves 8x the observed working set with no
measured peak to justify it, and asked to either report that peak or
right-size the default. The peak is reported above, and the default is
right-sized to the smallest preset that clears the ceiling with a real
margin rather than to the first one that is comfortably large.

### Upgrade impact

cozystack-api applies OpenAPI schema defaults on the read path and
stores what the client sent: `Get` runs `applySpecDefaults` on its way
out (`pkg/registry/apps/application/rest.go:1232`) and the write puts
`app.Spec` into `HelmRelease.spec.values` (`rest.go:1652`). That splits
existing brokers in two.

**Brokers with no stored `resourcesPreset`** pick up the new chart
default on the next Helm reconcile. For them this upgrade is:

- a rolling restart of every replica, and
- a rise in guaranteed memory reservation from 128Mi to 512Mi per
replica, because the sanitize helper sets memory request equal to limit
— roughly 1.1Gi extra reserved per 3-replica instance.

**Brokers whose values went through a read-modify-write keep `t1.nano`
and are not resized.** `Update` opens with `r.Get` (`rest.go:489`), so a
GET-then-PUT — `kubectl edit` is the unambiguous case, and any client
that reads before writing behaves the same — freezes the schema default
of the moment into the stored values, and a stored value beats the chart
default. Those brokers are not restarted, not resized, and keep hitting
the ceiling this PR raises.

Nothing downstream can separate a deliberate `t1.nano` from one a round
trip baked in, so this is documented rather than migrated. After the
upgrade the affected brokers are the ones that still report the old
preset:

```
kubectl get rabbitmq -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name,PRESET:.spec.resourcesPreset
```

The same command before the upgrade tells you nothing: the cozyrd
default is still `t1.nano` then, so the read path fills it in for both
groups. Afterwards, anything reading `t1.nano` carries a stored value;
setting `resourcesPreset: s1.nano` explicitly on the ones that should
move is the fix.

PVCs are untouched, so there is no data loss, but node headroom should
be checked before upgrading.

Also worth knowing when reading the parameter table: the preset does not
only apply to a fully omitted `resources`.
`cozy-lib.resources.defaultingSanitize` merges it key by key, so a
broker pinned to `resources: {cpu: 700m}` keeps its CPU and moves from
128Mi to 512Mi with everyone else. The field descriptions now say so,
and a test pins it.

The values, generated schema, ApplicationDefinition, typed API, and
README all carry the same default. Tests cover the new default, an
explicit legacy override, explicit resources, and a partially set
`resources` map inheriting the preset for what it leaves out.

Validation:

- `helm template` with the chart's `cozy-lib` dependency renders `limits
250m / 512Mi` and `requests 0.025 / 536870912` for the default; an
explicit `t1.nano` renders `128Mi / 134217728`; explicit `700m / 2Gi`
renders `0.07 / 2147483648`. The unit test asserts exactly these values.
- Provider counterpart: cozystack/terraform-provider-cozystack#34.

### Screenshots

Not a UI change.

### Downstream repositories

The default is restated by the Terraform provider, so the linked
provider PR updates it and documents the state-plan implication. No
other trigger-map entry matches this diff.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
https://github.com/cozystack/terraform-provider-cozystack/pull/34
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(rabbitmq): raise the default resource preset from t1.nano to s1.nano so default-sized brokers are no longer OOMKilled. Brokers with no stored `resourcesPreset` are restarted on upgrade and their guaranteed memory reservation grows from 128Mi to 512Mi per replica; check node headroom before upgrading. Brokers whose values already pin `resourcesPreset`, including ones where a read-modify-write through the API froze the old default into them, keep their old size: list them after the upgrade with `kubectl get rabbitmq -A -o custom-columns=NAME:.metadata.name,PRESET:.spec.resourcesPreset` and set `resourcesPreset: s1.nano` explicitly on the ones that should move. The preset also fills any resource a partially set `resources` leaves out, so a broker pinned to cpu alone moves too. A tenant that opted into `resourceQuotas` and sits near its memory ceiling has the extra reservation rejected by quota admission, which shows up as a StatefulSet that never scales rather than as pods left Pending.
```
2026-09-22 21:53:02 +04:00
scooby87 7d07bbe3ce [kafka] Migrate from ZooKeeper to KRaft mode (#2352)
## Summary

- Replace ZooKeeper-based Kafka deployments with KRaft (Kafka Raft)
metadata mode
- Existing ZooKeeper instances migrate automatically and in place via a
Helm `pre-install,pre-upgrade` hook Job that drives Strimzi's
ZK-to-KRaft migration state machine
- New instances deploy directly in KRaft mode with broker and controller
`KafkaNodePool`s
- The node pools are owned by the migration hook, not by regular chart
manifests (see "Why the pools are hook-managed")
- Kafka version is user-configurable (v3.9 default, v3.8 available)
- Strimzi operator is unchanged in this PR (stays at 0.45.1-rc1); 0.45.x
is the last line that supports ZooKeeper→KRaft migration
- ZooKeeper removed from the dashboard UI (ApplicationDefinition)

## Migration strategy

Both fresh installs and ZooKeeper→KRaft migrations are driven by a
single Helm hook Job (`templates/migration-hook.yaml`,
`pre-install,pre-upgrade`) that owns the `KafkaNodePool`s. On every
install and upgrade the Job:

1. Ensures both node pools exist (idempotent): a broker pool named
exactly `kafka` for a migrating cluster — so Strimzi adopts the existing
`<cluster>-kafka-N` brokers and their PVCs in place — or `b-<hash>` for
a fresh cluster, plus a controller pool `c-<hash>`. The controller pool
pins `strimzi.io/next-node-ids` above the broker range so the broker
pool keeps node IDs `[0..N-1]` and Strimzi cannot hand the existing
broker IDs to the fresh controllers. The shared `kafka` name is claimed
atomically (via `kubectl create`) so only one migration per namespace
can win it.
2. Inspects the Kafka CR: no CR → fresh install (pools ensured; the main
apply then creates the KRaft Kafka CR, nothing to migrate); already
`strimzi.io/kraft=enabled` or `status.kafkaMetadataState: KRaft` → done;
any other read error → fail closed. Otherwise (a ZooKeeper cluster) it
migrates.
3. Annotates `strimzi.io/node-pools=enabled` then
`strimzi.io/kraft=migration`, polls `status.kafkaMetadataState` through
`ZooKeeper → KRaftMigration → KRaftDualWriting → KRaftPostMigration →
PreKRaft → KRaft`, and only once a safe state is reached flips
`strimzi.io/kraft=enabled`. If it times out in an intermediate state it
aborts without finalising, so Helm fails the upgrade and the next
reconcile retries from where it left off.

Data is migrated in place: brokers keep their node IDs and reuse their
existing PVCs, nothing is copied. Migration is one-way — Strimzi has no
supported KRaft→ZooKeeper rollback once complete, so lowering the
`version` field or rolling the app back to a pre-KRaft chart is
unsupported (take a backup first).

### Why the pools are hook-managed

The hook must create the pools and walk the migration state machine
*before* the main release apply. A resource a hook creates cannot also
be a regular chart manifest across the ZK→KRaft boundary: Helm's main
apply finds the pool live in the cluster but absent from the previous
(ZooKeeper) release revision and aborts with `original object
KafkaNodePool ... not found`, leaving the release `failed` even though
the cluster migrated fine (reproduced under server-side apply, the
helm-controller default). Managing the pools only from the hook avoids
that, so `templates/kafkanodepools.yaml` and the deployed-version gate
ConfigMap (`templates/version.yaml`) are removed and the pools are
applied idempotently by the Job on every install and upgrade.

## Bug fixes included

- ZooKeeper `metricsConfig` referenced wrong ConfigMap key
(`kafka-metrics-config.yml` instead of `zookeeper-metrics-config.yml`)
- ZooKeeper `storageClass` used `kafka.storageClass` instead of
`zookeeper.storageClass`
- WorkloadMonitor used `.Values.replicas` instead of
`.Values.kafka.replicas`
- `validateNameLength` now caps a Kafka app name at the KRaft controller
pod-hostname budget (`<release>-c-<hash>-<id>` must fit 63 chars),
instead of admitting names that later fail the chart render forever

## What's next

A follow-up PR will upgrade Strimzi to 0.51.0 and Kafka to 4.2 once all
instances are confirmed migrated to KRaft. That PR is already prepared
on branch `KRaft-upgrade-strimzi-051`.

## Test plan

- [x] `helm template` renders correctly for new KRaft instances (no
ZooKeeper refs)
- [x] `helm template` with custom values (1 replica, external, topics,
v3.8)
- [x] Version validation rejects unsupported versions via JSON schema
- [x] helm-unittest: 86/86 across 9 suites
- [x] `hack/kafka-migration-hook.bats`: 12/12 — the real hook script
executed against a stub kubectl (resolution branches, fail-closed reads,
atomic collision guard, finalize guard)
- [x] Operator template renders with the unchanged Strimzi 0.45.1-rc1
- [x] Live migration on a disposable cluster under server-side apply:
fresh install → healthy KRaft, release `deployed`; in-place migration of
a 20000-message ZooKeeper cluster → release `deployed`, broker PVCs
adopted in place (same UIDs), and the KRaft data byte-for-byte identical
to the pre-migration ZooKeeper data (matching sha256 over every
`partition:offset:key:value`)

### Release note
```release-note
[kafka] Enable KRaft metadata mode with automatic, in-place ZooKeeper-to-KRaft migration. Kafka version is now configurable (v3.9, v3.8). The Strimzi operator is unchanged (0.45.1-rc1).
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added chart-level Kafka version selector (default v3.9) and
KRaft-aware node pool rendering with controller storage sizing.

* **Updates**
* Default Kafka appVersion updated to 3.9.0; operator and image tag
defaults aligned to 0.45.0.
* Templates now omit ZooKeeper resources and metrics when KRaft is in
use.

* **Tests**
* New Helm unit tests for Kafka CR, node pools, metrics, topics,
dashboard mappings, and workload monitors.

* **Documentation**
* Values, schema, and README updated to document new version and
controller storage options.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-22 19:19:21 +03:00
Aleksei Sviridkin f4e923aa82 docs(kubernetes): restate the apiServer extraArgs contract for OIDC
The extraArgs field documentation told operators to add
--requestheader-uid-headers themselves, which the chart now renders
whenever OIDC is on. Describe what it renders per version, when it
defers to an operator entry, when it merges into one, and that it still
refuses to render shapes the apiserver would not start on.

The version field says it too: choosing v1.32 is what makes the chart
render the feature gate next to extraArgs, and a reader of that field
alone would otherwise not learn that the two interact. The package's
hack/update-versions.sh writes that description itself on every make
update, so it carries the same text, or the next version bump would
drop the sentence again.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-22 18:23:48 +03:00
Aleksei Sviridkin d870b392fa fix(kubernetes): render the aggregation-layer UID header for tenant OIDC
A tenant kube-apiserver publishes requestheader-uid-headers into the
extension-apiserver-authentication ConfigMap only when the flag is set,
and the chart never set it. Without that key the request-header
authenticator on an aggregated apiserver does not know which header
carries the UID and ignores it, so a caller who has a UID is served
without one. Operators had to add the flag by hand through the
controlPlane.apiServer.extraArgs escape hatch.

This forwards a UID the caller already has and does not create one.
ServiceAccount tokens carry a UID and gain from it. OIDC users
authenticated through spec.oidc.mode: System do not: the
AuthenticationConfiguration the chart generates maps username and
groups only, and with no claimMappings.uid upstream returns an empty
UID (plugin/pkg/authenticator/token/oidc/oidc.go, getUID, read at
v0.35.0), so there is nothing to carry. Mapping a claim would give
those users a UID and change the identity of users who have none
today, which is a separate decision and is tracked separately.

The half read rather than run: that an existing UID is lost without the
published key comes from the controller writing the ConfigMap and the
request-header authenticator consuming it, not from a live cluster. That
OIDC users have no UID is unambiguous in the source named above.

Render --requestheader-uid-headers=X-Remote-Uid whenever OIDC is on,
plus --feature-gates=RemoteRequestHeaderUID=true on v1.32, where the
gate is still alpha and the apiserver rejects the flag without it.
v1.31 gets nothing: the flag does not exist there and an unknown flag
is fatal to the apiserver.

The deciding version is the effective one, not the image tag.
--emulated-version in the same passthrough moves it down, and the gate
resolves at the emulated version through featureSpecAtEmulationVersion
(featureSpecAtEmulationAndMinCompatVersion by v1.35), so an emulated
1.32 needs the gate line even on a v1.35 cluster, and below 1.32 the
gate is PreAlpha and cannot be enabled at all, which leaves nothing
safe to render.

Both flags are new, so every extraArgs shape reaching this code was
rendering before it existed, and turning one of those into a render
error would strand a running tenant on a HelmRelease that stops
converging while its control plane carries on. The chart therefore
yields to an operator entry that already satisfies the requirement,
renders nothing at all when the operator turned RemoteRequestHeaderUID
off, and otherwise merges its gate term into a copy of the operator's
--feature-gates value. Carrying that value into the merged entry
decides the gate whether the tenant control plane collapses duplicate
flags into the last one or hands both to the apiserver, which merges
them in the order it reads them.

Operator entries are read the way the apiserver reads them: header list
elements matched whole and untrimmed, folded as EqualFold folds them,
and refused when one is empty after trimming, as checkForWhiteSpaceOnly
refuses it; gate names compared exactly; gate values resolved as
strconv.ParseBool resolves them; blanket AllAlpha and AllBeta honoured;
an empty value treated as no entry at all. Each flag is judged by its
last entry carrying a value. The apiserver's CSV reader strips quotes
the chart does not parse, so quotes on that entry make the chart render
nothing, while quotes on an earlier entry the collapse drops do not.

The render fails only where a uid-headers list of the operator's
describes a control plane that already refuses to start: one that
omits X-Remote-Uid, one with an element that is empty after trimming,
and any such list while the gate is off. Withholding the chart's own
flag rescues none of them, because the flag the apiserver rejects is
the operator's.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-22 18:23:48 +03:00
Akanksha Trehun 422b6f65ea test(vpn): add default render helm unit tests
vpn was one of the remaining user-facing apps with no
helm-unittest coverage.

Signed-off-by: Akanksha Trehun <akankshatrehun@gmail.com>
2026-09-22 18:43:00 +04:00