chore(release): promote v1.7.0-rc.3 -> v1.7.0 (#5161)

Promotes `v1.7.0-rc.3` to stable `v1.7.0` without rebuilding containers.
Their digests remain bit-for-bit what passed rc e2e; the tag-rewritten
packages tree is re-serialized as a temporary OCI candidate and pinned
here by digest. On merge, pull-requests-release.yaml verifies that
candidate against this tree before creating the write-once `v1.7.0` tag,
retagging the candidate and container digests to `v1.7.0` (+`:latest`
when this is the newest stable), publishing the stable cozy-installer
chart, and publishing the release. Do NOT squash-merge (decision B): the
stable tag must attach to a real merge commit.

✅ RC full e2e was verified for `v1.7.0-rc.3` before promotion.

ℹ️ E2E already ran against the immutable rc, so this promote PR does not
run E2E by default. A maintainer may add the `full-e2e` label to run the
full suite again.

✅ Includes `docs/changelogs/v1.7.0.md`, which finalize uses verbatim as
the published release body.

✅ Website docs PR opened/refreshed on `cozystack/website` (branch
`update-docs-v1.7.0`), generated from the `release-1.7.0` staging
branch. **Do NOT merge that PR until `v1.7.0` is published** — merging
it early flips the site's latest-version pointer to an unpublished
version and 404s its docs/API links (see the PR body).
This commit is contained in:
myasnikovdaniil authored and GitHub committed 2026-10-29 16:19:36 +05:00
commit a32334aeab
51 files changed
+1098 -135

No files matched your search

+891
View File
@@ -0,0 +1,891 @@
<!--
https://github.com/cozystack/cozystack/releases/tag/v1.7.0
-->
# Cozystack v1.7.0
Cozystack v1.7.0 adds **Proxmox VE as a substrate for tenant Kubernetes workers**, turns worker pools into first-class **`KubernetesNodes` resources**, introduces a **community Marketplace for External-Apps repositories** together with a reworked dashboard and Admin portal, extends the unified backup interface to **MongoDB, Redis, Valkey, RabbitMQ and Kafka** (plus PostgreSQL point-in-time recovery), and brings **TLS and admission-time hardening to the managed databases**. It also adds **managed Valkey**, **distributed tracing**, a **ComputePlane tenant module**, **arm64 image builds** and **slim installation variants**.
Platform components bumped in this release: **Talos v1.13.6 → v1.14.2** (DRBD 9.3.4, ZFS 2.4.4), **Cilium 1.19.5 → 1.20.2**, **KubeVirt v1.9.0** (declarative volume hotplug), **CDI v1.66.1**, **etcd-operator v0.5.6**, **Kube-OVN v1.15.26**, **CloudNativePG 1.30.0**, **Kamaji 26.9.4-edge**, **SeaweedFS chart 4.45**, **Gateway API CRDs v1.6.1**, **ingress-nginx 1.11.6**, and the build moves to **Go 1.27.1 with the Kubernetes 1.37 libraries**.
The platform migration `targetVersion` moves from **54 (v1.6.0) to 61**, so migrations **54 through 60** run as pre-upgrade hooks on the way to v1.7.0. Read the breaking-changes section before upgrading.
> **Note:** Items marked *shipped in v1.6.x* were also released in a v1.6.1–v1.6.5 patch release.
## Feature Highlights
### Proxmox substrate and `KubernetesNodes` worker pools
Managed tenant Kubernetes clusters can now run their worker VMs on an **external Proxmox VE cluster** instead of KubeVirt. A new `substrate` value on the `kubernetes` and `kubernetes-nodes` applications selects `kubevirt` (the default, unchanged) or `proxmox`. The Cluster API Proxmox provider (capmox) and the in-cluster IPAM provider ship as optional packages enabled through `bundles.enabledPackages`, and tenants on the Proxmox substrate get the Proxmox CSI driver and cloud-controller-manager so PersistentVolumes work out of the box. The tenant apiserver is published through a LoadBalancer and worker disks default to linked clones.
Worker pools are no longer an array inside the `Kubernetes` resource: each pool is now its own **`KubernetesNodes`** resource with its own health checks and its own autoscaler provision-time budget. The API accepts an edit of a pool's `spec.cluster`, but the pool's release then refuses to render it, and its `spec.storageClass` is meant to stay fixed too, though for now only the dashboard enforces that. Existing pools are adopted automatically by platform migration 56; each adopted pool rolls its worker VMs once, because the worker TalosConfigTemplate is now named by a hash of its content (#3523). New per-pool knobs include `kernelModules` for Talos (GPU pools load the NVIDIA modules automatically), CPU requests and limits, guest serial-console logging, a registry mirror for air-gapped environments, and an opt-in **golden Talos image** that CDI clones on the storage layer instead of importing the image once per worker.
### Community Marketplace and a reworked console
A new **Marketplace for External-Apps repositories** lets operators connect third-party package repositories: `cozypkg validate/init/push/tap/untap/search`, a cluster-scoped `Tap` API, keyless cosign verification through `cozypkg validate --require-signature`, and a dashboard Repositories view (now under Admin) that connects and disconnects repositories. Connecting a repository installs nothing: an operator installs its applications with `cozypkg add`, after which they appear in the catalog and their categories in the sidebars. Learn more in the [documentation](https://cozystack.io/docs/v1.7/marketplace/).
The dashboard opens on the Console, the Marketplace opens the application list directly, and tenant administration moves into an always-available **Admin portal** that shows the tenant hierarchy across every accessible tenant. Virtual machines gain a **serial console** tab and clipboard paste in the VNC console, LoadBalancer services show a connection hostname, console branding is configurable, and tenants can **import VMs from VMware vSphere** with `VMImportSource` and `VMImportTask` resources (opt-in).
### Backups for more engines, and PostgreSQL point-in-time recovery
The unified `BackupJob` / `RestoreJob` / `BackupClass` interface gains native drivers for **MongoDB** (logical backup, restore into a differently-named instance, PITR), **Redis** (and **Valkey**, through the same strategy), **RabbitMQ** (definitions) and **Kafka** (topic metadata), all bound by the `cozy-default` class, and **PostgreSQL** restores now support **point-in-time recovery** through `spec.options.recoveryTime`. Restores gained safety rails: concurrent restores are refused, a restore into another PostgreSQL major version is refused, a backup is held while its application is being restored, and each Postgres instance gets its own default WAL prefix.
### TLS for managed databases and stricter external exposure
MariaDB, RabbitMQ, MongoDB and Redis (including Sentinel) gain chart-managed or operator-managed **TLS**, the OpenSearch HTTP and Dashboards certificates of external instances now come from cert-manager, and the CA is published to tenants as a key-free `<release>.tenant-ca` trust anchor (also for NATS and Qdrant). External exposure is now fail-closed: Redis can only be exposed with TLS (Valkey has no TLS option yet and is exposed in plaintext), external PostgreSQL refuses plaintext sessions, the tenant Gateway supports TLS-passthrough listeners for native database ports (set on the `TenantGateway` resource by hand, since no chart value renders them yet), and an opt-in `networking.encryption.enabled` adds IPsec east-west encryption for the default Kube-OVN datapath.
### Distributed tracing and new managed services
Monitoring gains a **VictoriaTraces** backend with a Grafana traces datasource, a per-tenant OTLP collector, and an opt-in **shared-central** topology where tenants send spans to one store in `tenant-root` with per-tenant write-rate limits. Tracing is off by default and enabled with a `tracingStorages` entry. New managed services and modules include **Valkey** (BSD-licensed, running on the migrated freshworks-oss redis-operator), the **ComputePlane** tenant module for untrusted-code applications, an opt-in OpenEBS **LocalPV hostpath** StorageClass, `talos-log-collector`, KEDA-based **read-replica autoscaling for PostgreSQL**, a PostGIS image flavor, and Kafka **KRaft** mode with automatic in-place ZooKeeper migration.
### Multi-architecture images and slim installs
Package images now build for **linux/amd64 and linux/arm64** by default, and Talos, Harbor, the Velero KubeVirt plugin, ingress-nginx, cozy-proxy, the cozystack-scheduler and the keycloak-kms-proxy ship multi-arch releases. New **`isp-slim`, `isp-slim-generic` and `isp-hosted-slim`** installation variants install only the base platform with Cilium-only networking; applications, operators, monitoring, backups, etcd and SeaweedFS are opt-in through `bundles.enabledPackages`.
## ⚠️ Breaking Changes and Required Actions
Several changes in this release alter behavior of running clusters. Read this section before applying the v1.7.0 Platform Package. Platform migrations **54 through 60** run as pre-upgrade hooks: they adopt existing RedisFailovers onto the new operator group (54), orphan tenant FluxCD HelmReleases (55), adopt worker pools into `KubernetesNodes` (56), pin the SeaweedFS volume size limit of existing instances at 1000 (57), freeze HTTPCache on the packages artifact the cluster already runs (58), copy VMI-only hotplugged volumes into their VM spec (59), and carry each FoundationDB `cluster.version` over to the new `version` value while recording the FoundationDB, OpenSearch, Kubernetes and KubernetesNodes releases that need attention (60).
### Pre-upgrade checks and required actions
**1. Talos v1.14.2 needs `drbd_transport_tcp` (#4598).** Before upgrading a node, add `drbd_transport_tcp` to its Talos `machine.kernel.modules` next to `drbd`: Talos v1.14.2 no longer loads it on demand, and without it DRBD cannot connect to any peer. The `talm` cozystack preset from talm v0.36.0 loads it explicitly (cozystack/talm#252). Talos v1.14 also moves etcd's HTTP metrics and health endpoints from port 2379 to 2383, and the Cozystack host network policy now blocks 2383 from outside the cluster. Talos accepts a direct upgrade from v1.12.0 or later, so the v1.13 line can be skipped.
**2. Tenant Kubernetes worker pools move to `KubernetesNodes` (#3315).** `spec.nodeGroups`, `spec.nodeHealthCheck` and `spec.maxNodeProvisionTime` are removed from the `Kubernetes` resource and move to `KubernetesNodes`; existing pools are adopted on upgrade and roll their worker VMs once (#3523), and the removed fields stay accepted but ignored on upgraded clusters (editing one returns an admission warning). A pool whose release name `kubernetes-nodes-<cluster>-<pool>` would exceed 53 characters, or whose pool name a release name cannot carry, is not adopted: its objects are kept, listed in ConfigMap `cozy-system/cozystack-migration-56-unadopted`, and need to be adopted or recreated by hand. A `Kubernetes` cluster name is now capped at 32 characters. Existing pools keep their roles; a cluster that uses the `ingressNginx` addon and is created after the upgrade, or had no pool with that role, needs a `KubernetesNodes` pool with `roles: [ingress-nginx]`, otherwise the ingress controller DaemonSet schedules nothing. The `addons.fluxcd` addon is removed (#3379); tenants that had it enabled keep their running Flux release, which must be uninstalled by hand inside the tenant cluster if no longer wanted.
**3. Kamaji rolls every tenant control plane (#4398).** Updating Kamaji to 26.9.4-edge makes every tenant kube-apiserver rewrite its arguments and roll on the first reconcile, and it rolls a second time when the tenant's `kubernetes` release updates the talos-csr-signer sidecar image, which every Cozystack upgrade does; a tenant with OIDC enabled can roll a third time when the chart's own flags land. Kamaji now sets `--enable-admission-plugins` itself: the chart moves an `--enable-admission-plugins=` entry in `controlPlane.apiServer.extraArgs` into `admissionControllers`, and between the Kamaji update and the upgrade of the tenant's `kubernetes` release a tenant with such an entry runs its kube-apiserver with Kamaji's default admission plugins, which do not include NodeRestriction. Plan the upgrade window accordingly, and do not roll the Kamaji image back afterwards without checking the release notes of the PR.
**4. Cilium takes its addresses from a reserved block on Kube-OVN variants (#4778).** `networking.podCIDR` must be `/20` or wider on these variants; Cilium now uses the last `/21` of it, which Kube-OVN no longer hands out. Existing clusters move through a post-upgrade hook and each Cilium agent restarts twice (clusters on v1.6.5 already moved in that release; the restarts apply when upgrading from v1.6.4 or earlier).
**5. KubeVirt eviction and hotplug changes (#4403, #4453).** The cluster-wide eviction strategy becomes `LiveMigrateIfPossible`: VMs that cannot live-migrate (ReadWriteOnce disks, GPU or host-device passthrough) are shut down and restarted by their `runStrategy` instead of blocking a drain. A running VM keeps the old strategy until its next restart, so restart non-migratable VMs once after upgrading if their nodes need to drain. KubeVirt also runs with `DeclarativeHotplugVolumes`; migration 59 copies VMI-only volumes into `VM.spec`, which keeps them attached on tenant Kubernetes workers. On a `VMInstance` the same upgrade re-renders the VM from the chart and KubeVirt unplugs the copied volume from the running VM: before upgrading, detach such a volume inside the guest, remove it with `virtctl removevolume`, and add the disk to the VMInstance's `spec.disks` instead. At v1.6.5 a disk added to `spec.disks` attaches only at the VM's next restart; this procedure follows from the charts and has not been run on a live VM.
**6. External exposure now requires TLS (#4639, #4847, #2682).** A Redis release can only be exposed with `tls.enabled: true` (existing external instances keep working), external PostgreSQL refuses plaintext sessions unless `tls.required: false` is set, and external OpenSearch moves from the operator's CA to the chart's cert-manager CA on first reconcile (set `tls.issuer: operator` beforehand to opt out). Clients that connected with `sslmode=disable` or skipped certificate verification must be updated.
**7. Tenant route hostnames are mandatory (#3475, #3342, #4638).** An `HTTPRoute` or `TLSRoute` in a `tenant-*` namespace must declare a non-empty `spec.hostnames`; routes that relied on inheriting the listener's hostnames are rejected and must name them. Existing routes keep serving, but their next update (a Helm upgrade included) is refused until they name their hostnames. The same now applies to `GRPCRoute` (#3872). TLS-passthrough listener names are now reserved by the listener spec alone, so hostnames answered by a passthrough listener no longer get an HTTPS-terminate listener under HTTP-01.
**8. Managed PostgreSQL and MariaDB user passwords (#4078).** `users[].password` is no longer honored: passwords are always auto-generated into the `<release>-credentials` Secret. Existing releases keep their current passwords, and a `password` left in values is ignored with an admission warning. On the first reconcile after the upgrade, users that were removed from `users` while on v1.6.x but whose roles were left behind are dropped, and their objects move to `cozystack:orphaned` (#4574); clients still logging in with such a role lose access.
**9. Smaller breaking changes.** `vm-instance` now refuses `resources.cpu`, `resources.sockets` or `resources.memory` values that cannot size a VM (#4405, #4569); fix such values before the next upgrade of the release. `ApplicationDefinition.spec.release.prefix` must match `^[a-z0-9-]*$` (#3411). The multus package stages the reference CNI plugins into `/opt/cni/bin` by default; set `networking.stageCniPlugins: false` to decline (#3195); the multus resources default and the `networking.multus.resources` override are described under #4830. The FoundationDB operator no longer ships the 7.1 client, so upgrade 7.1.x clusters to 7.3 first (#4512). The HTTPCache application is removed from the catalog; instances keep working on a frozen artifact (#4200). The SeaweedFS app now defaults `master.volumeSizeLimitMB` to 400 for new instances while existing ones stay at 1000.
**10. A platform variant switch on a live cluster is refused (#4790, backport #5021).** The platform render now looks up the live `cozystack.networking` Package and fails when the variant it is about to render differs from it, because the operator deletes every HelmRelease of that Package the new variant does not install: `isp-full` to `isp-hosted` removes Cilium and Kube-OVN, `isp-full` to `isp-slim` removes Kube-OVN. The error names the value that lets the switch through, `networking.acknowledgeVariantChange: "<new variant>"` in the platform Package values; it confirms that one switch only. The guard fires on any variant change, including Talos to generic, where nothing is deleted. A cluster whose `cozystack.networking` variant was set by hand (for example `cilium-kilo`) fails the upgrade until `cozystack.networking` is listed in `bundles.disabledPackages`: acknowledging does not help there, because the render writes the bundle's variant over the manual one. Helm template, client-side dry-run and first installs are not checked.
**11. A golden-image disk into an unsuitable LINSTOR class is refused (#4796, backport #5022).** A new `vm-disk` created from a golden image is now refused at render time when CDI would CSI-clone it into a LINSTOR StorageClass that does not allow remote volume access from every node (a node-local class, for example), or when both StorageProfiles name a snapshot class that does not exist. Before, such a disk stayed pending forever without an error. Existing disks are exempt. Use the golden image's StorageClass, or have the platform administrator set the CDI `cloneStrategyOverride` to `copy` in the kubevirt-cdi package.
**12. Pod rolls on upgrade.** Besides the Kamaji and Cilium rolls above and the Redis Sentinel and data pod rolls noted under Databases and messaging, the upgrade changes the pod specs of several running workloads: every Kafka cluster rolls once because the JVM environment changes (#5019); the SeaweedFS master, volume, filer and S3 components run the in-tree patched image (#5028); the `kube-multus` DaemonSet rolls node by node to pick up its new resources (#5018); and the LINSTOR controller and satellite pods move to the rebuilt `piraeus-server` image that carries the snapshot-restore fix (#5020). Every PostgreSQL instance restarts at least once in the upgrade that moves the CloudNativePG operator from 1.28 to 1.30.0 (#3951), as a test cluster upgraded from v1.6.5 showed, and from v1.6.4 or earlier an instance with a memory limit below 512Mi and no `shared_buffers` of its own also takes the `shared_buffers` restart (#4724); a cluster with one instance is unavailable while its pod restarts. Redis data pods roll once too, one pod at a time.
**13. FoundationDB selects a release line, not a version (#5085, backport #5108).** `cluster.version` no longer selects the deployed version. A release selects its release line with `version: v7.3` (the default) or `version: v7.4`, and the platform pins the patch in `packages/apps/foundationdb/files/versions.yaml`: `v7.3` deploys 7.3.63 and `v7.4` deploys 7.4.1. Migration 60 carries each release's line over from `cluster.version`, and records a release whose patch differs from the pinned one in the `cozystack.io/migration-60-needs-attention` annotation of the `cozy-system/cozystack-version` ConfigMap. A cluster that runs another patch of its line is moved to the pinned one, downwards included: on a test cluster a 7.3.69 cluster was moved to 7.3.63, and the migration's annotation entry was the only lasting signal. A step down inside one line is a patch downgrade, which FoundationDB's [documentation](https://apple.github.io/foundationdb/administration.html#version-specific-notes-on-downgrading) does not exclude: it rules out downgrades between non-patch releases. A release whose `cluster.version` is on a line the platform no longer offers, such as 7.1, is recorded and its render is refused, so it stays on its current revision. Setting `version: v7.3` does not end that refusal while `cluster.version` still names 7.1 (on a test cluster such a release read back as `version: v7.3` and was still refused), and the operator, which no longer ships the 7.1 client, cannot move a running 7.1 cluster (#4512): move 7.1.x clusters to 7.3 before upgrading the platform. A release whose `cluster.version` is on a different line than its `version` is refused with a message that names both. The API prints no warning for `cluster.version`, and it admits an edit of a release that stores a `version` the chart does not offer; the chart refuses that value when it renders. Where platform migrations do not run, set `version` to the line each cluster runs before upgrading. Check the running patch of every FoundationDB cluster before upgrading, and read the annotation afterwards.
**14. Image overrides of Kubernetes, KubernetesNodes and OpenSearch are no longer read (#5085, backport #5108).** `images.kubectl`, `images.waitForKubeconfig` and `images.talosCsrSigner` of `Kubernetes`, `images.kubectl` of `KubernetesNodes` and `images.opensearch` of `OpenSearch` are removed, and these charts use the images the platform ships instead. A release that still carries one of them keeps the value stored, the chart ignores it, and the API shows no warning for it: on a test cluster, editing such a value through the API printed no warning about it (an edit of a `Kubernetes` resource printed only the existing `spec.nodeGroups` warning). Migration 60 lists such releases in the `cozystack.io/migration-60-needs-attention` annotation. On the first reconcile OpenSearch nodes move to the image for their selected version and Kubernetes components whose images changed restart. Before upgrading, check custom OpenSearch images: if one runs a newer version than the chart maps, switching back can leave the nodes unable to read their data. An OpenSearch release that stores `version: null` now fails to render instead of falling back to `v2`, while the API still reads it back as `v2`; set its `version`. Air-gapped installations need `docker.io` and `ghcr.io` mirrored on the management-cluster nodes.
**15. A disk still importing holds its release during the upgrade (#5087, backport #5126).** An existing `vm-disk` whose DataVolume is not ready when the platform upgrades, an import still running for example, can keep its HelmRelease in a failing upgrade and occupy one helm-controller worker until the release reconciles on its own; on a test cluster that took two failed 10-minute attempts, about 20 minutes. The cause is ordering: cozystack-controller, which copies the readiness expressions onto existing HelmReleases, starts only after the `cozystack.networking` Package is upgraded, which holds Cilium and Kube-OVN on the `isp-full` variants, and every attempt that starts before that copy waits on the DataVolume until it times out. The wait is bounded; without #5087 such a release retried without end. Finish running imports before upgrading; `kubectl get dv -A` shows the phase of every DataVolume.
**16. Read the migration log right after the upgrade.** The log of the `cozystack-migration-hook` Job in `cozy-system` carries the migrations' warnings; read it with `kubectl logs -n cozy-system job/cozystack-migration-hook` once the migrations finish. The chart keeps the Job until the next run of the hook replaces it, but on a test cluster, whose nodes had reported `DiskPressure` during the upgrade, its container log could no longer be read about 20 minutes after the run; the cause was not established. Migration 60's records are also kept in the `cozystack.io/migration-60-needs-attention` annotation, which it writes only when it recorded at least one release.
**17. Free disk space on the nodes.** Observed on a test cluster: two nodes reported `DiskPressure` during the platform upgrade and the kubelet evicted pods from one of them, so make sure the nodes have free disk space before upgrading.
### Breaking changes in this release
* **feat(opensearch)!: issue HTTP and Dashboards certificates from cert-manager**: The OpenSearch HTTP API and Dashboards already served TLS, from the operator's own private CA and with a SAN built from cluster.local while the credentials Secret advertised the release on the platform domain; for an external instance both certificates now come from a per-release cert-manager CA whose anchor the platform publishes into the tenant as <release>.tenant-ca, while an internal one keeps the operator CA unless `tls.issuer: cert-manager` is set. Any client that worked against the old endpoint was skipping verification, because no hostname it was handed could have matched that SAN ([**@Arsolitt**](https://github.com/Arsolitt) in #2682).
* **feat(multus)!: stage reference CNI plugins from the multus image**: Install the reference CNI plugins (bridge, macvlan, ipvlan, ...) into /opt/cni/bin on generic-Linux installs, where nodes do not otherwise provide them. On by default on every platform: Talos ships five of the fourteen, and the four that overlap are rewritten with the same upstream version. On generic clusters this replaces same-named files in /opt/cni/bin every time the Multus pod is recreated, so a locally managed plugin there will not survive. Set networking.stageCniPlugins: false to decline ([**@lexfrei**](https://github.com/lexfrei) in #3195).
* **feat(kubernetes)!: manage worker pools as KubernetesNodes, adopt on upgrade (Phase 2, breaking)**: Worker node pools of a managed Kubernetes cluster are now managed as separate `KubernetesNodes` resources. BREAKING: `spec.nodeGroups`, `spec.nodeHealthCheck` and `spec.maxNodeProvisionTime` are removed from the `Kubernetes` CR; pool configuration, worker health checks, and the autoscaler provision-time budget move to `KubernetesNodes`, on which the API accepts an edit of `spec.cluster` but the pool's release then refuses to render it, and `spec.storageClass` is meant to stay fixed: the dashboard does not let it change, but the API accepts an edit, and the pool then rolls its workers onto the new class. A `Kubernetes` cluster name is now capped at 32 characters ([**@IvanHunters**](https://github.com/IvanHunters) in #3315).
* **feat(gateway)!: add TLS-passthrough listeners to TenantGateway**: TenantGateway supports layer-4 TLS-passthrough listeners on native database ports for SNI-based external exposure. On HTTP-01 gateways a hostname a TLS-passthrough listener answers, including any name beneath a wildcard entry, no longer earns an HTTPS-terminate listener or a Gateway-issued certificate once a TLSRoute claiming that hostname can be served on the listener and forwards to a backend that resolves; a hostname claimed only by a TLSRoute stops earning one too. No chart value renders `spec.tlsPassthroughListeners` yet, so such a listener is set on the `TenantGateway` resource by hand. An existing TLSRoute whose hostname no passthrough listener answers stays unattached after the upgrade; remove it, or declare a `tlsPassthroughListeners` entry for its hostname ([**@lexfrei**](https://github.com/lexfrei) in #3342).
* **feat(kubernetes)!: remove FluxCD addon**: Remove the FluxCD addon (`addons.fluxcd`). It relied on AGPL-3.0 vendored charts and was disabled by default; tenant clusters that had enabled it keep their running Flux release via an automatic upgrade migration. To remove that orphaned Flux from such a tenant, uninstall its `flux-operator`/`flux-instance` Helm releases inside the tenant cluster by hand ([**@kvaps**](https://github.com/kvaps) in #3379).
* **feat(api)!: require the application release prefix to be dot-free**: `ApplicationDefinition.spec.release.prefix` must match `^[a-z0-9-]*$` (dot-free). A create, or an update that sets a new prefix, is rejected at admission when the prefix contains a dot or any other character, while an existing definition whose prefix is left unchanged is not revalidated; correct it before applying this CRD ([**@lexfrei**](https://github.com/lexfrei) in #3411).
* **fix(gateway)!: require tenant routes to declare their hostnames**: The tenant route hostname policy now requires an HTTPRoute or TLSRoute in a `tenant-*` namespace to declare a non-empty `spec.hostnames`. A route that omitted the field, or set it to an empty list, was previously admitted without any hostname being checked against the tenant apex, even though such a route inherits the hostnames of the listener it attaches to rather than matching nothing. A route in a `tenant-*` namespace relying on that inheritance is now rejected and has to name its hostnames ([**@lexfrei**](https://github.com/lexfrei) in #3475).
* **feat(postgres,mariadb)!: drop plaintext user passwords**: Passwords for managed PostgreSQL and MariaDB users can no longer be set via `users[].password` — they are always auto-generated and stored only in the `<release>-credentials` Secret, never rendered into the init-script or the SQL it runs. Existing releases keep their current passwords on upgrade; a `password` left in values is ignored with an admission warning but stays the live credential until rotated. Password rotation is deferred to a follow-up controller design ([**@scooby87**](https://github.com/scooby87) in #4078).
* **chore(platform)!: remove the HTTPCache application**: Remove the HTTPCache application from the catalog. New instances can no longer be created from the catalog and the `HTTPCache` values type is gone from the `api/apps/v1alpha1` Go module. A platform migration keeps existing instances working: on a cluster that runs one, the application is frozen on the packages artifact that cluster was already rendering from, pinned by digest, and stays manageable through the Cozystack API and the dashboard, but receives no further updates, including security updates to the nginx build. On a cluster with no instance the application is removed outright ([**@kvaps**](https://github.com/kvaps) in #4200).
* **feat(kamaji)!: update to 26.9.4-edge and read every apiServer extraArgs entry**: Update Kamaji to 26.9.4-edge. Tenant kube-apiserver arguments are now derived from a hash of the control plane's extraArgs instead of from its length, so on the first reconcile after the upgrade every tenant control plane rewrites those arguments from scratch and rolls once. The rebuilt list also orders them differently, Kamaji's own flags sorted first and the operator's kept verbatim after them, except the flags Kamaji manages itself, so the roll happens whether or not the tenant sets any extraArgs. A tenant with OIDC enabled takes a second roll when the chart's own flags land ([**@lexfrei**](https://github.com/lexfrei) in #4398).
* **fix(kubevirt)!: evict non-migratable VMs instead of blocking the drain**: VMs that cannot live-migrate no longer block a node drain. The cluster-wide eviction strategy is now LiveMigrateIfPossible instead of LiveMigrate. VMs that can migrate still migrate. VMs that cannot, for example with a ReadWriteOnce disk or GPU or host device passthrough, are shut down and started again by their runStrategy. This applies to any pod eviction, not only a drain. A VM whose disk is bound to the drained node stays down until the node returns ([**@lexfrei**](https://github.com/lexfrei) in #4403).
* **fix(vm-instance)!: refuse resources that cannot size the VM**: Resources.cpu and resources.sockets must now be positive whole numbers, and resources.memory a positive quantity. A cpu or sockets that is not a plain whole number within uint32 (for example 500m, 1k or 1.5), or a zero or negative memory, now fails at render time with an error naming the field, instead of silently becoming a one-core VM or a zero memory request. A release carrying such a value fails its next upgrade until the value is fixed. A value such as 1e1 or 2., which used to be truncated to one core, now sizes the VM with the count it spells ([**@lexfrei**](https://github.com/lexfrei) in #4405).
* **feat(kubevirt)!: drop HotplugVolumes for declarative volume hotplug**: KubeVirt now runs with DeclarativeHotplugVolumes instead of the deprecated HotplugVolumes feature gate, and kubevirt-csi no longer uses the VMI-level hot-unplug. Volume hotplug requests are applied through VM.spec. The upgrade copies volumes that older kubevirt-csi versions attached only to the VMI into VM.spec, which keeps them attached on tenant Kubernetes workers; on a `VMInstance` the same upgrade re-renders the VM from its chart and KubeVirt unplugs the copied volume (see pre-upgrade item 5). The check in the PR description is optional ([**@lexfrei**](https://github.com/lexfrei) in #4453).
* **fix(foundationdb-operator)!: drop the amd64-only 7.1 init container**: The FoundationDB operator now starts on arm64. It no longer ships the 7.1 client, because no arm64 build of it exists, so it cannot manage or upgrade a FoundationDB cluster on 7.1.x. Upgrade such clusters to 7.3 before installing this release ([**@lexfrei**](https://github.com/lexfrei) in #4512).
* **fix(vm-instance)!: refuse guest memory below KubeVirt's 1M floor**: Refuse `resources.memory` below 1M at render time, since KubeVirt does not admit a VM with less. A value above zero and below 1M rendered before and is now refused. The cpu, sockets and memory descriptions list the refused values, and the guard error marks an unquoted number as read rather than written ([**@lexfrei**](https://github.com/lexfrei) in #4569).
* **feat(talos)!: bump Talos to v1.14.2**: Bump Talos to v1.14.2, which ships DRBD 9.3.4 and ZFS 2.4.4. DRBD 9.3.4 fixes a sender thread spinning on a dead connection, two-phase-commit hangs and a data divergence after a reconnect. Before upgrading a node, add `drbd_transport_tcp` to its Talos `machine.kernel.modules` next to `drbd`: Talos v1.14.2 no longer loads it on demand, and without it DRBD cannot connect to any peer. We recommend skipping the v1.13 line and upgrading nodes straight to v1.14.2. Talos accepts a direct upgrade from v1.12.0 or later ([**@lexfrei**](https://github.com/lexfrei) in #4598).
* **feat(gateway)!: reserve a passthrough listener's name by the spec alone**: The tenant Gateway renders the api, vm-exportproxy and cdi-uploadproxy passthrough listeners only on the publishing tenant, for the names publishing.exposedServices lists while gateway.enabled is on. Under certMode http01 a name a TLS-passthrough listener answers no longer gets an HTTPS listener on 443, with or without a TLSRoute behind it. On the publishing tenant, an HTTPRoute on such a name whose TLSRoute forwards nowhere loses its listener and certificate. A TLSRoute attached by hand to a tls-* listener of another tenant loses its parent ([**@lexfrei**](https://github.com/lexfrei) in #4638).
* **feat(redis)!: refuse external access without TLS at admission**: A Redis release can be exposed outside the cluster only when it is created with `tls.enabled: true`. Creating one with `external: true` and no TLS, turning `external` on for an existing instance, or changing `tls.enabled` while `external` is on is refused at admission. Instances already external without TLS keep working and stay editable ([**@lexfrei**](https://github.com/lexfrei) in #4639).
* **fix(cilium)!: take cilium's own addresses from a block kube-ovn does not hand out**: On the kube-ovn variants cilium now takes its router and gateway api ingress ips from the last /21 of networking.podCIDR, which kube-ovn no longer hands out, so pods stop landing on addresses cilium holds. existing clusters move on upgrade through a post-upgrade hook, and each cilium agent restarts twice, once for the config change and once in the hook (clusters on v1.6.5 already moved; the restarts apply when upgrading from v1.6.4 or earlier). networking.podCIDR needs /20 or wider on these variants, and a /21 of /29s caps them at 256 nodes ([**@yankawai**](https://github.com/yankawai) in #4778, shipped in v1.6.5).
* **fix(postgres)!: refuse plaintext sessions on external instances**: An external Postgres now refuses sessions that are not encrypted. The chart renders `hostssl` rules for all users and databases with scram-sha-256 (md5 on PostgreSQL 13, matching CloudNativePG's default rule) and rejects plain `host` connections, so a client using `sslmode=disable` is refused. The new `tls.required` value follows `external` by default; set it to `false` to keep accepting plaintext sessions ([**@yrahman82**](https://github.com/yrahman82) in #4847).
* **fix(platform)!: refuse a networking variant change on a live cluster**: The platform render now refuses to change the variant of a live `cozystack.networking` Package, because the operator would then delete the CNI HelmReleases the new variant does not install (`isp-full` to `isp-hosted` removes Cilium and Kube-OVN). To go ahead, set `networking.acknowledgeVariantChange` to the new networking variant named in the error. A cluster whose variant was set by hand, such as `cilium-kilo`, needs `cozystack.networking` in `bundles.disabledPackages` before the upgrade ([**@lexfrei**](https://github.com/lexfrei) in #4790, backport #5021).
* **fix(vm-disk)!: refuse a golden-image clone CDI would leave unbound**: A new disk from a golden image is refused with a clear error when CDI would CSI-clone it into another LINSTOR StorageClass that does not allow remote volume access from every node, instead of staying pending forever; a clone that would have landed on a node already holding a replica of the golden image is refused as well. Use the golden image's StorageClass, or have the platform administrator set the CDI `cloneStrategyOverride` to `copy` in the kubevirt-cdi package ([**@lexfrei**](https://github.com/lexfrei) in #4796, backport #5022).
* **fix(kubernetes, opensearch, foundationdb)!: restrict managed application images**: Image overrides let tenants run arbitrary containers in the management cluster, including helpers with access to Talos CA secrets and Kubernetes API credentials, so the Kubernetes, KubernetesNodes and OpenSearch charts stop reading their `images.*` values and use the images the platform ships. Stored overrides are accepted without an admission warning and ignored; on the first reconcile OpenSearch nodes move to the image of their selected version and Kubernetes components whose images changed restart. FoundationDB replaces `cluster.version` with `version` (`v7.3` or `v7.4`, mapped to 7.3.63 and 7.4.1); migration 60 carries the release line over and records the releases that need attention, and the chart refuses a render whose `cluster.version` and `version` name different lines. An OpenSearch release that stores `version: null` is refused instead of rendering `v2` ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5085, backport #5108).
## Major Features and Improvements
### Kubernetes and worker pools
* **feat(computeplane): provision a ComputePlane tenant module for untrusted-code apps**: ComputePlane tenant module — a single-tenant, Cozystack-managed cluster for untrusted-code applications, delivered as an operator-owned extra module wrapping the kubernetes app ([**@kvaps**](https://github.com/kvaps) in #3280).
* **feat(kubernetes): per-pool kernel modules for Talos worker nodes**: Worker node pools (`KubernetesNodes`) accept a `kernelModules` field, emitted as Talos `machine.kernel.modules`. A Talos system extension installs a kernel module without loading it, so a GPU pool previously came up with no working driver and no way to fix it through values ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3571).
* **feat(kubernetes): route tenant worker image pulls through a registry mirror**: Add `talos.registryMirrors` to route tenant worker node image pulls through a registry mirror for air-gapped or flaky-egress environments. The key is new in this release and takes effect only on `KubernetesNodes`, so set it on each pool; the `Kubernetes` resource accepts the same key, but no template of the `kubernetes` chart reads it ([**@IvanHunters**](https://github.com/IvanHunters) in #3575).
* **feat(kubernetes): default gpu-operator to an OS-provided driver on Talos**: The GPU Operator addon of tenant Kubernetes clusters now defaults to a driver provided by the operating system, which is the shape of the Talos workers these clusters have run since 1.6.0 ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3585).
* **feat(kubernetes): let a node group log the guest serial console**: Worker node groups accept `logSerialConsole`, which streams a worker VM's guest serial console into a `guest-console-log` container readable with `kubectl logs`. Off by default ([**@lexfrei**](https://github.com/lexfrei) in #3637).
* **feat(kubernetes-nodes): let CDI trust a self-hosted Talos image factory**: Add `talos.imageFactoryCA` so CDI can verify a self-hosted Talos Image Factory served with a certificate from an internal CA. Previously the worker image import failed with `x509: certificate signed by unknown authority` ([**@batonogov**](https://github.com/batonogov) in #3797).
* **feat(kubernetes): let a node group set the CPU limit of its worker VMs**: A node group can now set `podCpuLimit`, the CPU ceiling of its worker VMs, independently of the vCPU count that sizes the guest. Without it KubeVirt keeps deriving that ceiling from the vCPU count, which leaves the QEMU emulator and IO threads sharing the guest's own allowance ([**@lexfrei**](https://github.com/lexfrei) in #3859).
* **feat(kubernetes): let a node group set the CPU request of its worker VMs**: A node group can now set `podCpuRequest`, the CPU request of its worker VMs, next to `podCpuLimit`. Without it KubeVirt copies a declared CPU limit into the request, so raising a worker's CPU ceiling also raised what it asked the scheduler for and could leave the group unschedulable ([**@lexfrei**](https://github.com/lexfrei) in #3862).
* **feat(cluster-api): add the Proxmox infrastructure provider and in-cluster IPAM as optional packages**: Add optional Cluster API packages for Proxmox — the capmox infrastructure provider and the in-cluster IPAM provider it requires. Neither is deployed unless named in `bundles.enabledPackages`, since capmox drives a hypervisor outside the cluster and does nothing without Proxmox API credentials ([**@themoriarti**](https://github.com/themoriarti) in #4087).
* **feat(kubernetes): add a proxmox substrate for managed clusters and worker pools**: Add a `substrate` value to the `kubernetes` and `kubernetes-nodes` applications, selecting whether worker VMs run on KubeVirt in this cluster (`kubevirt`, the default and unchanged) or on an external Proxmox VE cluster through capmox (`proxmox`) ([**@themoriarti**](https://github.com/themoriarti) in #4088).
* **feat(kubernetes): storage and cloud-controller integration for Proxmox-backed tenants**: Add the Proxmox CSI driver and cloud-controller-manager for tenant clusters on the `proxmox` substrate, giving them PersistentVolumes backed by Proxmox storage ([**@themoriarti**](https://github.com/themoriarti) in #4089).
* **feat(kubernetes): boot tenant worker disks from a shared golden Talos image (CDI clone)**: Tenant worker pools can boot their system disk from a shared golden talos image instead of importing it from the Image Factory once per worker ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4171).
* **feat(kubernetes): let kubevirt-csi detach without the HotplugVolumes gate**: Kubevirt-csi supports --enable-vmi-hotplug-fallback, so it can detach volumes without the KubeVirt HotplugVolumes feature gate. The flag is set to false together with dropping `HotplugVolumes` (#4453) ([**@lexfrei**](https://github.com/lexfrei) in #4451).
* **feat(kubernetes-nodes): build the Proxmox worker template from the pool's Talos values**: Add `hack/proxmox-talos-template.sh`, which builds the Proxmox VM template a `substrate: proxmox` pool clones from the pool's own `talos.version` and `talos.schematicID` in the nocloud flavour, with the guest agent the default schematic needs ([**@themoriarti**](https://github.com/themoriarti) in #4742).
### Virtualization and storage
* **feat(local-storage): add opt-in OpenEBS LocalPV hostpath package**: Add opt-in cozystack.local-storage package that installs OpenEBS LocalPV hostpath and provides the cozystack-local-hostpath StorageClass, it is not enabled by any built-in variant and is not the default StorageClass ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3266).
* **feat(migration): tenant self-service VM import from VMware vSphere**: Import virtual machines from VMware vSphere into a tenant. A VMImportSource registers a connection to vCenter and a VMImportTask imports named machines, producing ordinary VMDisks and VMInstances that survive deletion of the import ([**@kvaps**](https://github.com/kvaps) in #3931).
* **feat(kubevirt): expose migration configuration through platform values**: Cluster-wide KubeVirt migration settings (bandwidth per migration, parallel migrations per cluster, parallel outbound migrations per node) can now be set through the platform value `kubevirt.migrations` instead of hand-patching the generated `Package`, which the platform undoes on every render; leaving the value unset preserves existing defaults ([**@yankawai**](https://github.com/yankawai) in #4254, shipped in v1.6.4).
* **feat(vpn): build shadowbox for amd64 and arm64**: The VPN app now uses a shadowbox image that Cozystack builds from the upstream v1.12.3 source, pinned by digest, instead of the floating amd64-only `:stable` upstream tag. The image builds for amd64 and arm64; releases publish the arm64 variant once the multi-arch release build is in place ([**@lexfrei**](https://github.com/lexfrei) in #4521).
* **feat(seaweedfs): alert before an instance stops taking writes**: Alert when a SeaweedFS volume server nears its free-space reserve, cannot host another volume, or keeps failing to place writes, and document how to size an instance for continuous WAL archiving ([**@scooby87**](https://github.com/scooby87) in #4573).
### Databases and messaging
* **[kafka] Migrate from ZooKeeper to KRaft mode**: Enable KRaft metadata mode with automatic, in-place ZooKeeper-to-KRaft migration. Kafka version is now configurable (v3.9, v3.8). The Strimzi operator is unchanged (0.45.1-rc1) ([**@matthieu-robin**](https://github.com/matthieu-robin) in #2352).
* **feat(mariadb): add TLS support via cert-manager**: Chart-managed TLS via cert-manager, opt-in enforcement, and the CA published to tenants as a key-free trust anchor ([**@Arsolitt**](https://github.com/Arsolitt) in #2680).
* **feat(rabbitmq): add TLS support for AMQPS and management HTTPS**: Opt-in TLS for AMQP and the management API, with the CA published to tenants as a key-free trust anchor. Enabling TLS moves the metrics port from 15692 to 15691 (ServiceMonitors selecting the port named `prometheus` must be updated) and moves the `port` value in the `<release>-default-user` binding Secret from 5672 to 5671 ([**@Arsolitt**](https://github.com/Arsolitt) in #2683).
* **feat(mongodb): add TLS support via PSMDB operator-managed certificates**: Stop exposing TLS private key material to tenants and publish the CA as a key-free trust anchor ([**@Arsolitt**](https://github.com/Arsolitt) in #2692).
* **feat(redis): add opt-in TLS for Redis and Sentinel**: Add opt-in TLS for Redis and Sentinel. Set `tls.enabled: true` to serve both on a TLS-only listener backed by a per-release cert-manager chain; the tenant reads the CA certificate as the `<release>.tenant-ca` tenant secret ([**@Arsolitt**](https://github.com/Arsolitt) in #2729).
* **feat(keycloak): expose organizationsEnabled realm toggle**: Add `organizationsEnabled` value to the keycloak-configure chart to enable the Keycloak Organizations feature on the platform realm (default: false) ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3031).
* **feat(nats): publish ca.crt-only trust anchor to tenants**: Expose the CA certificate to tenants as a key-free `<release>.tenant-ca` Secret ([**@lexfrei**](https://github.com/lexfrei) in #3340).
* **feat(qdrant): publish ca.crt-only trust anchor to tenants**: Expose the CA certificate to tenants as a key-free `<release>.tenant-ca` Secret ([**@lexfrei**](https://github.com/lexfrei) in #3341).
* **feat(valkey): add managed Valkey service**: Add managed Valkey service — a BSD-3-Clause, Linux Foundation-governed alternative to managed Redis, running on the freshworks-oss redis-operator via native spec.engine: Valkey (versions 8.1.8 and 7.2.13) ([**@kvaps**](https://github.com/kvaps) in #3380).
* **feat(clickhouse): let the backup sidecar trust a private S3 CA, unblocking the backup e2e round-trip**: Add `backup.endpointCA` so the clickhouse-backup sidecar can verify an S3 endpoint whose certificate is signed by a private CA, such as Cozystack's in-cluster SeaweedFS. Leave it unset for publicly-trusted endpoints; existing releases are unaffected ([**@androndo**](https://github.com/androndo) in #3385).
* **feat(redis-operator): migrate from archived spotahome to freshworks-oss fork**: Migrate from the archived spotahome/redis-operator to the actively-maintained freshworks-oss/redis-operator fork (same RedisFailover CRD, adds native spec.engine: Redis|Valkey) ([**@kvaps**](https://github.com/kvaps) in #3406).
* **feat(clickhouse): add configurable version parameter**: The ClickHouse server and Keeper images were hardcoded to `24.9.2.42`, so — unlike postgres, mariadb, mongodb, opensearch, rabbitmq and redis — ClickHouse instances carried no engine `version` in the Cozystack API ([**@scooby87**](https://github.com/scooby87) in #3476, shipped in v1.6.3).
* **feat(postgres): add per-user createRole and bypassRls attributes**: Add `users.<name>.createRole` and `users.<name>.bypassRls`, which grant the CREATEROLE and BYPASSRLS attributes to a managed user so a tenant can create and manage its own roles, including roles that bypass row-level security, without the postgres superuser ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3505).
* **feat(postgres): allow preloading an allowlist of shipped modules**: Add `postgresql.sharedPreloadLibraries`, a closed allowlist of modules shipped in the PostgreSQL image (`pg_stat_statements`, `auto_explain`, `pgaudit`, `pg_prewarm`) that are preloaded at server start ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3506).
* **feat(kafka): add topic-data backup/restore example and e2e roundtrip**: Add a topic-data backup/restore example and e2e round-trip on the generic Job backup strategy (data-only: topic records with a frozen-end-offset consistency cut and partition-preserving replay; consumer offsets, topic configs, ACLs and users are out of scope) ([**@androndo**](https://github.com/androndo) in #3580).
* **feat(keycloak): let the KMS proxy trust a private Vault CA**: The KMS-encrypting DB proxy could previously only reach a Vault whose certificate chains to a publicly trusted root, so a Vault behind an internal load balancer with a self-signed certificate was reachable only over plain HTTP ([**@sircthulhu**](https://github.com/sircthulhu) in #3874, shipped in v1.6.3).
* **feat(postgres): database read-replica autoscaling via KEDA**: Add optional horizontal autoscaling of read replicas via KEDA, configured through the new `autoscaling` values block; KEDA ships as a new optional platform component. Requires CloudNativePG >= 1.28.4 for the Cluster scale subresource selector ([**@scooby87**](https://github.com/scooby87) in #3954).
* **feat(mongodb): support the system-bucket backup flow (useSystemBucket)**: MongoDB backups now support the platform system bucket via backup.useSystemBucket, so the default cozy-default BackupClass works without per-app S3 credentials — the backup driver injects the system-bucket storage onto the cluster at backup time, once the release has rendered the flag ([**@androndo**](https://github.com/androndo) in #3967).
* **feat(postgres-operator): alert when the WAL archive backlog does not drain**: Page when a Postgres cluster's WAL archive queue has not drained for 30 minutes, which the existing archiving alert misses while some archives still succeed ([**@scooby87**](https://github.com/scooby87) in #4572).
* **feat(postgres): add a postgis image flavor**: Add a `flavor` field accepting `postgresql` (default) or `postgis`. `postgis` bootstraps the cluster from the CloudNativePG PostGIS image, so PostGIS extensions can be enabled per database through `databases.<name>.extensions` ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4587).
* **feat(keycloak): expose zone placement for Keycloak and the database proxy**: Expose affinity and topologySpreadConstraints for Keycloak and the keycloak-kms-proxy, and add a PodDisruptionBudget for the proxy when it runs more than one replica. Defaults are unchanged ([**@sircthulhu**](https://github.com/sircthulhu) in #4606, shipped in v1.6.5).
* **feat(redis,postgres): certify the per-engine external hostname**: Server certificates of external redis and postgres releases also carry the `<release>.<engine>.<apex>` name that the tenant Gateway's TLS-passthrough listeners are designed to route by. Existing external TLS releases get their server certificate reissued once on upgrade, from the same CA ([**@lexfrei**](https://github.com/lexfrei) in #4627).
* **feat(openbao): optional static-key auto-unseal and apiserver egress label**: The OpenBAO app can auto-unseal with OpenBAO's static-key seal (`seal.type: static`) from a Secret that a cluster administrator creates in the tenant namespace, with n-1 rotation through `seal.previousSecretName` and `seal.previousKeyId` ([**@yankawai**](https://github.com/yankawai) in #4766, shipped in v1.6.5).
* **feat(redis): run Sentinel on the same image as the Redis pods**: Sentinel pods now run the same Redis image as the data pods (redis:8.4.0 for v8, redis:7.4.7 for v7) instead of the end-of-life redis:6.2.6-alpine. Existing instances roll their Sentinels and their data pods once on upgrade (the exporter sidecar image changes as well, #4522) ([**@yrahman82**](https://github.com/yrahman82) in #4831).
* **feat(redis): size the Sentinel pods separately from Redis**: New optional `sentinel.resources` and `sentinel.resourcesPreset` parameters size the three Sentinel pods independently of the Redis pods. When unset, Sentinels keep the Redis sizing as before, so existing instances are unchanged ([**@yrahman82**](https://github.com/yrahman82) in #4832).
* **feat(redis): expose min-replicas-to-write and min-replicas-max-lag**: Add optional minReplicasToWrite and minReplicasMaxLag values to bound writes lost on failover; unset by default ([**@yrahman82**](https://github.com/yrahman82) in #4848).
### Backups and restore
* **feat(backups): support PostgreSQL point-in-time recovery (PITR)**: PostgreSQL RestoreJob now supports point-in-time recovery (PITR). Set spec.options.recoveryTime (RFC3339) to restore to an exact instant ([**@androndo**](https://github.com/androndo) in #3383).
* **feat(backups): add a native MongoDB (psmdb) backup-strategy driver**: Add a native MongoDB (psmdb) backup-strategy driver — logical backup to the cozy-default BackupClass, restore into a differently-named instance, and point-in-time recovery via spec.options.recoveryTime ([**@androndo**](https://github.com/androndo) in #3562).
* **feat(backups): RabbitMQ definitions backup via a dedicated strategy driver**: Add RabbitMQ definitions backup/restore via a dedicated strategy.backups.cozystack.io/Rabbitmq driver; it gates each run on the RabbitmqCluster being ready and records the stored object on Backup.status.artifact.uri ([**@androndo**](https://github.com/androndo) in #3990).
* **feat(backups): add Redis backup/restore strategy driver**: Add backup and restore support for the Redis application (spotahome RedisFailover) via a new Redis backup strategy, wired into the cozy-default BackupClass. Supports in-place and to-copy restore; PITR is not supported ([**@androndo**](https://github.com/androndo) in #3992).
* **feat(backups): Kafka topic-metadata backup via a dedicated strategy driver**: Add a Kafka topic-metadata backup strategy bound by cozy-default — backs up topic definitions and configs (partitions, replication factor, non-default configs) via the Kafka Admin API, with in-place and to-copy restore; message data, consumer offsets, ACLs, quotas and KafkaUsers are out of scope ([**@androndo**](https://github.com/androndo) in #4000).
* **feat(backups): back up Valkey with the Redis strategy**: Valkey apps can now be backed up and restored. The platform `cozy-default` BackupClass routes `apps.cozystack.io/Valkey` to the existing `cozy-default-redis` strategy, which dumps an RDB snapshot to the system bucket and restores it in place or into another Valkey app; a Valkey backup restores only into a Valkey app ([**@androndo**](https://github.com/androndo) in #4898, backport #5032).
### Networking and gateway
* **feat(gateway): advertise HTTP/2 via ALPN on Gateway API listeners**: Gateway API listeners now advertise HTTP/2 via TLS ALPN (h2, then http/1.1) by default, restoring the browser HTTP/2 support lost in the migration from ingress-nginx. Opt out platform-wide with gateway.http2=false, or per tenant Kubernetes cluster via addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false ([**@lllamnyp**](https://github.com/lllamnyp) in #3398).
* **feat(kube-ovn): build the chart and image in-tree, retiring the wrapper repo**: The Kube-OVN chart and image are now built in this repository instead of cozystack/kubeovn-chart. OVN_IPSEC_KEY_DIR now defaults to /var/lib/ovs_ipsec_keys, so enabling IPsec on Talos nodes no longer leaves them without ESP security associations ([**@kvaps**](https://github.com/kvaps) in #3460, shipped in v1.6.4).
* **feat(gateway): let a tenant pick its Gateway class, and follow that class for TLS termination**: Tenant Gateways can now run on a GatewayClass other than the bundled Cilium one. The platform sets the default in `gateway.className` and bounds tenant choice with `gateway.tenantSelectableClasses`; a tenant selects its own with `tenant.spec.gatewayClass` ([**@lexfrei**](https://github.com/lexfrei) in #3861).
* **feat(vpc): expose allowSubnets per subnet**: Subnets accept an optional `allowSubnets` list of CIDRs admitted into the private subnet, so a VM on the subnet can act as a VPN or router gateway without a manual patch of the kube-ovn Subnet ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4478).
### Monitoring and observability
* **feat(monitoring): auto-route tenant logs using namespace labels**: Container logs from a tenant that runs its own monitoring stack now go to that tenant's VictoriaLogs instead of the root one. The node fluent-bit routes them by the namespace.cozystack.io/monitoring label, and cozystack-controller re-renders the node agents when that label changes on any namespace ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2195).
* **feat(monitoring): add KubeVirt VM dashboard**: Add a KubeVirt virtual-machines dashboard ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2198).
* **feat(talos-log-collector): add talos-log-collector package**: Add talos-log-collector package to collect Talos machine and kernel logs into VictoriaLogs, opt-in through `bundles.enabledPackages` ([**@IvanHunters**](https://github.com/IvanHunters) in #3260).
* **feat(monitoring): drop the Grafana rebuild for the upstream image + catalog plugins**: Grafana now runs on the upstream `grafana/grafana` image; the VictoriaLogs datasource and panel plugins are mirrored from the official Grafana catalog into the in-cluster `grafana-dashboards` server and installed at startup, so air-gapped installs keep working ([**@kvaps**](https://github.com/kvaps) in #3378).
* **feat(workloadmonitor): bucket metrics endpoint override**: `cozystack-controller` meters S3 bucket sizes for billing by discovering a VictoriaMetrics endpoint through a namespace label, which assumed the monitoring stack scraping SeaweedFS always runs in the same cluster ([**@lllamnyp**](https://github.com/lllamnyp) in #3443, shipped in v1.6.3).
* **feat(monitoring): VictoriaTraces backend + Grafana datasource (tracing phase 1)**: Add a VictoriaTraces backend and a Grafana traces datasource to the monitoring stack (distributed tracing, phase 1). Tracing is off by default; enable it per tenant by adding a `tracingStorages` entry (`cluster` or `single` mode) ([**@scooby87**](https://github.com/scooby87) in #3762).
* **feat(monitoring): per-tenant OTLP collector (tracing phase 2)**: Deploy a per-tenant OpenTelemetry (OTLP) collector for traces once a VictoriaTraces backend is configured ([**@scooby87**](https://github.com/scooby87) in #3763).
* **feat(monitoring,tenant): shared-central tracing topology (phase 4)**: Add an opt-in shared-central traces store. A tenant sets tracingCentral to send its spans to one VictoriaTraces store in tenant-root instead of its own, and tenant-root opts in to hosting it with tracingCentralHost on its Monitoring ([**@scooby87**](https://github.com/scooby87) in #3775).
* **feat(monitoring): add optional email receiver to alertmanager**: Add optional email alerting through alerta.alerts.email, with configurable severity and alert-name selection and an SMTP password read from the fixed Secret alertmanager-email-password ([**@yankawai**](https://github.com/yankawai) in #3800, shipped in v1.6.5).
* **feat(monitoring): hold each shared-central tenant to a write rate**: Hold each shared-central tracing tenant to a write rate. The platform sets the rate with monitoring.tracingCentralTenantBytesPerSecond, and each central tenant's collector refuses spans above it, which bounds what one tenant writes into the store every tenant shares ([**@scooby87**](https://github.com/scooby87) in #4648).
### Dashboard, API and marketplace
* **feat(api): add configurable resources and replicas for cozystack-engine components**: The cozystack-engine charts gain `resources` (requests/limits) values for cozystack-api, cozystack-controller and lineage-controller-webhook, and a `cozystackController.replicas` value for cozystack-controller (default: 1) ([**@2mind**](https://github.com/2mind) in #2956).
* **feat(dashboard): rework the Marketplace entry point and the Admin portal**: The Console is now the dashboard's entry point instead of the Marketplace, so opening the dashboard shows what a tenant already runs rather than a "choose a product" landing page; the Marketplace opens the grouped application list directly, one click away ([**@kvaps**](https://github.com/kvaps) in #3928, shipped in v1.6.3).
* **feat(marketplace): community marketplace for External-Apps repositories**: Add a community marketplace for External-Apps repositories — cozypkg validate/init/push/tap/untap/search, a cluster-scoped Tap API in core.cozystack.io with an operator-side artifact materializer, keyless cosign verification (run by `cozypkg validate --require-signature`; `cozypkg tap` checks the artifact's structure but not its signature), a two-lane publication gate, and a dashboard Repositories view ([**@IvanHunters**](https://github.com/IvanHunters) in #3940).
* **feat(dashboard): show a connection hostname for LoadBalancer services**: The application Services tab shows `<service>.<suffix>` under the external IP of LoadBalancer services when the new platform value `publishing.serviceDomain` is set ([**@lexfrei**](https://github.com/lexfrei) in #4035).
* **feat(dashboard): brand the token-proxy login page from the console config**: The token-proxy login page now renders the deployment's branding (title and logo from `branding.titleText` / `branding.logoText` / `branding.logoSvg`) instead of the hardcoded default, so unauthenticated users on non-OIDC clusters see a branded sign-in screen ([**@scooby87**](https://github.com/scooby87) in #4255).
* **feat(dashboard): serve console branding without kube-api access**: The console serves its branding as a static asset (`/branding/config.json`) read with no kube-api call, so branding renders even when the apiserver does not authenticate the session and without the console-config-reader RBAC; the served file is narrowed to the documented white-labeling keys ([**@scooby87**](https://github.com/scooby87) in #4256).
* **feat(dashboard): move Repositories from the Marketplace to Admin**: The Repositories view moved from the Marketplace sidebar to Admin, under Administration. The old /marketplace/taps URL redirects to /admin/taps ([**@kvaps**](https://github.com/kvaps) in #4286).
* **feat(marketplace): register a tapped repository's apps on connect**: Tapping a marketplace repository now registers its applications in the catalog on connect, without a manual `cozypkg add`. #5071 removes that on-connect registration again within this release: connecting a repository installs nothing, and `cozypkg add` installs its applications ([**@IvanHunters**](https://github.com/IvanHunters) in #4308).
* **feat(dashboard): serial console for virtual machines, and paste into the VNC console**: Virtual machines gain a serial console tab, where pasting is instant and the output can be selected and copied ([**@IvanHunters**](https://github.com/IvanHunters) in #4356).
* **feat(api): take Application name limits from the application's own schema**: Application name limits now come from the application's own schema instead of being hardcoded per kind in the aggregated API server ([**@lllamnyp**](https://github.com/lllamnyp) in #4367).
* **feat(dashboard): show categories brought by tapped repositories in the sidebars**: Categories brought by tapped repositories now appear in the marketplace and console sidebars, after IaaS, PaaS and NaaS, instead of only in the full application list ([**@tym83**](https://github.com/tym83) in #4741).
### Platform and installer
* **feat(scheduler): set scheduling-class label on mutated pods**: The lineage-controller-webhook now sets a scheduler.cozystack.io/scheduling-class label on mutated pods (in addition to the existing annotation), enabling SchedulingClass authors to write podAffinity rules that match pods across applications ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2621).
* **feat(platform): add opt-in in-cluster east-west encryption toggle**: Add opt-in `networking.encryption.enabled` (off by default) that enables in-cluster east-west transport encryption for the default KubeOVN datapath via overlay IPsec; complements, and does not replace, application-level TLS ([**@lexfrei**](https://github.com/lexfrei) in #2984).
* **feat(controller): expose bucket class on BucketClaim Workloads**: Expose bucket class on BucketClaim Workloads via the workloads.cozystack.io/bucket-class label ([**@harshitrwt**](https://github.com/harshitrwt) in #3501).
* **feat(platform): make the Keycloak realm name configurable**: The Keycloak realm name is now configurable via `authentication.oidc.realmName` (default `cozy`), threaded consistently through the apiserver OIDC issuer, dashboard, linstor-gui, Grafana, the managed-Kubernetes apiserver and the generated kubeconfig. Set it to the same realm as the talm `oidcIssuerUrl` ([**@scooby87**](https://github.com/scooby87) in #4237).
* **feat(build): build package images for amd64 and arm64**: Package images build for linux/amd64 and linux/arm64 by default, with Go builders cross-compiling on the build platform; a local `make image` now needs a docker-container builder, or PLATFORM=linux/amd64 / LOAD=1 for a single architecture. CI still publishes amd64 ([**@lexfrei**](https://github.com/lexfrei) in #4498).
* **feat(system): update ingress-nginx, cozy-proxy, cozystack-scheduler and keycloak-kms-proxy to multi-arch releases**: Ingress-nginx v1.11.6, cozy-proxy v0.4.0, cozystack-scheduler v0.4.0 and keycloak-kms-proxy 0.2.3 now ship linux/arm64 images ([**@lexfrei**](https://github.com/lexfrei) in #4549).
* **feat(build): build Talos, Harbor, the Velero KubeVirt plugin and the unlisted images for arm64**: The Talos installer image is built for amd64 and arm64, with arm64 profiles that carry drbd and zfs, and the bootbox matchbox image network-boots arm64 machines as well as amd64 ones ([**@lexfrei**](https://github.com/lexfrei) in #4552).
* **feat(platform): add slim installation variants**: Add the isp-slim, isp-slim-generic and isp-hosted-slim variants ([**@lexfrei**](https://github.com/lexfrei) in #4595).
## Improvements (minor)
* **[cozystack-scheduler] Publish SchedulingClass demo**: Add demo scripts to the cozystack-scheduler system packages ([**@lllamnyp**](https://github.com/lllamnyp) in #2252).
* **perf(lineage): cache owner GETs and short-circuit re-admission**: The lineage mutating webhook caches owner lookups for 60 seconds and skips the owner walk for non-Pod objects that already carry lineage labels, cutting admission latency on clusters with many tenants ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2667).
* **refactor(dashboard): validate the schema form before page-level checks**: Re-applies the standalone console's dashboard fix onto the in-tree console after the UI was vendored into the monorepo, so the schema form is validated before page-level checks run ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3116).
* **chore(kubernetes): drop the force-delete flag that blocks partial cleanup**: Drop the cluster-autoscaler force-delete-unregistered-nodes flag; the clusterapi provider does not implement force deletion, and the flag only made batch removal of long-unregistered machines all-or-nothing ([**@lexfrei**](https://github.com/lexfrei) in #3899).
## Security
* **feat(kubeovn-webhook): add client-certificate verification, off by default**: The webhook now accepts --client-ca-file. Left empty (the default) it requests and logs client certificates without requiring them; set to a CA bundle it rejects callers that do not present a certificate signed by that CA ([**@kvaps**](https://github.com/kvaps) in #2698).
* **feat(cozystack-controller): extract application CA into a key-free tenant Secret**: Publish a key-free per-application CA trust anchor (`<release>.tenant-ca`) that tenants read to verify managed-application TLS, declared through a new `TenantProjection` CRD. Postgres is the first consumer ([**@lexfrei**](https://github.com/lexfrei) in #3407).
* **fix(dashboard): stop requesting offline_access so logout can revoke sessions**: Stop requesting the offline_access scope in the dashboard and linstor-gui auth-proxies so that logout revokes the session; a copied kc-access cookie is no longer refreshable after logout. Offline sessions already issued to existing users must be revoked manually via Keycloak ([**@IvanHunters**](https://github.com/IvanHunters) in #3447).
* **fix(api): drop caller-supplied internal.cozystack.io labels in tenantsecret writes**: TenantSecret writes no longer accept caller-supplied `internal.cozystack.io/*` labels and annotations, so a principal with write access to the API can neither plant nor strip the platform markers that decide tenant visibility on the backing Secret ([**@lexfrei**](https://github.com/lexfrei) in #3599).
* **feat(ingress-nginx): bump and restrict the admission webhook**: Update ingress-nginx from v1.11.2 and restrict the admission webhook so it is reachable only by the API server. The previously shipped v1.11.2 image already contained a backport of the CVE-2025-1974 fix; it reported itself as a stock v1.11.2 build, which this update makes unambiguous. The release ships chart 4.11.6 with controller v1.11.6 (see #4549) ([**@kvaps**](https://github.com/kvaps) in #3798).
* **fix(kubeovn-webhook): restrict the webhook port to the API server**: The `kube-ovn-webhook` `/mutate-pods` handler returned namespace-derived annotations to any caller that could reach its serving port, and nothing previously stopped an arbitrary in-cluster pod from being that caller ([**@kvaps**](https://github.com/kvaps) in #3955, shipped in v1.6.3).
* **fix(registry): run validating admission on TenantSecret writes**: Run validating admission on TenantSecret writes, so ValidatingAdmissionPolicies and webhooks matching tenantsecrets are no longer bypassed ([**@IvanHunters**](https://github.com/IvanHunters) in #4226).
* **feat(harbor): restrict the portal and API to listed source ranges**: Add `managementWhitelist`, a list of IP addresses or CIDR ranges allowed to reach the Harbor web portal and API. Only `/v2/` and `/service/token` stay reachable from any address. The default is an empty list, which keeps the current behaviour. Not supported with Gateway API publishing: a non-empty list fails the render there ([**@lexfrei**](https://github.com/lexfrei) in #4300).
* **feat(keycloak): publish only the named realms on the login host**: With `ingress.adminHost` set, the login hostname publishes only the platform realm and any realm listed in the new `ingress.exposedRealms`, keeping the `master` realm login page and token endpoint off the public route. A deployment that serves additional realms on that hostname must list them ([**@sircthulhu**](https://github.com/sircthulhu) in #4363).
* **fix(ingress): trust CF-Connecting-IP only from Cloudflare ranges**: With `cloudflareProxy` enabled, `CF-Connecting-IP` is now trusted only from Cloudflare's published address ranges. Previously the ranges were passed under a key `ingress-nginx` ignores, so any client that reached the load balancer directly could set the header itself and bypass the ingress IP `whitelist` ([**@lexfrei**](https://github.com/lexfrei) in #4444, shipped in v1.6.4).
* **fix(backups): scrub legacy plaintext passwords from CNPG Backup snapshots**: The backup strategy controller now removes plaintext `users[].password` values left in `status.underlyingResources` of CNPG `Backup` objects taken before managed Postgres stopped accepting user passwords. It runs on every controller start and needs no action ([**@androndo**](https://github.com/androndo) in #4558).
* **fix(backups): restrict the kafka-metadata CLI Pod and correct demo personas**: The kafka-metadata backup demo now runs its CLI Pod with the restricted Pod Security context, and the kafka and nats backup demos now list steps 03, 04, 06, 07 and cleanup as admin steps, because they need Pod, exec, Secret and cluster-scoped permissions that tenant roles do not grant ([**@lexfrei**](https://github.com/lexfrei) in #4695).
* **fix(computeplane): do not grant tenants the cluster admin kubeconfig**: Tenant users no longer get read access to the ComputePlane cluster's admin kubeconfig Secret through the kubernetes app's dashboard Role ([**@lexfrei**](https://github.com/lexfrei) in #4722).
## Bug fixes
### Kubernetes and worker pools
* **fix(kubernetes): keep NFS publish from hanging on a dead server**: An NFS volume publish in tenant Kubernetes clusters no longer hangs forever when the NFS server goes away during the publish: the plugin's temporary mount of the export root is now soft, and a second publish to the same target path is refused while the first is still running, so kubelet retries stop piling up ([**@lexfrei**](https://github.com/lexfrei) in #4791, backport #5017).
* **fix(kubernetes): raise kubevirt-csi API client rate limits**: Raise the kubevirt-csi-driver API client rate limits (QPS/Burst, tunable via --kube-api-qps and --kube-api-burst) above the client-go defaults so a burst of concurrent volume attaches no longer times out as FailedAttachVolume ([**@lexfrei**](https://github.com/lexfrei) in #3428).
* **fix(kubernetes): reconcile KCCM EndpointSlices when tenant node VMIs change**: The KCCM now reconciles the LoadBalancer EndpointSlices when the VMIs backing the tenant nodes change, instead of keeping stale addresses that blackhole part of the inbound traffic ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3500).
* **fix(kubernetes): probe Keycloak CRDs by output in oidc-bootstrap cleanup**: The oidc-bootstrap hook no longer fails on clusters without the Keycloak operator CRDs, which blocked Helm upgrades of kubernetes-* releases at oidc.mode=None ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3520, shipped in v1.6.5).
* **fix(kubernetes): name TalosConfigTemplate by content hash**: Name the worker TalosConfigTemplate after a hash of its content so machine-config changes (searchDomains, certSANs, registries, nameservers) actually propagate instead of being silently rejected by the immutability webhook. Upgrading rolls existing Talos workers once, on KubeVirt and Proxmox pools alike ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3523).
* **fix(kubernetes): render the talos-reconcile Job for the default md0 group**: On a managed Kubernetes cluster left with the default (undeclared) `nodeGroups`, the Job that creates each worker's `TalosConfigTemplate` and patches the control-plane's cert SANs only iterated the user-supplied node-group map, so it silently skipped the implicit `md0` group ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3535, shipped in v1.6.1).
* **fix(kubernetes): render the aggregation-layer UID header for tenant OIDC**: Tenant clusters with `spec.oidc.mode` set to `System` or `CustomConfig` now render `--requestheader-uid-headers=X-Remote-Uid` on the tenant kube-apiserver, plus the `RemoteRequestHeaderUID` feature gate on v1.32, so that an aggregated API server can read the caller's UID out of the request headers ([**@lexfrei**](https://github.com/lexfrei) in #3539).
* **fix(kubernetes): retry the tenant CNI install instead of uninstalling it**: The tenant Cilium HelmRelease had no install strategy, so Flux used its default `RemediateOnFailure`, which uninstalls Cilium — the tenant cluster's only CNI — between retry attempts whenever the initial install ran long (for example while waiting for the first worker node to register), then reinstalls it on the next attempt, repeating without end ([**@lexfrei**](https://github.com/lexfrei) in #3552, shipped in v1.6.3).
* **fix(kubernetes): retry the tenant CSI install instead of uninstalling it**: The tenant CSI HelmRelease now retries a failed install instead of being uninstalled between attempts, so a failed install no longer takes the tenant's CSI node plugin and its propagated StorageClasses away while it retries. The same strategy applies to the upgrade action, which it has to: a retried install runs as an upgrade ([**@lexfrei**](https://github.com/lexfrei) in #3581).
* **fix(kubernetes): keep tenant addon CRDs when a release is uninstalled**: Tenant addon CRDs for Gateway API, VerticalPodAutoscaler, Prometheus Operator and VolumeSnapshot now carry `helm.sh/resource-policy: keep`. A failed install that remediates by uninstalling the release no longer deletes those CRDs and, with them, every custom resource of those kinds in the tenant cluster ([**@lexfrei**](https://github.com/lexfrei) in #3586).
* **fix(kubernetes): bound the remaining pre-delete hook deletions**: Bound the pre-delete hook's remaining unbounded deletions and limit its Job to a single attempt, so a stuck finalizer on a DataVolume, LoadBalancer Service or KamajiControlPlane no longer parks the hook and blocks the uninstall ([**@lexfrei**](https://github.com/lexfrei) in #3604).
* **fix(kubernetes): delete orphaned resources in new post-delete job**: Fixed the `<name>-talos-ca` and `<name>-talos-tls-cert` Secrets and the `<name>-oidc-bootstrap` Job being left in the tenant namespace after a Kubernetes cluster is deleted ([**@crossRT**](https://github.com/crossRT) in #3856).
* **fix(kubernetes): break bootstrap-token Job out of Helm hook to unblock fresh installs**: Tenant `bootstrap-token-<id>` Secret is now created in the main install phase instead of a post-install/post-upgrade hook, unblocking fresh installs and any HR reconcile that finds MachineDeployment in a non-Ready state ([**@IvanHunters**](https://github.com/IvanHunters) in #3887, shipped in v1.6.5).
* **fix(kubernetes): widen and expose the worker node join timer budgets**: Widen the worker node join timer budgets and make them configurable: the cluster-autoscaler maxNodeProvisionTime is now set per MachineDeployment (default 30m), the MachineHealthCheck Ready=Unknown and Ready=False timeouts are values (defaults 5m and 10m), and nodeStartupTimeout defaults to 20m ([**@lexfrei**](https://github.com/lexfrei) in #3898).
* **fix(kubernetes): validate MachineHealthCheck condition timeouts at render time**: Reject zero and negative MachineHealthCheck condition timeouts at render time, and accept the canonical Go duration form ("20m0s", "1h0m0s") for the worker join timer values ([**@lexfrei**](https://github.com/lexfrei) in #3901).
* **fix(kubernetes): pin KubeVirt CSI sidecars to SIG Storage releases**: Pins the KubeVirt CSI sidecars to versioned SIG Storage images, fixing sidecars that previously would not run on pre-x86-64-v3 CPUs ([**@yankawai**](https://github.com/yankawai) in #3934, shipped in v1.6.4).
* **fix(kubernetes): apply worker TalosConfigTemplate server-side**: Apply the worker TalosConfigTemplate server-side so the talos-reconcile Job is no longer OOMKilled on clusters with a large CRD count. The Job previously failed on every reconcile after the first install, which meant on every chart upgrade ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4163, shipped in v1.6.5).
* **fix(kubernetes): fail the OIDC bootstrap Job when the tenant CA is missing**: A System-mode OIDC install no longer reports Ready while the tenant OIDC kubeconfig Secret still holds the install-time placeholder — the bootstrap Job now fails (and retries) instead of only logging a warning when the tenant cluster's CA data isn't available yet ([**@pujitha24**](https://github.com/pujitha24) in #4232).
* **fix(cluster-api): backport live migration before host eviction**: `cluster-api-provider-kubevirt` now attempts live migration before falling back to drain and delete when a guest node is evicted, uses the guest node's own name for the drain fallback instead of the wrong identifier, and uncordons a recreated guest — so an eviction no longer forces an unnecessary VM restart when a live migration would have sufficed ([**@yankawai**](https://github.com/yankawai) in #4291, shipped in v1.6.4).
* **fix(kubernetes): give System-mode tenant OIDC users a UID**: Users authenticated through `spec.oidc.mode: System` on a tenant Kubernetes cluster now carry a UID, taken from the issuer's `sub` claim, where they previously had an empty one. Audit records, `kubectl auth whoami` and the `X-Remote-Uid` header that reaches an aggregated API server inside the tenant all show it ([**@lexfrei**](https://github.com/lexfrei) in #4379).
* **fix(kubernetes-nodes): refuse a resources.cpu that cannot be a core count**: On the kubevirt substrate, a worker pool whose resources.cpu cannot be a whole vCPU core count (for example 500m, 1.5 or a value above 4294967295) now fails at render time with a clear error, instead of rendering a VM template that is refused later or dropping the pool's instanceType ([**@lexfrei**](https://github.com/lexfrei) in #4386).
* **fix(kubernetes): render the OIDC UID header only from v1.33**: The tenant chart renders the aggregation-layer UID header flag with OIDC on only from an effective Kubernetes v1.33, and never renders a --feature-gates term of its own. v1.32 tenants render no UID flags, which matches every released version, so nothing regresses ([**@lexfrei**](https://github.com/lexfrei) in #4407).
* **fix(kamaji): drop any subchart nested under the vendored kamaji chart**: `make update` now drops any subchart nested under the vendored kamaji chart, and a unit test fails if one appears, because such a subchart (for example the bundled kamaji-etcd) would deploy regardless of `kamaji-etcd.deploy` ([**@lexfrei**](https://github.com/lexfrei) in #4459).
* **fix(capi): ship the core provider manifests the repository actually holds**: Ships the `startupProbe` in the packaged Cluster API core provider components; new installs get it, while a cluster already running the core provider keeps its manager without the probe until the provider is re-applied ([**@themoriarti**](https://github.com/themoriarti) in #4483).
* **fix(kamaji): resolve Gateway API availability once at startup**: Kamaji now checks for the Gateway API once at startup instead of on every TenantControlPlane reconcile, so a slow discovery endpoint can no longer use up the reconcile timeout ([**@lexfrei**](https://github.com/lexfrei) in #4501).
* **fix(kubernetes): log kubevirt-csi temp NFS mount dir removal failure**: The kubevirt-csi node plugin now logs a warning when it cannot remove its temporary NFS mount dir instead of dropping the error ([**@lexfrei**](https://github.com/lexfrei) in #4562).
* **fix(kubernetes): let kubevirt-csi hot-unplug from an orphaned VMI**: The kubevirt-csi controller can now hot-unplug a volume from a VMI that no VM owns, instead of failing with 403 Forbidden ([**@lexfrei**](https://github.com/lexfrei) in #4563).
* **fix(kubernetes): accept a golden worker image that exists only as a PVC**: A worker pool on osImage.builtin now accepts a golden image that exists in cozy-public only as a PVC, matching vm-disk; it is refused only when neither the DataVolume nor the PVC exists ([**@lexfrei**](https://github.com/lexfrei) in #4578).
* **fix(kubernetes): force the kubevirt-csi temp NFS unmount after a timeout**: The kubevirt-csi node plugin no longer hangs an NFS volume publish forever when unmounting its temporary mount after the NFS server went away; the unmount now times out and is retried with force ([**@lexfrei**](https://github.com/lexfrei) in #4584).
* **fix(kubernetes, e2e): unblock autoscaling below min size, size the computeplane pool, fix the backupcontroller build**: Unblocks node-pool autoscaling below the minimum size, sizes the ComputePlane pool correctly, and repairs the backup-controller test build that a merge race had broken ([**@lexfrei**](https://github.com/lexfrei) in #4768).
### Virtualization and storage
* **fix(seaweedfs): stop the S3 gateway from being OOM-killed on large objects**: SeaweedFS is now built in-tree with a patch that bounds the S3 gateway's read-ahead, so downloads of large objects, including CDI imports from a Bucket URL, are no longer cut off tens of megabytes in. Every SeaweedFS component runs this image, which reports itself as `4.45 cozystack-patched`; the patch is dropped once a SeaweedFS release carrying the upstream fix is vendored ([**@scooby87**](https://github.com/scooby87) in #4819, backport #5028).
* **fix(linstor): mark volumes restored from a snapshot DRBD-initialized**: A PVC restored from a LINSTOR VolumeSnapshot no longer gets a replica with an empty volume that DRBD marks UpToDate, which could present an empty filesystem to a pod. The fix is a backport of LINBIT/linstor-server@9d8cdf5 onto the shipped v1.33.3 build; volumes restored before it are not repaired and can be checked as described in #4862 ([**@lexfrei**](https://github.com/lexfrei) in #4866, backport #5020).
* **fix(vm-disk): name the importer error in the disk's WorkloadsReady message**: The `WorkloadsReady` condition of a VM disk now includes the importer error CDI reports, such as a 404 or an unresolvable host, so a disk whose source keeps failing shows why it is stuck instead of only `ImportInProgress` ([**@scooby87**](https://github.com/scooby87) in #4815, backport #5016).
* **fix(vm-disk): let CDI adopt the PVC a Velero restore recreates**: A VMDisk restored by Velero, alone or as part of a VMInstance, no longer fails its next Helm upgrade with `Destination PVC ... already exists`: the vm-disk chart now sets `cdi.kubevirt.io/allowClaimAdoption` on the DataVolume it renders, so CDI adopts the restored PVC and keeps its data. A PVC named `vm-disk-<name>` left behind without an owner is now adopted, with its old contents, by a new VMDisk of the same name ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4955, backport #5084).
* **fix(seaweedfs): make naming audit fail closed on kubectl and payload errors**: `hack/seaweedfs-naming-audit.sh`, used by operators to classify SeaweedFS instances before the naming-migration cleanup, was fail-open — any `kubectl` failure or unreadable Helm release payload produced an empty result table indistinguishable from a genuinely clean fleet ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3436, shipped in v1.6.1).
* **fix(velero): apply CRD updates on upgrade via CreateReplace**: Velero's CRDs stayed frozen at whatever version was first installed, since Helm never touches a chart's `crds/` directory on upgrade; when the Velero image moved to a version that added new backup phases, the apiserver rejected phase transitions against the stale CRDs and backups silently stopped while the HelmRelease stayed green ([**@lexfrei**](https://github.com/lexfrei) in #3727, shipped in v1.6.2).
* **fix(vm-instance): disable helm readiness wait on the VMInstance release**: VMInstance HelmReleases no longer hang InProgress or roll back healthy VMs; the release no longer waits on VirtualMachine readiness, which helm-controller's kstatus poller cannot assess correctly for KubeVirt VMs. Existing VMInstance releases are brought in line automatically on upgrade ([**@IvanHunters**](https://github.com/IvanHunters) in #3735).
* **fix(linstor): use severity warning instead of warn in piraeus-datastore alerts**: LINSTOR/DRBD alerts carried severity `warn`, which Alerta rejects, so they were never delivered. They now use the standard `warning` severity, and the keda at-max autoscaler alert moves from `info` to `informational` for the same reason ([**@yankawai**](https://github.com/yankawai) in #3799, shipped in v1.6.5).
* **fix(vm-disk): fail on storageClass/source edit of existing disk**: Editing storageClass or source on an existing VMDisk now fails with a clear message instead of being silently ignored, for disks created on v1.7.0 or later; on older disks a `storageClass` edit is still ignored (a `source` edit is refused). Delete and recreate the disk to change these immutable fields ([**@scooby87**](https://github.com/scooby87) in #3969).
* **fix(virtual-machine): guard Kube-OVN IP lookup**: Render VM instances on clusters where the Kube-OVN IP API is not installed ([**@lfinmauritius**](https://github.com/lfinmauritius) in #4013).
* **fix(cdi): stop make update from deleting first-party templates**: `make update` in kubevirt-cdi no longer deletes its first-party templates ([**@lexfrei**](https://github.com/lexfrei) in #4097).
* **fix(kubevirt): verify the vendored operator manifest and add a test target**: Kubevirt-operator now verifies the upstream manifest against a pinned sha256 before vendoring it, and the package has a unit-test target so hack/helm-unit-tests.sh no longer skips it ([**@lexfrei**](https://github.com/lexfrei) in #4099).
* **fix(linstor): wait for temporary probe devices**: Backports a LINSTOR upstream fix that waits for a ZFS probe device to appear before reading its I/O properties, instead of racing udev; without the wait, a pool could be left with incorrect block sizes and failed replica placement on 4K pools ([**@yankawai**](https://github.com/yankawai) in #4100, shipped in v1.6.4).
* **fix(vm-instance): let complete resources replace the instance type**: A complete `resources` block (whole-number cpu and sockets, plus memory) now sizes the VM directly and suppresses the `instanceType` matcher, instead of rendering both and having KubeVirt refuse the VM ([**@lexfrei**](https://github.com/lexfrei) in #4109).
* **fix(cdi): omit the CDI clone strategy override when unset**: `cloneStrategyOverride` in the `kubevirt-cdi` package can now be left empty, which omits the field and lets CDI pick a clone strategy per StorageProfile. The default is unchanged at `csi-clone` ([**@lexfrei**](https://github.com/lexfrei) in #4138).
* **fix(linstor): reconnect only the stalled peer in the satellite plunger**: The satellite plunger disconnected a stalled DRBD peer but then tried to reconnect every peer of the resource at once, which failed with "Device has a net-config" for peers that were never disconnected and left the stalled one permanently unreconnected ([**@yankawai**](https://github.com/yankawai) in #4184, shipped in v1.6.4).
* **fix(seaweedfs): raise the bucket ceiling by sizing volumes smaller**: Raise how many buckets a new SeaweedFS instance can serve on the default 10Gi volume PVC from two to eight at `replicationFactor: 2`, by defaulting the master volume size limit to 400 MB instead of 1000 and lowering the two-copy growth count from 6 to 3, and expose the limit as `master.volumeSizeLimitMB` on the app chart ([**@lexfrei**](https://github.com/lexfrei) in #4251).
* **fix(seaweedfs): parse naming-audit timestamps portably across GNU and BSD date**: `hack/seaweedfs-naming-audit.sh` read PV and `first_deployed` timestamps with `date -u -d`, a spelling only GNU `date` accepts ([**@scooby87**](https://github.com/scooby87) in #4282).
* **fix(bucket): stop the ACME cleanup hook from talking to localhost, and gate the class**: The ACME cleanup hook now reaches the apiserver instead of reporting success while doing nothing ([**@IvanHunters**](https://github.com/IvanHunters) in #4289).
* **fix(linstor): add opt-in graceful satellite shutdown on Talos**: Adds an opt-in Talos satellite `preStop` hook that releases unused Secondary DRBD resources, ZFS zvols included, before the satellite stops, without forcing, demoting or calling the Kubernetes API ([**@yankawai**](https://github.com/yankawai) in #4292, shipped in v1.6.4).
* **fix(vm-instance): supply cloud-init network-data to netplan guests**: A new `netplanDhcp` value, off by default, makes the chart supply cloud-init network-data that requests DHCP on every ethernet interface and accepts the DNS search domains offered over DHCP ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4306).
* **fix(vm-disk): report DataVolume readiness through WorkloadsReady**: Report whether CDI finished populating a disk in the WorkloadsReady condition, which names the stuck DataVolume and its phase. kubectl get shows WorkloadsReady in a new WORKLOADS column for every application kind, and the console badge turns amber while a disk is not populated, so a disk whose import or clone never completes no longer looks healthy ([**@lexfrei**](https://github.com/lexfrei) in #4380).
* **fix(vm-default-images,kubernetes-worker-image,vm-disk): wait for golden images to import and refuse cloning a failed one**: The golden image catalogs now report Ready only after their images have finished importing, and a failed or stalled import keeps the catalog not Ready ([**@lexfrei**](https://github.com/lexfrei) in #4392).
* **fix(cdi): drop the ExpandDisks gate from the CDI object**: Stop listing the KubeVirt `ExpandDisks` feature gate on the CDI object, where CDI ignored it. Disk expansion is unaffected ([**@lexfrei**](https://github.com/lexfrei) in #4415).
* **fix(cdi): validate kubevirt-cdi values against a schema**: Kubevirt-cdi now validates its values against a schema, so an invalid `cloneStrategyOverride` such as `csi_clone` fails at render time instead of when the CDI object is applied ([**@lexfrei**](https://github.com/lexfrei) in #4417).
* **fix(migration): restore the class reclaim policy on imported volumes**: Disks imported from VMware no longer keep a Retain reclaim policy after the import. The volume gets its StorageClass policy back once it is bound to its VMDisk, so deleting an imported VMDisk deletes its volume instead of leaving an orphaned Released PersistentVolume ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4480).
* **fix(linstor): adopt existing CR on 409 AlreadyExists in k8s backend**: The LINSTOR controller no longer gets stuck refusing writes to a property after a retried Kubernetes API request, which could leave a deleted volume in Released with its storage still allocated ([**@lexfrei**](https://github.com/lexfrei) in #4532).
* **fix(vm-disk): refuse to clone a golden image that does not exist**: A new disk with source.image now fails with a clear error when the golden image does not exist in cozy-public, instead of waiting forever for a clone that never starts ([**@lexfrei**](https://github.com/lexfrei) in #4567).
* **fix(linstor): build piraeus-server's protobuf code on an arm64 builder**: The arm64 half of a release fails to build piraeus-server. linstor-server's build always downloads the `linux-x86_64` protoc, and the builder stage runs on the build platform ([**@lexfrei**](https://github.com/lexfrei) in #4576).
* **fix(kubevirt): route the export proxy to the Service KubeVirt creates**: Requests to vm-exportproxy.<host> now reach the KubeVirt export proxy: the Ingress and TLSRoute route to the virt-exportproxy Service KubeVirt creates, and tenants are allowed egress to the export proxy ([**@lexfrei**](https://github.com/lexfrei) in #4637).
### Databases and messaging
* **fix(kafka): pass the container limits to the Kafka JVMs**: The Strimzi cluster operator, brokers, controllers, entity operators and the drain-time Cruise Control no longer size their JVM heap from host memory on kernels built without cgroup v1 memory support, which got them OOMKilled; each JVM now gets its memory limit explicitly. The Kafka migration hook no longer runs out of memory on upgrade in a tenant with resource quotas. Existing Kafka clusters roll once on upgrade ([**@lexfrei**](https://github.com/lexfrei) in #4865, backport #5019).
* **fix(harbor): guarantee CPU for redis sentinels**: The bundled Redis Sentinel pods now run with guaranteed resources (250m CPU / 128Mi, requests == limits) so the HA set converges reliably on CPU-contended nodes ([**@lexfrei**](https://github.com/lexfrei) in #3446).
* **fix(keycloak): keep KeycloakRealmUser roles when spec.roles is unset**: A `KeycloakRealmUser` that omits `spec.roles` or leaves `roles` out of a `spec.clientRoles` entry no longer has the matching Keycloak role mappings deleted under the default `full` reconciliation strategy. Users keep `default-roles-<realm>` and the `offline_access`, `uma_authorization` and `account` roles it grants ([**@lexfrei**](https://github.com/lexfrei) in #3531).
* **fix(etcd): fail the CRD vendoring when the keep stamp misses**: `make update` in `packages/system/etcd-operator-crds` vendors three CRDs and stamps `helm.sh/resource-policy: keep` onto each with an awk step anchored on the `controller-gen.kubebuilder.io/version:` line. That line is upstream's to emit ([**@lexfrei**](https://github.com/lexfrei) in #3595).
* **fix(tenant): bound the pre-delete cleanup waits and the hook Job**: Bound the pre-delete cleanup hook's HelmRelease deletions and give its Job a wall-clock deadline, so a tenant whose children cannot finalize no longer leaves the hook waiting indefinitely and blocking the uninstall ([**@lexfrei**](https://github.com/lexfrei) in #3598).
* **fix(harbor): retry the nested system install instead of uninstalling it**: Retry a failed Harbor install instead of uninstalling the release, so a slow first install no longer deletes the Redis volume and starts over ([**@lexfrei**](https://github.com/lexfrei) in #3621).
* **fix(tenant): retry failed etcd and seaweedfs installs in place**: Retry a failed etcd or seaweedfs install instead of uninstalling the release, so a slow first install no longer runs the cleanup hooks that delete the etcd data volumes, the SeaweedFS volumes and the filer metadata database ([**@lexfrei**](https://github.com/lexfrei) in #3630).
* **fix(tenant): retry failed monitoring and gateway installs in place**: Retry a failed monitoring or gateway install instead of uninstalling the release, so a slow first install no longer runs the cleanup hooks that delete the tenant's metrics and logs volumes or its ACME account key ([**@lexfrei**](https://github.com/lexfrei) in #3633).
* **fix(tenant): render the chart without the platform's injected values**: The tenant chart now renders without the values the platform injects, so `helm template` works on it standalone, and its `cozy-lib` dependency is declared in Chart.yaml; this clears two of the three `helm lint` errors it reported, the remaining one being the repository-wide relative icon URL ([**@lexfrei**](https://github.com/lexfrei) in #3636).
* **fix(clickhouse): assign DB_LIST so backup retention cleanup runs**: ClickHouse backup retention now prunes old snapshots. The scheduled backup job's cleanup step never ran because `DB_LIST` was left unset, so snapshots accumulated in object storage indefinitely; it now honors the configured retention (`cleanupStrategy` / `--keep-last=N`) ([**@scooby87**](https://github.com/scooby87) in #3760).
* **fix(kafka): make topics[].config optional**: Kafka topics can now omit the optional Strimzi config map entirely, instead of the chart requiring one even when there is nothing to set ([**@yankawai**](https://github.com/yankawai) in #3935, shipped in v1.6.4).
* **fix(rabbitmq): right-size the default resources preset to s1.nano**: Raises the default `resourcesPreset` from `t1.nano` (128Mi) to `s1.nano` (250m CPU, 512Mi), because a default-sized RabbitMQ 4.2 broker was OOMKilled on `t1.nano` and never became Ready ([**@yankawai**](https://github.com/yankawai) in #3936, shipped in v1.6.4).
* **fix(kafka): delete release topics before the topic operator is removed**: Delete release-owned KafkaTopics before removing the Strimzi topic operator ([**@yankawai**](https://github.com/yankawai) in #3938).
* **fix(postgres): let in-place restore re-bootstrap instead of wedging**: An in-place PostgreSQL `RestoreJob` (restoring an app back into itself) never came back up — the cluster stayed in `Setting up primary` with crash-looping recovery pods, or the HelmRelease upgrade failed outright ([**@androndo**](https://github.com/androndo) in #3959, shipped in v1.6.3).
* **fix(etcd-operator): gate VMServiceScrape on VictoriaMetrics operator CRD**: Stop requiring the VictoriaMetrics operator to install etcd-operator; the VMServiceScrape now only renders when the operator's CRD is present ([**@lllamnyp**](https://github.com/lllamnyp) in #3960).
* **fix(etcd): spread members across nodes with default soft pod anti-affinity**: Etcd clusters now spread their members across nodes by default via a soft podAntiAffinity, and expose an `affinity` value so operators can set a hard rule; previously a single node could host a quorum of members and take out the cluster when it failed ([**@scooby87**](https://github.com/scooby87) in #3962).
* **fix(clickhouse): scheme the backup S3_ENDPOINT on the system-bucket flow**: On the default `useSystemBucket` backup flow, the ClickHouse chart passed a scheme-less host straight into the `clickhouse-backup` sidecar's `S3_ENDPOINT`, which `altinity/clickhouse-backup`'s AWS SDK endpoint resolver rejects outright — so every ClickHouse backup and restore on the default flow failed with `was not a valid URI` ([**@androndo**](https://github.com/androndo) in #3963, shipped in v1.6.3).
* **fix(mongodb): fill the dashboard credentials on a first install**: The dashboard credentials Secret is now filled with the database password and connection URI on first install; previously it shipped empty until someone ran `helm upgrade` by hand, because the lookup it relied on could only see users the Percona operator had not yet created ([**@yankawai**](https://github.com/yankawai) in #4014, shipped in v1.6.4).
* **fix(postgres): point the trust-anchor comment at the published projection**: The TLS comment in `db.yaml` told operators to take `ca.crt` from `<release>-credentials`. That key is not there. The chart renders that Secret from `.Values.users`, so it holds passwords and nothing else ([**@lexfrei**](https://github.com/lexfrei) in #4063).
* **fix(harbor): give the nginx proxy explicit resources so it survives the tenant LimitRange**: Set resources for the nginx proxy and raise the portal preset so neither is OOMKilled by the tenant LimitRange default (128Mi) ([**@IvanHunters**](https://github.com/IvanHunters) in #4095).
* **fix(rabbitmq): bound the topology operator Secret cache**: Update messaging-topology-operator to 1.20.2, which bounds its Secret cache to Secrets labeled `rabbitmq.com/topology-operator=true` instead of caching every Secret in the cluster ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4130).
* **fix(clickhouse): keep the Keeper name inside the StatefulSet volume-name limit**: Shortens the generated Keeper name for application names longer than 15 characters, fixing the Keeper `StatefulSet` failing to create for those names ([**@yankawai**](https://github.com/yankawai) in #4133, shipped in v1.6.4).
* **fix(nats): merge config.merge.accounts with the generated accounts map**: `config.merge.accounts` is now merged into the generated NATS accounts map instead of being rendered as a second, conflicting copy of the same key ([**@yankawai**](https://github.com/yankawai) in #4134, shipped in v1.6.4).
* **fix(opensearch): remove the data PVCs when the application is deleted**: Adds a post-delete hook that reclaims an OpenSearch application's data PVCs when it is removed, instead of leaving them orphaned ([**@yankawai**](https://github.com/yankawai) in #4136, shipped in v1.6.4).
* **fix(foundationdb): let tenant roles read the connection ConfigMap**: Tenants previously had no way to obtain their FoundationDB connection string: the cluster file lives in a ConfigMap tenant subjects could not read and the dashboard resource map did not expose. The ConfigMap is now in the dashboard resource map with a Role/RoleBinding granting the tenant `use` access level get/list/watch on it ([**@yankawai**](https://github.com/yankawai) in #4148, shipped in v1.6.4).
* **fix(opensearch-operator): raise vm.max_map_count on every node from a DaemonSet**: Set `vm.max_map_count` on every node from a DaemonSet in the operator's namespace, so OpenSearch no longer depends on a privileged init container that PodSecurity baseline refuses in a tenant namespace ([**@lexfrei**](https://github.com/lexfrei) in #4152).
* **fix(harbor-rd): carry the release prefix in the resource selectors**: The ApplicationDefinition selectors for Harbor's Secret, Service and Ingress now carry the release prefix, so they match the objects the chart creates. No user-visible change — the Secret already reached tenants through a label-based workaround, and nothing consumes the Service and Ingress labels ([**@kvaps**](https://github.com/kvaps) in #4170).
* **fix(opensearch-operator): align DNS_BASE with the platform cluster domain**: The operator's DNS_BASE now follows the platform cluster domain (`networking.clusterDomain`) instead of the upstream `cluster.local` default, so the securityconfig job and OpenSearch Dashboards reach the cluster on installations that use `cozy.local`. On a cluster whose `networking.clusterDomain` is not `cluster.local`, the default `cozy.local` included, the `<release>-securityconfig-update` Job of an instance created before this release keeps pointing at `.svc.cluster.local`, which does not resolve there, and the operator does not replace it. Deleting that Job in the instance's namespace makes the operator create a new one with the platform's cluster domain (on a test cluster the new Job connected to the cluster within seconds); that the instance then finishes initializing was not confirmed ([**@lexfrei**](https://github.com/lexfrei) in #4185).
* **fix(postgres): follow-up review remarks from #3954 (KEDA autoscaling)**: `helm unittest` green: postgres 66/66, cozy-lib-tests 45/45, keda 12/12. Both behaviour changes are mutation-verified (reverting the fix reddens the suite). No generated artifacts affected (no `values.yaml`/`values.schema.json`/`Chart.yaml`/`README.md` changes) ([**@scooby87**](https://github.com/scooby87) in #4188).
* **fix(opensearch): give the securityconfig update job its own resources**: Stop the securityconfig update job being OOMKilled at the tenant namespace memory default, which left new clusters stuck with security uninitialized and every node unready ([**@IvanHunters**](https://github.com/IvanHunters) in #4228).
* **fix(opensearch-operator): keep a single-replica cluster electable after bootstrap**: Exclude the bootstrap node from the voting configuration before removing it, so a single-replica OpenSearch cluster stays able to elect a cluster-manager ([**@IvanHunters**](https://github.com/IvanHunters) in #4231).
* **fix(kafka): keep the pre-delete hook's credentials as long as its Job**: Deleting a Kafka release no longer wedges on a pre-delete hook that cannot reach the apiserver ([**@IvanHunters**](https://github.com/IvanHunters) in #4280).
* **fix(postgres): read the live Cluster when deciding the WAL-archive prefix**: An in-place restore no longer loses its fresh WAL-archive prefix to a stale Cluster read, which left the restored cluster archiving onto the source's prefix and the RestoreJob failing on its deadline ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4299, shipped in v1.6.5).
* **fix(opensearch-operator): let a permanently yellow cluster restart**: A single-replica OpenSearch release no longer gets stuck ahead of every rolling restart once an index with a replica, such as the security audit log or any index created with default settings, leaves the cluster permanently yellow ([**@lexfrei**](https://github.com/lexfrei) in #4317).
* **fix(opensearch-operator): stop excluding a node whose drain cannot finish**: A rolling restart or upgrade that cannot drain its next node no longer leaves that node excluded from shard allocation for as long as the restart is stuck ([**@IvanHunters**](https://github.com/IvanHunters) in #4323).
* **fix(opensearch-operator): name the component reconciler a stall sits in**: The OpenSearch cluster controller now logs a watchdog report naming which of its nine component reconcilers has been running for over a minute, and keeps reporting on an interval while it stays stuck, instead of leaving a stalled reconcile with no trace in the operator log at all ([**@IvanHunters**](https://github.com/IvanHunters) in #4333, shipped in v1.6.4).
* **fix(opensearch-operator): bound every call to a cluster's HTTP API**: Time out calls to a cluster's HTTP API, so one unanswered request no longer stops the operator reconciling ([**@IvanHunters**](https://github.com/IvanHunters) in #4341).
* **fix(kafka): use the shared wait_hr_ready body in the topic-data example**: The topic-data backup example now reports why a HelmRelease wait failed, same as the other backup examples ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4436).
* **fix(kafka): delete the release's node pools on uninstall**: Deleting a Kafka application now also deletes its KafkaNodePools, so its brokers stop instead of running on after the application is gone. Volumes are kept, as before ([**@lexfrei**](https://github.com/lexfrei) in #4490).
* **fix(redis): run the multi-arch redis_exporter so the sidecar starts on arm64**: Redis, Valkey and Harbor's Redis now use the multi-arch redis_exporter image, so the exporter sidecar starts on arm64 nodes ([**@lexfrei**](https://github.com/lexfrei) in #4522).
* **fix(mariadb): correct webhook cert-manager override key path**: Correct webhook cert-manager override key path so cert-manager is actually used for the mariadb-operator webhook certificate ([**@chathurabuddi**](https://github.com/chathurabuddi) in #4537).
* **fix(tenant): fail fast on an empty tenant apex and correct gateway docs**: Tenant charts now fail rendering with a clear error when a tenant's apex host is empty and no per-app host override is set, instead of silently emitting a malformed hostname; the tenant `gateway` parameter description is corrected to match actual behavior (an unset `gateway` never auto-enables a Gateway) ([**@IvanHunters**](https://github.com/IvanHunters) in #4538).
* **fix(postgres): reassign dropped roles' objects to an unprivileged role**: Objects owned by a user removed from `users`, or by an orphaned managed database role, are now reassigned to the unprivileged `cozystack:orphaned` role instead of the `postgres` superuser. A view or function of the removed role that ran with its owner's privileges can lose access it had through that role's grants or memberships. On the first reconcile after the upgrade, users that were removed from `users` while on v1.6.x but whose roles were left behind are dropped, and their objects move to `cozystack:orphaned`; clients still logging in with such a role lose access ([**@lexfrei**](https://github.com/lexfrei) in #4574).
* **fix(clickhouse, workloadmonitor): add the cluster domain to keeper hosts and stop a PVC nil dereference**: ClickHouse Keeper node hostnames now include the cluster domain, and the WorkloadMonitor controller no longer panics on a PVC without a storage class ([**@lexfrei**](https://github.com/lexfrei) in #4608).
* **fix(nats): enable JetStream for the generated account**: Enable JetStream on the account generated for users when jetstream.enabled is true, keeping any explicit account-level JetStream setting. Instances with users roll their NATS pods once on upgrade from v1.6.4 or earlier; v1.6.5 already rolled them ([**@yankawai**](https://github.com/yankawai) in #4675, shipped in v1.6.5).
* **fix(tenant): skip Cilium policies on isp-hosted without Cilium**: The tenant chart no longer fails to install on the isp-hosted variants when the host has no Cilium. There, the Cilium network policies are created only if the host serves the Cilium policy API; otherwise tenants get no network isolation from Cozystack. Other variants are unchanged ([**@lexfrei**](https://github.com/lexfrei) in #4710).
* **fix(postgres): cap the default shared_buffers at a quarter of the memory limit**: Default shared_buffers to a quarter of the memory limit with a 16MB floor, capped at the memory request, when the limit is below 512Mi and shared_buffers is not set ([**@yankawai**](https://github.com/yankawai) in #4724, shipped in v1.6.5).
* **fix(mongodb): set the s3 bucket and prefix on the bootstrap restore**: A mongodb app created with bootstrap.enabled now gets a valid PerconaServerMongoDBRestore. the restore carries the s3 bucket and prefix of backup.destinationPath, the same location the backup storage writes to, so the api server accepts it and the operator looks for the backup in the right place. before this the api server rejected the restore because the bucket was missing, and the release failed. On upgrade, an app that still has `bootstrap.enabled: true` creates its restore on the first reconcile and the operator restores the backup into the cluster as it is at that moment, so set it to false first on every MongoDB app that should not be restored ([**@yankawai**](https://github.com/yankawai) in #4779, shipped in v1.6.5).
* **fix(redis): re-send the Sentinel auth-pass when the operator resets a Sentinel**: The redis-operator now re-sends the Redis password to a Sentinel it resets, so a Sentinel that restarted in place (for example after a node reboot) and lost its auth-pass recovers on the next check instead of staying down until its pod is deleted ([**@yrahman82**](https://github.com/yrahman82) in #4833).
* **fix(mariadb): mount the chart's my.cnf under a .cnf name**: The chart's my.cnf is applied again. Since mariadb-operator v0.28.1 it was mounted under a name the server ignores, so MariaDB instances ran on stock defaults. Every existing instance rolls its pods once on upgrade (replicas first, then the primary with a switchover) and then runs with the chart's settings ([**@lexfrei**](https://github.com/lexfrei) in #4843).
* **fix(etcd): leave compaction to etcd and pass the real quota to defrag**: The defrag job no longer compacts on top of etcd's periodic auto-compaction, and it evaluates its 80% quota threshold against the cluster's real quotaBackendBytes instead of 2 GiB ([**@yankawai**](https://github.com/yankawai) in #4873, shipped in v1.6.5).
### Backups and restore
* **fix(backups): gate every default Strategy CR on its CRD**: Every default backup `Strategy` CR the backupstrategy-controller renders is now created only once its CRD exists, so the release no longer sticks in `UpgradeFailed` on an upgrade from v1.6 and a fresh install with external S3 no longer fails; the CRDs are created first and the default strategies follow once the backupstrategy-controller re-renders (`backupStorage.reconcileDefaultObjects`, on by default). This extends the earlier MongoDB-only gate below ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4929, backport #4935).
* **fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit**: The release no longer fails with "Secret ... Too long" on the upgrade that creates the default backup strategies, because the Altinity, Job and FoundationDB strategies now store their pod templates schemaless like Kafka, RabbitMQ and Redis ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4941, backport #5023).
* **fix(backups): prepare standalone VMDisk restore targets**: An in-place RestoreJob for a standalone VMDisk now suspends the disk's HelmRelease, deletes its DataVolume and, with `keepOriginalPVC`, renames the original PVC before Velero restores it, instead of racing Flux and CDI over the same PVC. Such a restore fails before changing anything when a running Pod, such as the launcher of a VM that uses the disk, mounts the disk's PVC, and a restore with `keepOriginalPVC` no longer reports Succeeded over an empty PVC that CDI recreated under the disk's name. Real VMInstance and VMDisk backup/restore round-trips now run in e2e ([**@androndo**](https://github.com/androndo) in #3427, backport #5044).
* **fix(backups): request S3 checksum only when required for barman-cloud (non-AWS S3 / Ceph RGW)**: CNPG's barman-cloud plugin sidecar defaulted to computing a flexible checksum on every upload, which several S3-compatible backends (Ceph RGW, some MinIO / Cloudflare R2 builds) reject outright, so every backup and WAL-archive upload to those backends failed and `ScheduledBackup`s never stored anything ([**@androndo**](https://github.com/androndo) in #3417, shipped in v1.6.2).
* **fix(backupstrategy-controller): repair lookup-gated backup objects**: The default backup `Strategy` CRs and the Velero `BackupStorageLocation` are gated on a Helm `lookup` performed while the referenced object is still being created; when that lookup came back empty the objects were skipped permanently, since helm-controller does not re-render a release whose chart and values are unchanged ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3524, shipped in v1.6.2).
* **fix(backupstrategy-controller): gate the default MongoDB strategy on bootstrap**: Gate the default MongoDB backup strategy on the bootstrap window, so a fresh platform install no longer fails with "no matches for kind MongoDB" and applies nothing ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3815).
* **fix(backupstrategy-controller): gate the default MongoDB Strategy on its CRD**: Fixed a permanent failure of the backupstrategy-controller release when upgrading a cluster to a version that adds a new backup-strategy kind: the default MongoDB Strategy is now withheld until its CRD is served, instead of failing the whole release with "no matches for kind" ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3980).
* **fix(backups): give the barman-cloud sidecar its own resources**: The barman-cloud sidecar previously had no resource requests or limits, so in tenant namespaces it inherited a 128Mi `LimitRange` default and was OOMKilled during backups even while the surrounding `ObjectStore` and `Cluster` stayed healthy ([**@yankawai**](https://github.com/yankawai) in #4220, shipped in v1.6.4).
* **fix(backups): stop the redis backup demo from reading empty values**: The Redis backup demo reads its keys through `kubectl exec`, so a short-lived Pod can no longer drop its output, and it waits for the platform's `cozy-default-redis` strategy to appear instead of failing on its first read ([**@lexfrei**](https://github.com/lexfrei) in #4399).
* **fix(backups): fail Velero RestoreJob rename for non-VMInstance kinds**: A Velero-strategy RestoreJob that requests a different targetApplicationRef.name for an application kind other than VMInstance (Postgres, MariaDB, ClickHouse, MongoDB, VMDisk) now fails with an explicit message instead of reporting Succeeded with the application restored under its original name ([**@androndo**](https://github.com/androndo) in #4426).
* **fix(backups, postgres): start a Postgres restore from the backup it names**: A Postgres RestoreJob now starts from the base backup its Backup refers to (recoveryTarget.backupID) instead of whichever backup the barman-cloud plugin picks, which on an archive with several timelines could restore another branch's data; a recoveryTime before that backup's end keeps the previous time-based choice ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4443).
* **fix(backups): stop the nats, kafka-metadata and rabbitmq backup demos from reading empty values**: Stop the nats, kafka-metadata and rabbitmq backup demos from reading empty CLI output ([**@IvanHunters**](https://github.com/IvanHunters) in #4458).
* **fix(backups): hold a BackupJob while its application is being restored**: A BackupJob that has not started yet, scheduled or on demand, now waits while a RestoreJob is restoring the same application, instead of running against it. It stays Pending with a Ready=False RestoreInProgress condition and runs once the restore ends ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4544).
* **fix(backups): share the walkthrough wait helpers and fix the S3 preflight cleanup**: The backup walkthroughs share one set of wait helpers that keep kubectl errors visible, the kafka topic-data demo no longer reads empty replies from a throwaway Pod, the nats and kafka-metadata demos no longer exec into a CLI Pod they refused, and the e2e S3 access preflight says why client setup or cleanup failed and deletes its test object reliably ([**@lexfrei**](https://github.com/lexfrei) in #4635).
* **fix(backups, postgres): refuse a restore into another PostgreSQL major**: A Postgres restore whose backup was taken on another PostgreSQL major version than the target's spec.version now fails before touching the target, instead of purging it and then failing to recover ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4669).
* **fix(backups): give each Postgres instance its own default WAL prefix**: A Postgres app deleted and recreated under the same name no longer archives its WALs and base backups into its predecessor's backup stream. The default CNPG WAL-archive prefix now carries the application UID; clusters that have already been backed up keep their current prefix ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4670).
* **fix(backups): refuse a foreign -orig PVC and record the kept disk's reclaim policy**: An in-place VM restore with keepOriginalPVC now fails, before the VM is halted, when the <disk>-orig-<hash> PVC it would keep the disk under already exists and was not created by this RestoreJob (for instance one left by a deleted RestoreJob of the same name), instead of skipping the rename and restoring without keeping the current disk ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4671).
* **fix(backups): delete the original PVC of an in-place VM restore without keepOriginalPVC**: An in-place VMInstance RestoreJob with keepOriginalPVC=false now deletes each original disk PVC, and waits for it to be gone, before Velero restores it ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4672).
* **fix(backups): create no driver object for a job that already finished**: A BackupJob or RestoreJob that the controller has just marked Failed no longer gets a driver object created for it by the next reconcile, which could start on a cache that had not yet seen the Failed status and retry the step that failed — leaving, for instance, an untracked Velero Backup behind a Failed BackupJob, or a restore that halts the VM after its RestoreJob failed ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4673).
* **fix(backups): hold a Postgres backup until the plugin rollout is over**: The CNPG backup driver no longer loses the first backup of a Postgres app whose Cluster had no barman-cloud plugin yet (a new platform-flow app, or one restored in place) ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #4674).
* **fix(backups): refuse concurrent restores and skip the CNPG purge only for this restore's Cluster**: A RestoreJob now fails with reason ConcurrentRestore when another unfinished RestoreJob is already restoring the same application, instead of running alongside it; create it again once the first restore has finished ([**@androndo**](https://github.com/androndo) in #4754).
### Networking and gateway
* **fix(multus): drop the CPU limit and make resources configurable**: The `kube-multus` container drops its 100m CPU limit and gets a 4Gi memory limit (v1.6.5 set no memory limit), so a node with hundreds of pods can recover after a CNI outage instead of being throttled past kubelet's CRI deadline. Resources are set through the new `networking.multus.resources` platform value, and an override that removes the memory limit or request fails the render ([**@lexfrei**](https://github.com/lexfrei) in #4830, backport #5018).
* **fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates**: After a platform upgrade the tenant `gateway` HelmRelease no longer fails with `timeout waiting for: [TenantGateway/<namespace>/cozystack status: 'InProgress']` and retries without end while a listener certificate is pending, holding a helm-controller worker and failing the parent tenant release with it. The release now waits only until the controller has observed the current TenantGateway spec, not for listeners or certificates ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5086, backport #5127).
* **fix(ingress-nginx): drop the admission-webhook policy a rebase brought back**: Remove the namespaced admission-webhook `CiliumNetworkPolicy` that was re-added by mistake. It rendered without checking that the Cilium API exists, so on a cluster without Cilium every tenant with `ingress: true` failed to install with `no matches for kind "CiliumNetworkPolicy"`. The clusterwide admission-webhook policy is unchanged and still renders only where the Cilium API is served ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5088, backport #5128).
* **fix(http-cache,vpa): fix haproxy 3.3 backend name conflict and vpa-for-vpa OOM**: Rename haproxy backend to avoid HAProxy 3.3 name conflict between frontend and backend fix(vpa): increase vpa-for-vpa recommender memory limit from 512Mi to 1Gi to prevent OOMKilled restarts ([**@IvanHunters**](https://github.com/IvanHunters) in #2654).
* **fix(cozystack-basics): normalize tenant hostname VAP comparisons with lowerAscii**: Tenant hostname admission policies now compare against the tenant apex and the platform root apex case-insensitively ([**@lexfrei**](https://github.com/lexfrei) in #3208).
* **fix(cozystack-basics): gate the hostname VAP policies on the VAP API**: The hostname `ValidatingAdmissionPolicy` templates rendered unconditionally, so a first install on a cluster where the `ValidatingAdmissionPolicy` API is unavailable dropped the policies permanently — a later cluster upgrade that gains the API would not bring them back ([**@lexfrei**](https://github.com/lexfrei) in #3409, shipped in v1.6.1).
* **fix(kube-ovn): reload kubeovn-webhook serving certificate on cert-manager renewal**: `kube-ovn-webhook` loaded its TLS serving certificate once at startup and never re-read it; once cert-manager renewed the backing Secret and the old certificate expired, the apiserver's calls to the webhook failed verification and, because the `MutatingWebhookConfiguration` uses `failurePolicy: Fail`, every pod creation in tenant namespaces was rejected — including `virt-launcher` pods, blocking VMI startup ([**@IvanHunters**](https://github.com/IvanHunters) in #3557, shipped in v1.6.2).
* **fix(ingress-nginx): pull the protobuf-exporter from the project namespace**: The protobuf-exporter sidecar is now pulled from ghcr.io/cozystack instead of a personal namespace. Same image and digest; no version change ([**@kvaps**](https://github.com/kvaps) in #3783).
* **fix(networking): prevent webhook admission outages during namespace churn**: Enforce the Ingress and Pod admission webhook deny on both the caller and the webhook endpoint, so it no longer depends on Cilium performing service load-balancing ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3997).
* **fix(ingress-nginx): re-pin to the segfault-free v1.11.5 rebuild**: Re-pins the ingress controller to the rebuilt `v1.11.5`. The digest pinned before this segfaults ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4012).
* **fix(cert-manager): enable the Gateway API solver in the host cert-manager**: Enable the Gateway API solver in the host cert-manager, so ACME HTTP-01 challenges succeed when the platform Gateway is enabled ([**@lexfrei**](https://github.com/lexfrei) in #4039, shipped in v1.6.5).
* **fix(networking): harden webhook policy assumptions**: Use effective ingress admission port, report unsupported Cilium service routing correctly, and reject stale backend evidence in gateway tests ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4058).
* **fix(cacert): make the comments match the reconciler**: Comments in the CA-extraction package claim guards the code does not have. No behaviour change; one test is renamed to match the property it pins ([**@lexfrei**](https://github.com/lexfrei) in #4064).
* **fix(kubeovn): retry VM migration setup after scheduling**: Retries the VM migration network setup when the target Pod or source/target scheduling information is not yet available, instead of failing outright, and folds the Kube-OVN image into the regular in-tree build matrix — so a migration that starts before Kube-OVN has finished scheduling both endpoints no longer aborts ([**@yankawai**](https://github.com/yankawai) in #4253, shipped in v1.6.4).
* **fix(tcp-balancer): rename backends and pin HAProxy so the chart starts on 3.3+**: Rename HAProxy backends and pin the image to the chart appVersion so the balancer starts on HAProxy 3.3+. whitelistHTTP with an empty whitelist now fails the render; kubernetesAndTalos with an empty whitelist leaves ports 6443 and 50000 unfiltered ([**@yankawai**](https://github.com/yankawai) in #4366, shipped in v1.6.5).
* **fix(ingress): run the default backend on arm64**: The ingress default backend now uses the multi-arch custom-error-pages image instead of the amd64-only defaultbackend image, so it starts on arm64 nodes ([**@lexfrei**](https://github.com/lexfrei) in #4517).
### Monitoring and observability
* **fix(monitoring): send alerts to every Alertmanager replica**: vmalert now notifies each Alertmanager replica directly instead of the shared Service, so the Alertmanager cluster deduplicates alerts and a long-running alert no longer produces repeated firing and resolved notifications ([**@yankawai**](https://github.com/yankawai) in #4889, backport #4943, shipped in v1.6.5).
* **fix(monitoring): strip infrastructure metadata from tenant logs**: Container logs keep only the pod, namespace and container name of the Kubernetes metadata: the node name (`host`), labels, annotations, pod ID and image are removed from every container log, platform namespaces included, so queries and alerts on those fields must be updated ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2196).
* **fix(telemetry): make the documented opt-out cover both collectors**: Telemetry is reported by two components — `cozystack-operator` and `cozystack-controller` — but only the operator had a reachable opt-out switch; the documented `cozystackOperator.disableTelemetry` setting could never reach the controller, which kept posting application counts every 15 minutes even after an operator followed the published opt-out procedure ([**@kvaps**](https://github.com/kvaps) in #3791, shipped in v1.6.3).
* **fix(monitoring): give kube-apiserver its own scrape size limit**: Kube-apiserver is scraped with its own 128MB size limit (`vmagent.apiserverMaxScrapeSize` in the monitoring-agents component) instead of the global 32MB, so a large `/metrics` no longer makes KubeAPIDown fire on a healthy API server ([**@IvanHunters**](https://github.com/IvanHunters) in #4427).
* **fix(workloadmonitor): populate VM instance-profile label from the KubeVirt preference annotation**: `WorkloadMonitor` read a `kubevirt.io/cluster-instanceprofile-name` annotation that KubeVirt never writes; the real annotation is `kubevirt.io/cluster-preference-name`. Reading the correct key means a VM's instance profile — a Windows guest, for example — now reaches the billing metric pipeline instead of being silently dropped ([**@IvanHunters**](https://github.com/IvanHunters) in #4448, shipped in v1.6.4).
### Dashboard, API and marketplace
* **fix(marketplace): do not install a connected repository's components on connect**: Connecting a marketplace repository no longer creates a `Package` for it, so a publisher's install chart no longer runs under Flux's cluster-admin account without an operator step. Connecting only materializes the repository's `PackageSource`s; installing its applications is an explicit `cozypkg add`, after which they appear in the catalog, and a `Package` removed with `cozypkg del` is no longer re-created on the next source revision. The dashboard connects and disconnects repositories but does not install from them ([**@IvanHunters**](https://github.com/IvanHunters) in #5071, backport #5117).
* **fix(api): preserve foreign HelmRelease metadata on Application update**: Updating an Application no longer strips finalizers, ownerReferences, and foreign labels/annotations from the backing HelmRelease, which could cause deletion to skip helm uninstall and orphan the release's resources ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2914, shipped in v1.6.5).
* **fix(dashboard): re-authenticate on 401 instead of failing silently**: On a 401 (expired session) the console now sends the user to sign in again instead of silently showing a stale page. With OIDC enabled the user returns to the page they were on; without OIDC they land on the start page after entering a token ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3115).
* **fix(cozy-lib): converge the CA trust-anchor helper on the canonical name**: The CA trust-anchor helper emits the canonical `<release>.tenant-ca` name and rejects non-certificate input ([**@lexfrei**](https://github.com/lexfrei) in #3408).
* **fix(cozy-lib): round the sanitized CPU request up to a whole milli-CPU**: Rounds the sanitized CPU request up to a whole milli-CPU, so a `cpuAllocationRatio` that does not divide the limit evenly no longer renders a fractional value the API server rejects ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3522, shipped in v1.6.5).
* **fix(cozy-lib): make the cozyconfig helpers survive uninjected values**: The cozyconfig helpers now return their documented empty defaults when the platform has not injected `_cluster` / `_namespace`, instead of aborting the render, so a chart using them can be rendered and unit-tested on its own ([**@lexfrei**](https://github.com/lexfrei) in #3647).
* **fix(manifestutil): report one message when the CRD wait is cancelled**: A CRD wait cut short by a cancelled context now always reports one error naming the CRD it was waiting on, instead of sometimes including the name and sometimes not, depending on which of two exit paths the scheduler happened to pick ([**@lexfrei**](https://github.com/lexfrei) in #3742, shipped in v1.6.4).
* **fix(api): declare OpenAPIModelName for core and sdn types**: The `core` and `sdn` API groups did not declare `OpenAPIModelName` the way the `apps` group already did, so their published OpenAPI definition names were Go import paths while every `$ref` pointing at them escaped each slash — the two spellings never matched, the reference dangled, and `kubectl apply --validate` failed on any resource against a `cozystack-api` built after the underlying Kubernetes 0.35 change ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3808, shipped in v1.6.2).
* **fix(dashboard): small Console fixes from a triage pass**: Ki-range sizes now render as Ki instead of raw bytes, the tenant picker is hidden on cluster-scoped admin pages, capacity drill-downs distinguish a permission error from a broken page, an immutable field whose parent object is absent is now applied, a blocked submit scrolls to the field that blocked it including fields inside maps and the source and dropdown widgets, and cluster addons with no enable switch now say so instead of showing a toggle that never appears ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3837).
* **fix(registry): preserve write options in aggregated storage**: The application, tenant-secret and security-group APIs now preserve Kubernetes write options (dry-run, field manager, field validation, delete propagation, grace periods, preconditions) when delegating to controller-runtime, instead of silently dropping them; a rejected write also keeps its machine-readable status, so a quota or admission failure now answers 403/409 with a reason instead of a generic 500 ([**@yankawai**](https://github.com/yankawai) in #3937, shipped in v1.6.4).
* **fix(api): repair an empty required field instead of failing the release**: A field written with no value in an application spec (`registryMirrors:` with nothing under it) no longer fails the whole HelmRelease when the schema marks that field required. It is treated as unset, so the chart default applies to it ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3956, shipped in v1.6.5).
* **fix(registry): preserve dry-run in tap storage**: Honor server-side dry-run when connecting or disconnecting a tap, which previously performed the write for real ([**@IvanHunters**](https://github.com/IvanHunters) in #4225).
* **fix(dashboard): let dashboard users read backup CRD schemas**: Grants dashboard users read access to the backup CRD schemas, fixing the backup create/edit forms that previously returned 403 ([**@androndo**](https://github.com/androndo) in #4236, shipped in v1.6.4).
* **fix(dashboard): raise the console nginx header buffers for large OIDC tokens**: The console no longer answers `400 request header or cookie too large` to users whose OIDC access token carries many group claims ([**@lexfrei**](https://github.com/lexfrei) in #4376).
* **fix(api): keep a suspended HelmRelease suspended across an app update**: App edit through cozystack API no longer resumes a suspended HelmRelease, so postgres restore waits for its purge before cluster is rendered again ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4411, shipped in v1.6.5).
* **fix(api): let one ApplicationDefinition own each application kind**: An application kind is now owned by one ApplicationDefinition, the one created first. A second definition declaring a kind already in use is refused at admission, and one that predates the check no longer manages that kind's releases or its API registration ([**@IvanHunters**](https://github.com/IvanHunters) in #4424).
* **fix(api): refuse only what a name declaration actually breaks**: An ApplicationDefinition whose openAPISchema cannot be parsed no longer refuses every create of its kind as an invalid name declaration, and annotation keywords (`title`, `examples`, `x-` extensions and the like) inside `x-cozystack-name` are now ignored instead of taking the kind offline ([**@lllamnyp**](https://github.com/lllamnyp) in #4429).
* **fix(api): leave failed golden images out of the image options**: The VM image picker no longer offers golden images whose import failed. The console now reads images from the image option source, and tenants no longer get read access to PersistentVolumeClaims in cozy-public ([**@lexfrei**](https://github.com/lexfrei) in #4568).
* **fix(api): pass the stored object to UpdatedObject in tenantsecret Update**: PATCH, `kubectl label` and `kubectl edit` on an existing TenantSecret no longer fail with 404 NotFound. A strategic merge patch, which client-side `kubectl apply` sends, still fails; use a merge or JSON patch ([**@pujitha24**](https://github.com/pujitha24) in #4603).
* **fix(api): let the garbage collector finalize Applications**: An application deleted with foreground or orphan propagation no longer leaves its HelmRelease stuck with the foregroundDeletion or orphan finalizer ([**@yankawai**](https://github.com/yankawai) in #4610, shipped in v1.6.5).
* **fix(dashboard, foundationdb): show the connection ConfigMap to tenants**: The application page shows the FoundationDB connection ConfigMap with the cluster file to tenants who may read it ([**@yankawai**](https://github.com/yankawai) in #4612, shipped in v1.6.5).
* **fix(dashboard): allow editing free-form objects in application forms**: Free-form object fields, such as NATS `config.merge` and `config.resolver`, get a JSON editor. The form keeps nested objects, arrays and primitive values. Input that is not a JSON object stays visible, and schema validation blocks the create or update request until it is fixed ([**@yankawai**](https://github.com/yankawai) in #4682, shipped in v1.6.5).
* **fix(dashboard): drop the resource map ConfigMaps once the map is unreadable**: The ConfigMaps tab now goes away when the application's resource map stops being readable, instead of keeping the names from the last successful read ([**@yankawai**](https://github.com/yankawai) in #4789).
### Platform and installer
* **fix(fluxcd): keep Helm uninstall working during flux-plunger recovery**: A HelmRelease deleted while flux-plunger is recovering it now reaches the Helm uninstall instead of leaving its resources (DataVolume, PVCs, RBAC, storage Secret) behind ([**@scooby87**](https://github.com/scooby87) in #4293, backport #4995).
* **fix(fluxcd): close the flux-plunger review follow-ups for the hook, drain and Warnings**: flux-plunger's pre-delete hook now lets helm-controller run the uninstall of a release being deleted before it removes the release's fence, gives each release its own share of its retries, and reports releases deleted under another manager's suspension. The shutdown drain reports the fences it leaves as errors and refuses a deadline that leaves no time, and the README states flux-plunger's recovery order and limits ([**@scooby87**](https://github.com/scooby87) in #4914, backport #4996).
* **fix(cluster-api): resync capi-operator every minute**: A provider left not ready across a capi-operator restart now recovers in about two minutes instead of about twenty ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4939, backport #5031).
* **fix(controller): carry healthCheckExprs and waitStrategy to existing releases**: Existing HelmReleases now get `healthCheckExprs` and `waitStrategy` from their ApplicationDefinition, which cozystack-api wrote only into the releases it built, so vm-disk and postgres releases created before these expressions were added stop waiting on DataVolume and KEDA ScaledObject during upgrades. Before, an existing disk with an import still in progress failed the platform upgrade after 10 minutes and retried without end, holding a helm-controller worker. HelmReleases rendered by another chart are left alone, and a failed update is now retried instead of dropped ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5087, backport #5126).
* **fix(keycloak-configure): patch HelmRelease in release namespace on teardown**: The `keycloak-configure` pre-delete Job cleared the Flux `HelmRelease` finalizer in a hardcoded namespace that did not match where the release actually installs (`cozy-keycloak`), so its `ServiceAccount` was forbidden to patch it, the Job retried forever, and the `HelmRelease` stuck in `Terminating` — blocking any uninstall or reinstall of Keycloak ([**@lexfrei**](https://github.com/lexfrei) in #3372, shipped in v1.6.1).
* **fix(platform): restore the isp-hosted dependency closure**: Rendering the preset at `origin/main` shows the scale: **24 emitted Packages, 24 of them blocked**, by 6 missing dependencies — two more than the issue lists (`cert-manager` and `prometheus-operator-crds` are also absent). `isp-full` renders 75 Packages with a fully satisfied closure, which is a clean control ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3391).
* **fix(platform,kubernetes-nodes): fail worker-pool adoption and unpin reads closed**: The worker-pool adoption migration and the kubernetes-nodes pre-delete unpin hook fail closed on transient apiserver errors instead of misreading them as absent objects, which could otherwise let a platform upgrade prune a live worker pool or let a pool deletion leak its VMs ([**@lllamnyp**](https://github.com/lllamnyp) in #3450).
* **fix(installer): make operator readiness honest in the chart and the install gate**: The cozystack-operator Deployment no longer reports Available with zero running pods. It ran replicas: 1 with maxUnavailable: 1, which made the minimum available replica count 0, so the condition stayed True while the operator was dead and anything reading it, including the E2E install gate, saw green ([**@lexfrei**](https://github.com/lexfrei) in #3466).
* **fix(flux-shard-operator): repair sharded helm-controller crashloop behind an HTTP proxy**: The cloned `helm-controller-shard<i>` Deployment inherited `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY` from the flux-aio all-in-one wiring even though a standalone shard needs no external egress; behind an unreachable proxy the controller's blocking startup HTTPS call never completed, the manager never served `/healthz`, and every HelmRelease sharded to that controller was frozen ([**@IvanHunters**](https://github.com/IvanHunters) in #3546, shipped in v1.6.2).
* **fix(apps/kubernetes): report the pre-delete cleanup outcome honestly**: The pre-delete cleanup hook no longer reports success when a cleanup step left something behind, and no longer reports a completed teardown as incomplete when it merely fell back to force-clearing finalizers. The Job log now names the steps that left residue ([**@lexfrei**](https://github.com/lexfrei) in #3592).
* **fix(apps): make the post-delete cleanup hooks report failure**: The qdrant and harbor post-delete cleanup hooks now exit non-zero when a delete or a listing fails, instead of reporting success on every path. `helm uninstall` reports the failure rather than returning clean, and the hook Job and its pods stay in the namespace so the log survives to be read ([**@lexfrei**](https://github.com/lexfrei) in #3593).
* **fix(platform): make mongodb-application depend on its operator**: The mongodb application now waits for the mongodb operator, so a tenant can no longer create a MongoDB before the CRD that serves it exists ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3845).
* **fix(platform): correct what disabledPackages does to ouroboros**: The platform chart no longer says `bundles.disabledPackages` uninstalls a package. A name in that list only stops the Package document being rendered. `helm.sh/resource-policy: keep` leaves the existing Package CR on the cluster and the component keeps running, so deleting the Package CR is what uninstalls it ([**@lexfrei**](https://github.com/lexfrei) in #4098).
* **fix(platform): accept the container gpu-operator variant in iaas**: Accept `container` as a value for `bundles.iaas.gpuOperatorVariant`, so the gpu-operator variant the platform already ships can be selected from platform values instead of a hand-crafted Package CR ([**@lexfrei**](https://github.com/lexfrei) in #4101).
* **fix(platform): declare the injected _cluster values in system charts**: Declare the platform-injected `_cluster` and `_namespace` values in the ten system charts that read them, so each chart renders with `helm template` outside a cluster instead of failing on a nil map. No change to what an installed release renders ([**@lexfrei**](https://github.com/lexfrei) in #4102).
* **fix(apps): let the cleanup hooks watch the objects they delete**: The Harbor, ClickHouse, MariaDB and Qdrant cleanup hooks now wait for the objects they delete to actually disappear, with the wait bounded so a stuck claim cannot hang the whole uninstall ([**@yankawai**](https://github.com/yankawai) in #4135, shipped in v1.6.4).
* **fix(installer): keep node_modules out of the packages artifact**: The packages OCI artifact no longer picks up node_modules from a developer's working copy, which made the dashboard chart unloadable on every cluster that pulled the bundle ([**@IvanHunters**](https://github.com/IvanHunters) in #4290).
* **fix(build): pass the shared buildx flags from every image target**: Every package image build honours PLATFORM, SBOM and BUILDX_EXTRA_ARGS; local dashboard and metallb builds now follow the host architecture unless PLATFORM is set ([**@lexfrei**](https://github.com/lexfrei) in #4485).
* **fix(build): let a builder stage inherit its base stage's platform**: The check that builder stages compile natively is red on main, and this is a false positive. The shadowbox Dockerfile builds its webpack stage as `FROM source AS app`, where `source` is a `--platform=$BUILDPLATFORM` stage. The check reads only the literal FROM line, so it reports `app` as emulated ([**@lexfrei**](https://github.com/lexfrei) in #4545).
* **fix(platform): resume a retried migration Job from the stamped version**: A retried platform migration Job now resumes from the version stamped in the cozystack-version ConfigMap instead of re-running migrations that already finished in the same upgrade ([**@lexfrei**](https://github.com/lexfrei) in #4583).
* **fix(operator): rebuild each changed ArtifactGenerator generation once**: Rebuild an ArtifactGenerator once after each spec change, so an upgrade no longer leaves a component on artifacts built from the previous spec (for example a chart missing a library it now needs) ([**@lexfrei**](https://github.com/lexfrei) in #4594).
* **fix(build): honour PUSH and LOAD from the environment and in image-talos**: `LOAD` and `PUSH` set in the environment are now honoured, `LOAD=1` no longer pushes unless `PUSH=1` is also set, and the Talos installer is no longer published when `PUSH=0` ([**@lexfrei**](https://github.com/lexfrei) in #4654).
* **fix(installer): skip the packages artifact push and stamp with PUSH=0**: `make image-packages` no longer pushes the cozystack-packages artifact or stamps its digest into values.yaml when PUSH=0 ([**@lexfrei**](https://github.com/lexfrei) in #4663).
* **fix(platform): keep the legacy etcd-operator stopped after etcd adoption**: Keep the legacy etcd-operator stopped after migration 50 adopts the legacy etcd clusters ([**@yankawai**](https://github.com/yankawai) in #4738, shipped in v1.6.5).
* **fix(platform): stop treating a failed ConfigMap read as a fresh cluster**: A transient API error while the platform migration Job checks for the cozystack-version ConfigMap now fails the pod so the Job retries, instead of stamping the target version and silently skipping every pending migration ([**@lexfrei**](https://github.com/lexfrei) in #4792).
* **fix: typo in monitoring-agents values.yaml**: Fixes a typo in the monitoring-agents default values ([**@gireassen**](https://github.com/gireassen) in #4799).
## Dependencies & version updates
* **fix(cilium): bump vendored chart to v1.20.2**: Bump to v1.20.2, fixing Gateway API HTTPRoute attachment on shared Gateways (cilium/cilium#45693). This changes behaviour for every cilium Gateway, including Gateways inside tenant Kubernetes clusters. The cilium-operator image is patched in two places ([**@lexfrei**](https://github.com/lexfrei) in #3343).
* **fix(postgres-operator): align CNPG operator and CRDs to 1.28.2 for PVC resize-deadlock fix**: A simultaneous `resources` + `size` change on a single-instance PostgreSQL cluster could make the CloudNativePG operator delete the sole primary Pod, classify the PVC as `resizing`, and never recreate the Pod — wedging the cluster with zero instances and leaving the filesystem resize incomplete ([**@scooby87**](https://github.com/scooby87) in #3510, shipped in v1.6.1).
* **chore(etcd-operator): bump etcd-operator to v0.5.4**: Rolls up four upstream controller bug fixes: the operator no longer exempts the bootstrap seed from crash-loop self-heal, self-heal now also covers memory-backed etcd members, `--initial-cluster-state` is derived from cluster phase instead of the seed, and each EtcdCluster's PodDisruptionBudget switches from `maxUnavailable` to `minAvailable` (existing clusters are reconciled onto the new field automatically on upgrade) ([**@androndo**](https://github.com/androndo) in #3529, shipped in v1.6.1).
* **feat(gateway): update the CRD bundle to Gateway API v1.6.1 and gate GRPCRoute hostnames**: Update the vendored Gateway API CRD bundle to v1.6.1. TCPRoute and UDPRoute gain a served v1 that becomes their storage version, with v1alpha2 still served and now deprecated; every other kind keeps the versions it served at v1.5.1 ([**@lexfrei**](https://github.com/lexfrei) in #3872).
* **fix(cdi): update CDI to v1.66.1**: Updates CDI to v1.66.1, fixing HTTP imports that block volumes on 4Kn devices (DRBD/LINSTOR) via a repaired `qemu-img`, and a `resourceVersion` update storm on prime PVCs during `WaitForFirstConsumer` imports ([**@lexfrei**](https://github.com/lexfrei) in #3920, shipped in v1.6.4).
* **fix(postgres-operator): bump CloudNativePG to 1.30.0 for the Cluster scale-subresource selector**: Bump CloudNativePG to 1.30.0 so the Cluster scale subresource serves status.selector, enabling HPA-driven read-replica autoscaling (previously the HPA failed with InvalidSelector on 1.28.2) ([**@scooby87**](https://github.com/scooby87) in #3951).
* **feat(etcd-operator): bump to v0.5.5 and vendor EtcdDefragPolicy CRDs**: Bumps `etcd-operator` to v0.5.5, adding the `EtcdDefrag` and `EtcdDefragPolicy` APIs for on-demand and scheduled etcd defragmentation ([**@androndo**](https://github.com/androndo) in #3968, shipped in v1.6.4).
* **chore(keycloak): bump KMS proxy to 0.2.2**: Updates `keycloak-kms-proxy` to v0.2.2 and pins the image to its published digest, fixing decryption for reads whose result set spans a join ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3995, shipped in v1.6.3).
* **chore(kubevirt): update KubeVirt to v1.9.0**: Update KubeVirt to v1.9.0. Beta feature gates are on by default upstream now. The platform keeps `Template` off, which would deploy virt-template, and `ExternalNetResourceInjection`, which would hand `resourceName` injection for NetworkAttachmentDefinitions to an injector the platform does not run ([**@lexfrei**](https://github.com/lexfrei) in #4015).
* **chore(seaweedfs): bump chart to 4.45**: Bump the vendored SeaweedFS chart to 4.45, with the Lance and Iceberg table catalogs kept off and both gates of the volume resize hook comparing sizes numerically. Every SeaweedFS server moves to the 4.45 image, so the master, volume and filer StatefulSets and the s3 Deployment roll one pod at a time on upgrade; the image is the in-tree patched build described under #4819 rather than the stock upstream one ([**@IvanHunters**](https://github.com/IvanHunters) in #4079).
* **fix(cdi): pin the CDI release kubevirt-cdi is reconciled against and stop update overwriting it**: `make update` in kubevirt-cdi no longer overwrites the hand-maintained CDI object. It pins the CDI release that object is reconciled against instead of resolving GitHub's releases/latest at run time, and verifies the fetched asset against a pinned sha256 without writing under templates/ ([**@lexfrei**](https://github.com/lexfrei) in #4107).
* **feat(kube-ovn): bump kube-ovn to v1.15.26**: Bump kube-ovn to v1.15.26, which removes the OVN leader-checker integrity check that could delete a healthy raft DB and prevent ovn-central from electing a leader after a node address change or rolling restart ([**@IvanHunters**](https://github.com/IvanHunters) in #4331).
* **fix(kubevirt-cdi-operator): pin the vendored CDI release**: Pin the vendored CDI release to v1.66.1 and verify the downloaded manifest against a digest, so refreshing the package no longer moves the deployed CDI. The deployed version stays the same ([**@lexfrei**](https://github.com/lexfrei) in #4383).
* **chore(etcd-operator): bump to v0.5.6**: Bump to v0.5.6, which replaces EtcdMember Pods left in a terminal phase after a node shutdown and keeps EtcdCluster Available/Degraded conditions truthful while etcd is unreachable ([**@androndo**](https://github.com/androndo) in #4425).
* **chore(deps): roll up pending Renovate updates and move to Go 1.27.1**: Move the toolchain and builder images to Go 1.27.1 and raise Go dependencies to their latest compatible releases, closing the published advisories; the lineage webhook no longer rewrites int64 fields above 2^53 in admitted objects ([**@lexfrei**](https://github.com/lexfrei) in #4447).
* **chore(deps): move to the Kubernetes 1.37 libraries without the apimachinery fork**: Cozystack-api and the controllers build against the Kubernetes 1.37 libraries without the apimachinery fork. Plain watches no longer receive the WatchList initial-events-end bookmark, and objects in Table rows requested with includeObject=Object may carry another app's kind ([**@lexfrei**](https://github.com/lexfrei) in #4473).
* **chore(hetzner-robotlb): update robotlb to 0.1.0**: Update robotlb to 0.1.0 and pull it from the new treetscom organization. Load balancer targets now follow the Service's externalTrafficPolicy; set it to Local to keep targeting only the nodes that run the Service's pods ([**@sircthulhu**](https://github.com/sircthulhu) in #4557, shipped in v1.6.5).
* **fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2**: Velero-plugin-for-aws is bumped from v1.14.1 to v1.14.4, which no longer sends an empty x-amz-tagging header, so Velero backups work against S3-compatible stores that reject it, such as Cloudflare R2 ([**@yrahman82**](https://github.com/yrahman82) in #4834, shipped in v1.6.5).
## System Configuration
* **[cilium] Set resources requests/limits on cilium-operator**: Sets resource requests and limits on `cilium-operator`, which the upstream chart ships empty, so the Deployment stops triggering a no-CPU-limit conformance warning ([**@matthieu-robin**](https://github.com/matthieu-robin) in #2618).
* **fix(reloader): add startupProbe to survive slow first start under load**: Adds a startup probe to `cozy-reloader` so a slow first start is no longer killed by the liveness probe and left crash-looping, which also unblocks the LINSTOR stack that depends on it starting cleanly ([**@lexfrei**](https://github.com/lexfrei) in #3470, shipped in v1.6.4).
* **feat(operator): default system namespace memory via LimitRange**: System namespaces now get a default container memory limit through a LimitRange maintained by cozystack-operator, which keeps system components out of the Talos userspace OOM handler's victim set ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3494).
* **fix(platform): give node DaemonSets memory requests and limits**: Node DaemonSets (cilium, metallb speaker and frr-k8s, kube-ovn, linstor-satellite except its drbd-reactor sidecar, virt-handler, fluent-bit, node-exporter, multus, velero node-agent, kubevirt-csi-node) now declare memory requests and limits, so they are no longer selected as victims by the Talos userspace OOM handler when an unrelated workload drives node memory pressure; multus's own limits were later reworked in #4830 ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3495).
* **fix(monitoring-agents): give kube-state-metrics a sized memory limit**: Kube-state-metrics now carries a memory request and limit sized from a measured peak, so it is no longer a permanent candidate for the node OOM handler ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3846).
* **fix(cilium): size the agent memory limit for its BPF maps**: Set a 24Gi cilium-agent memory limit (v1.6.5 set none) so it covers the agent's BPF maps, which cilium sizes from node RAM. On nodes with around a terabyte of RAM or more, the 4Gi limit that #3495 had set got the agent OOM-killed on every start after a reboot, leaving the node without pod networking ([**@lexfrei**](https://github.com/lexfrei) in #4803).
## Development, Testing, and CI/CD
* **docs(release): add changelog for v1.6.5**: Adds `docs/changelogs/v1.6.5.md` to this release line, with a note that small PostgreSQL clusters restart on the upgrade to v1.6.5 for the new `shared_buffers` default ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in #4940, backport #5105).
* **fix(hack): keep the backup stack in the vminstance selective install**: An E2E run that selects only the `vminstance` suite now installs the backup stack, so its VMInstance and VMDisk backup round-trips run instead of timing out on the BackupStorageLocation wait ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5027, backport #5069).
* **ci(workflows): open the E2E breakpoint for maintainer-authored PRs**: Open the E2E SSH breakpoint automatically on failed maintainer-authored PRs, without requiring the `debug` label or moving the jobs onto self-hosted runners ([**@kvaps**](https://github.com/kvaps) in #3158).
* **chore(codeowners): scope CODEOWNERS to per-area reviewers**: Replaces the single catch-all owner line in `.github/CODEOWNERS` with area-scoped reviewer ownership, so each part of the tree routes to the people who actually own it (per the Contributor Ladder), instead of requiring the whole roster on every change ([**@kvaps**](https://github.com/kvaps) in #3228).
* **fix(ci): restore e2e for fork PRs and close the merge-gate bypass**: Fork pull requests now run e2e via a privileged workflow, and a fork PR can no longer merge on a skipped (never-run) required E2E Tests check ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3262).
* **fix(hack): declare the shared package targets and the root test target phony**: Declares the shared package targets and the root test target as phony, so `make` no longer treats a same-named file or directory as up to date ([**@lexfrei**](https://github.com/lexfrei) in #3353).
* **docs(agents): widen overview.md routing and condense activation rules**: `AGENTS.md` routes an agent to a documentation file based on how a request is phrased. Two things were wrong with that routing ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3373).
* **feat(release): cutting an rc freezes the line into release-X.Y**: Cutting a release candidate now creates the `release-X.Y` branch and freezes the release line. Features merged to `main` after the first rc no longer ship in that release; fixes must be cherry-picked or backported onto `release-X.Y` ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3399).
* **chore: ignore root-level Go binaries built from cmd/**: Adds the root-level binary names that `go build ./cmd/<name>` (without `-o`) drops in the repository root to `.gitignore`, so `git add .` cannot accidentally commit an ~80 MB executable ([**@lexfrei**](https://github.com/lexfrei) in #3412).
* **fix(e2e): improve failure diagnostics**: Improve diagnostic capture for dataplane, Talos, and COSI failures ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3421).
* **ci: drop unused packages:write from the pull-request workflow**: The `build`, `build-talos` and `finalize` jobs in `pull-requests.yaml` each requested `packages: write`, but none of them touches GHCR: the workflow's `REGISTRY` is OCIR and they authenticate with `OCIR_USER` / `OCIR_TOKEN` ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3422).
* **docs(release): add changelog for v1.6.0**: Adds `docs/changelogs/v1.6.0.md` (667 lines), generated locally with the process in `docs/agents/changelog.md`. Normally `tags.yaml`'s `generate-changelog` job produces this on the stable tag push, but that job never ran for `v1.6.0` — see the note below ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3429).
* **test(e2e): make kuberture split-horizon probes survive Cilium ingress-IP race**: The split-horizon-routing-proof step ran two external-dns probes as bare Pods and polled them by fixed name ([**@lexfrei**](https://github.com/lexfrei) in #3430).
* **fix(release): make promote-retag digest verification media-type-agnostic**: The v1.6.0 finalize run aborted partway through promoting rc images to stable tags because its post-copy digest check used `skopeo inspect --format '{{.Digest}}'`, which prints nothing for OCI artifacts like `cozystack-packages`, leaving most repositories without a stable tag and skipping the installer publish ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3435, shipped in v1.6.1).
* **fix(ci): decide E2E test coverage per changed path**: A changed package that no E2E suite covers now runs the full suite even when the same pull request also edits an unrelated suite, and changes to the kuberture and securitygroup packages select their own suite instead of escalating ([**@lexfrei**](https://github.com/lexfrei) in #3441).
* **fix(e2e): keep failure snapshots complete and gate tenant patch on aggregated-API auth**: Keep failure snapshots complete when PodSecurity blocks crust-gather's host-log leg, and wait for the aggregated API to authenticate before patching the root tenant so the install step no longer flakes on a system:anonymous 403 ([**@lexfrei**](https://github.com/lexfrei) in #3444).
* **ci(release): move release validation to rc time — e2e lane, changelog button, promote gate, docs at promote**: Moves release validation from promote time to rc time, making promotion itself a pure retag operation ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3449).
* **refactor(ci): drop the vestigial /tmp workspace from e2e (both workflows)**: Drop the vestigial /tmp/$SANDBOX_NAME workspace from the E2E PR and nightly jobs — they now run in $GITHUB_WORKSPACE, removing an unguarded `rm -rf /tmp/$SANDBOX_NAME` cleanup that failed the job on cleanup when SANDBOX_NAME was empty ([**@kvaps**](https://github.com/kvaps) in #3454).
* **fix(e2e): disable guest fsync on ephemeral CI VM disks**: Boots e2e QEMU guests with `cache=unsafe` so ephemeral CI VM fsyncs no longer stall the management-cluster etcd, removing cross-operator leader-election flakiness that was failing a different chainsaw suite on every run ([**@kvaps**](https://github.com/kvaps) in #3455, shipped in v1.6.4).
* **fix(ci): overlay images from the PR base branch, and publish per-line artifacts**: PR validation now overlays unbuilt packages from the PR's own base branch, so a `release-X.Y` PR is tested against its own line's charts instead of `main`'s controller binaries ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3471, shipped in v1.6.1).
* **chore(codeowners): don't gate per-app generated files on architect review**: Per-app generated API type mirrors and cozyrds manifests no longer require architect review on their own — only the source values.yaml they're generated from does, with the existing make-generate CI check guarding against drift ([**@lllamnyp**](https://github.com/lllamnyp) in #3507).
* **fix(ci): escalate unclassified paths in TIA instead of skipping e2e**: Test-Impact Analysis now classifies every changed path explicitly and escalates an unrecognised one to the full E2E suite ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3511).
* **feat(release): add backport-audit, a pre-release backport triage tool**: Add `backport-audit`, a maintainer tool that reports which `kind/backport` / `kind/backport-previous` PRs have not landed on a release branch (missing outright, partially backported, waiting on an open backport PR, or deliberately dropped) and exits non-zero while anything is outstanding, so a patch cut can gate on it ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3527).
* **fix(e2e): capture Talos logs for tenant worker join failures**: Capture authenticated Talos kernel and kubelet diagnostics when tenant Kubernetes workers miss the Ready deadline ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3548).
* **docs(release): add changelog for v1.6.1**: This PR adds the changelog for release `v1.6.1` ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in #3551).
* **fix(e2e): select catch pod logs by the nested release instance label**: Collect harbor, openbao and qdrant pod logs on a failed run ([**@lexfrei**](https://github.com/lexfrei) in #3564).
* **fix(e2e): collect diagnostics for unready pods and record every truncated capture**: The diagnostic report now collects per-pod evidence for pods that are Running but not Ready, which it previously skipped, and several collector sections that silently produced nothing now report correctly ([**@lexfrei**](https://github.com/lexfrei) in #3567).
* **fix(ci): stop labeled events from cancelling and re-running backports**: Stop `labeled` pull request events from cancelling the automatic backport run already in flight. A burst of labels from a third-party app could previously kill the backport the merge had just started, leaving the work to be redone by whichever label arrived last ([**@lexfrei**](https://github.com/lexfrei) in #3569).
* **feat(ci): auto-apply triage labels to issues**: New and reopened issues now get a `triage/*` label automatically, and a daily sweep labels any open issue that has none: `triage/accepted` when it is assigned or carries an `epic` or a priority above `priority/backlog`, `triage/needs-triage` otherwise ([**@lexfrei**](https://github.com/lexfrei) in #3574).
* **fix(e2e): budget tenant scheduling separately from workload readiness**: Waits for a tenant node to actually accept a Pod, on its own budget, before creating the workload used by the kubernetes suites, instead of sharing one 300s deadline between node scheduling and workload readiness — fixing intermittent timeouts where scheduling alone consumed most of the shared window ([**@lexfrei**](https://github.com/lexfrei) in #3579, shipped in v1.6.4).
* **fix(tests): freeze the EXIT-trap count for multus-install-cni-plugins.bats**: `hack/multus-install-cni-plugins.bats` arrived on main in 7739d1e29 carrying 12 EXIT traps, one day before the EXIT-trap ratchet in `hack/cozyreport.bats` was written, so the frozen list never learned about it ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #3584).
* **test(tests): pin the keep annotation on cert-manager and etcd-operator CRDs**: `cert-manager-crds` and `etcd-operator-crds` are the two packages this repository points at as the in-tree precedent for protecting CRDs with `helm.sh/resource-policy: keep`. Both carry the annotation, neither had a test, so it could be lost with nothing reporting the loss ([**@lexfrei**](https://github.com/lexfrei) in #3590).
* **docs(ci): correct the job-level permissions comment**: The comment above the top-level `permissions` block in the workflow files now says a job-level block replaces it instead of adding to it ([**@lexfrei**](https://github.com/lexfrei) in #3591).
* **fix(e2e): fail on a failed release of cilium and csi as on the parent, and report addon remediation**: The HelmRelease history guard now also covers the tenant addon releases. A failed or uninstalled Snapshot fails the run on the parent, cilium and csi, which retry in place; other addons get a note. The same classification is printed on the failure paths where the guard itself cannot run ([**@lexfrei**](https://github.com/lexfrei) in #3594).
* **fix(e2e): bound every read in the report collector and the step that runs it**: Every read in the diagnostic report collector now carries a wall-clock ceiling — module gates, the reads that run a command inside a LINSTOR pod, and the host commands included — and a ceiling that fires is recorded in the artifact rather than only in the job log ([**@lexfrei**](https://github.com/lexfrei) in #3596).
* **fix(tests): freeze the EXIT-trap count for run-kubernetes-talos-diagnostics_test.bats**: `hack/run-kubernetes-talos-diagnostics_test.bats` landed on main carrying eight EXIT traps and was not added to the frozen set in `hack/cozyreport.bats`, so `found` and `frozen` differ by exactly that one entry and the guard fails on every branch cut from main since then ([**@lexfrei**](https://github.com/lexfrei) in #3602).
* **test(backupstrategy-controller): anchor the metrics-absence guard to its container**: The `metrics.enabled=false` case in `deployment_test.yaml` guards against the `metrics.enable` typo regression by asserting that the metrics arg and port are absent. Both assertions address the controller by index, and `notContains` passes whenever its path resolves to something that lacks the forbidden content ([**@lexfrei**](https://github.com/lexfrei) in #3605).
* **test(apps): anchor index-addressed negative asserts to what they name**: Four negative assertions in two suites address an element by index. `notContains` passes whenever its path resolves to something that simply lacks the forbidden content, and a different element at the same index qualifies ([**@lexfrei**](https://github.com/lexfrei) in #3606).
* **test(e2e): pin the release-history read against the upstream type**: `hack/remediation-guard.bats` carried a test called "status.history extraction pins HR v2 status.history shape". It wrote a HelmRelease document containing the field names it expected, then read that same document back with the same jsonpath through `yq` ([**@lexfrei**](https://github.com/lexfrei) in #3607).
* **test(system): anchor index-addressed negative asserts to what they name**: Some negative assertions in these suites can't fail for the reason their test names give ([**@lexfrei**](https://github.com/lexfrei) in #3608).
* **ci(backport): skip a target whose backport already merged**: Skip re-running a backport whose target already has the merged change, instead of posting a false failure comment ([**@lexfrei**](https://github.com/lexfrei) in #3616).
* **test(e2e): capture host-netns address attribution and a healthy-pod reference for the dataplane collector**: Capture host netns address attribution and an in-run healthy-pod reference for the dataplane failure collector ([**@lexfrei**](https://github.com/lexfrei) in #3617).
* **docs(hack): fix a wrong claim about grep-error propagation in two contract tests**: `hack/promote-gate-contract.bats` and `hack/release-freeze-contract.bats` both carried the same wrong comment above their `code_lines` helper: it said a real grep error (exit 2) fails a pin through the helper's own exit status, distinct from an empty match (exit 1) ([**@lexfrei**](https://github.com/lexfrei) in #3620).
* **test(tests): let each bats file declare the EXIT traps it still holds**: Each hack/*.bats file now declares its own remaining EXIT-trap debt in a `# EXIT-TRAP DEBT: N` header comment, checked by hack/bats-no-exit-trap.bats, replacing the central inventory in hack/cozyreport.bats ([**@lexfrei**](https://github.com/lexfrei) in #3622).
* **test(cozystack-controller): guard the Secret grant by effective permission**: `packages/system/cozystack-controller/tests/rbac_test.yaml` has a test named "does not grant a blanket wildcard verb on secrets" that does not guard that ([**@lexfrei**](https://github.com/lexfrei) in #3627).
* **fix(tests): fail when a chart runs no Helm unit test suites**: `make helm-unit-tests` now fails a chart that runs no Helm unit test suites, instead of reporting success for a chart whose suite files were deleted, moved, or renamed out of the `tests/*_test.yaml` glob ([**@lexfrei**](https://github.com/lexfrei) in #3632).
* **test(tenant, harbor, seaweedfs, api): anchor guards that pass on a missing document**: Five helm-unittest suites asserted things that stayed green when the thing they guard disappeared. No chart or template logic changes here — only the tests, and each fix is measured in both directions ([**@lexfrei**](https://github.com/lexfrei) in #3639).
* **fix(hack): arm cozytest cluster captures for e2e suites only**: Cozytest.sh no longer arms its cluster-capture hooks for unit BATS suites, so a failing unit test fails fast instead of waiting out capture timeouts against a cluster that is not there ([**@lexfrei**](https://github.com/lexfrei) in #3645).
* **test(tenant): give these template lists their real reason**: Two helm-unittest suites in `packages/apps/tenant/tests/` said they restrict rendering because `namespace.yaml`'s `lookup` errors under helm-unittest. It does not ([**@lexfrei**](https://github.com/lexfrei) in #3651).
* **fix(tests): bound the e2e data-plane capture's reads and stop inferring absence from silence**: Bound every kubectl read in the e2e data-plane capture so a hung read no longer costs the whole capture, and stop the collector reporting an unanswered read as an absent pod, service or reachable address ([**@lexfrei**](https://github.com/lexfrei) in #3659).
* **test(tenant): anchor the operator-gate absence and refresh a spent reason**: Two tests in `packages/apps/tenant/tests/vmo_webhook_dependson_test.yaml` say the seaweedfs and gateway HelmReleases deliberately carry no operator gate, and assert that with `notExists: spec.dependsOn` and nothing else ([**@lexfrei**](https://github.com/lexfrei) in #3661).
* **docs(tests): correct five stale claims in the e2e capture tooling**: Five things in the e2e tooling said something the code no longer does, or did not say it at all. All five are prose; no executable line changes ([**@lexfrei**](https://github.com/lexfrei) in #3662).
* **test(ci): pin the label-writing workflows with contract tests**: Adds `hack/label-workflows-contract.bats`, following the pattern of `hack/promote-gate-contract.bats` / `hack/release-freeze-contract.bats` (same helpers: comment-stripped pins over job/input blocks, one filter per language since `pr-labeler.yaml` mixes YAML `#` and JavaScript `//` comments) ([**@shaurya703**](https://github.com/shaurya703) in #3664).
* **fix(ci): close the silent-green gaps in e2e selection and the helm unit sweep**: Select-e2e no longer reports an empty selection when it cannot enumerate the Chainsaw suites, which made an escalation to the full suite indistinguishable from "skip Chainsaw" and left the E2E gate green with nothing run ([**@lexfrei**](https://github.com/lexfrei) in #3670).
* **fix(e2e): stop the data-plane capture reporting what it never observed**: The host->pod data-plane capture no longer reports absence or reachability it never observed -- truncated list records are dropped instead of parsed, a LoadBalancer no probe reached is recorded as unprobed rather than reachable, an unread EndpointSlice backend is distinguished from an absent one, a tcpdump leg whose node was never identified is skipped instead of aimed at an unscheduled pod, and both callers report a capture their backstop cut short ([**@lexfrei**](https://github.com/lexfrei) in #3672).
* **test(release): clean up the guard test's scratch dir without an EXIT trap**: `make unit-tests` fails on a clean `main`: `hack/bats-no-exit-trap.bats` reports `release-freeze-contract.bats` as holding one EXIT trap with no declaration ([**@lexfrei**](https://github.com/lexfrei) in #3673).
* **fix(e2e): bound the tenant failure diagnostics per read and as a phase**: Bound every read in the tenant Kubernetes node-join failure diagnostics and give the collection phase its own wall-clock budget, so a wedged apiserver can no longer hold the Chainsaw op until it is killed and take the tenant crust-gather snapshot with it ([**@lexfrei**](https://github.com/lexfrei) in #3676).
* **fix(e2e): keep a failed image-mirror lookup from pinning the public factory**: A single failed apiserver lookup no longer sends a whole e2e run to the public Talos Image Factory ([**@lexfrei**](https://github.com/lexfrei) in #3677).
* **fix(e2e): budget each linstor wait separately in post-install-prep**: Gives each of the two LINSTOR waits in the post-install-prep script its own deadline, started when the previous one finishes, instead of sharing a single 15-minute window anchored at the script's start — fixing failures where a slow HelmRelease left the following Deployment wait too little time ([**@lexfrei**](https://github.com/lexfrei) in #3715, shipped in v1.6.4).
* **test(e2e): capture worker CPU throttling counters on node-join failure**: Capture tenant worker CPU throttling counters and their CFS ceiling when a node-join fails, so a worker held at its own limit can be told apart from one losing host CPU ([**@lexfrei**](https://github.com/lexfrei) in #3723).
* **chore(codeowners): keep architects as owners in every non-governance rule**: @kvaps and @lllamnyp are now code owners for every path except governance files, removing single-approver review bottlenecks; a unit test guards the invariant ([**@kvaps**](https://github.com/kvaps) in #3738).
* **fix(e2e): stop the sandbox cluster CIDR overlapping the pod range**: Moves the e2e sandbox's Kubernetes cluster CIDR off the range Kube-OVN uses for pod IPAM, fixing intermittent Cilium/Kube-OVN IP collisions that either refused a pod outright with "IP already in use" or silently broke host-to-pod traffic ([**@lexfrei**](https://github.com/lexfrei) in #3750, shipped in v1.6.4).
* **test(e2e): warm the ghcr.io mirror before the tenant suites need it**: Warms the ghcr.io mirror before the tenant suites need it, removing a frequent e2e flake where a tenant worker missed its node-Ready budget with no Nodes registered ([**@lexfrei**](https://github.com/lexfrei) in #3768).
* **fix(tests): let the mariadb waits cover a first boot**: The MariaDB, MongoDB, RabbitMQ and ClickHouse backup examples and the MariaDB Chainsaw suites now wait for a HelmRelease past the release's own install timeout, so a failed install reports its own reason instead of a bare timeout, and the examples' HelmRelease wait now tells a release that never appeared, one deleted mid-wait and a kubectl that could not answer apart ([**@lexfrei**](https://github.com/lexfrei) in #3769).
* **test(tests): make negated BATS assertions able to fail**: A hundred and twelve assertions across twelve BATS suites begin with `!`. A hundred and seven of them pass no matter what they find ([**@lexfrei**](https://github.com/lexfrei) in #3790).
* **fix(release): publish rewritten packages during RC promotion**: Publish the rewritten stable packages artifact when promoting a release candidate ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3795).
* **test(e2e): hold the tenant-cluster install wait to an ordering rule**: A test that creates a tenant `Kubernetes` cluster must not end while that release is still installing. The kind drops the install wait on its main resources, but the chart's blocking post-install hook still holds the install open, so a test that asserts and ends hands the rest of it to teardown ([**@lexfrei**](https://github.com/lexfrei) in #3802).
* **fix(e2e): capture both ends of the worker byte path on node-join failure**: A tenant worker that misses the node-Ready deadline while its kubelet image pull crawls leaves two mechanisms behind one symptom, and the failure artifact could not tell them apart: either the link is slow, or the pull keeps restarting and discarding what it fetched ([**@lexfrei**](https://github.com/lexfrei) in #3804).
* **fix(ci): de-escalate select-e2e and name every escalation**: I replayed the 150 most recently merged pull requests through `hack/select-e2e.sh` with every escalation branch instrumented. 118 of them (78.7%) ran the full 21-suite Chainsaw run, and the causes, counted once per escalating PR with the count where that cause was the only one in brackets, were `FULL_PATTERN` 88 [68], `NO_SUITE_FOR_GROUP` 24 [24], `CHAINSAW_SHARED` 18 [1] and `UNCLASSIFIED` 7 [4] ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3817).
* **fix(ci): give the nightly e2e runner headroom over its guest vCPUs**: Run the nightly e2e suite on a runner class larger than the sandbox's guest vCPU demand, and add a unit check that fails when an e2e lane loses that headroom ([**@lexfrei**](https://github.com/lexfrei) in #3833).
* **test(e2e): stop the mongodb backup suite before the restore leg**: The MongoDB backup suite now stops after a successful BackupJob; the to-copy restore leg stays uncovered until the credential handling behind it is fixed ([**@lexfrei**](https://github.com/lexfrei) in #3834).
* **test(e2e): measure tenant worker CPU as a rate, and the sandbox nodes under it**: The node-join failure diagnostics now read the tenant worker CPU counters twice a fixed interval apart, so consumed CPU and throttling are both available as a rate over the same window instead of as an average over the container's uptime, and they add the sandbox node's own /proc/stat over the Talos API, which is the only place the steal time under the sandbox is recorded ([**@lexfrei**](https://github.com/lexfrei) in #3836).
* **test(e2e): give tenant workers two vCPUs**: Tenant workers in the kubernetes e2e suites run with two vCPUs, since measurements showed the single-vCPU worker saturating its CPU ceiling for the entire node-join budget ([**@lexfrei**](https://github.com/lexfrei) in #3839).
* **fix(e2e): size tenant workers by resources, not by u1.large**: Tenant workers in the kubernetes e2e suites keep their two vCPUs but go back to 4Gi of memory, since the larger instancetype left workers unschedulable on the sandbox ([**@lexfrei**](https://github.com/lexfrei) in #3840).
* **test(e2e): time the guest console capture and baseline it on a green run**: Stamp each tenant worker guest console capture with the instant it was read and the instant its guest started, and take the same capture on passing runs so a failing boot has a healthy one to be compared against ([**@lexfrei**](https://github.com/lexfrei) in #3841).
* **test(e2e): split the tenant worker CPU time across QEMU threads**: On tenant node-join failure, capture how the compute container's CPU time splits across QEMU's vCPU, emulator and IO threads ([**@lexfrei**](https://github.com/lexfrei) in #3843).
* **chore(ci): namespace the backport labels without breaking the trigger**: Namespaces the backport labels without breaking the backport trigger, finishing an earlier rename ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3844).
* **test(tests): preserve Bats failure reports during fixture cleanup**: Test-level `EXIT` cleanup overrides the Bats TAP handler, so a failed assertion can disappear from test results. This moves cleanup in eight unit files after their assertions. Failed tests keep scratch files for inspection ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3848).
* **test(tests): run unit suites with Bats and retain POSIX coverage**: This runs the unit suite with Bats and keeps a separate `dash` pass for tests that source production shell helpers. Unit files are discovered from the filesystem, and each loads the audited helper that restores `set -u` in test bodies without changing subprocess shell options ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3849).
* **fix(e2e): drop the worker CPU limit until it can carry its own request**: Tenant workers in the kubernetes e2e suites no longer set a CPU limit, since without a paired request it defaulted the scheduler request to the full ceiling and left the workers unschedulable ([**@lexfrei**](https://github.com/lexfrei) in #3860).
* **test(e2e): run tenant workers on one vCPU with CPU headroom above it**: Tenant workers in the kubernetes e2e suites run on one vCPU with a CPU limit of two, testing whether the node-join freeze is a spin between sibling vCPUs ([**@lexfrei**](https://github.com/lexfrei) in #3863).
* **test(e2e): capture the sandbox kernel KVM exit counters across the node-join wait**: Capture the runner kernel's KVM exit counters on either side of the tenant node-join wait, so a node-join failure can be told apart from a nested-virtualisation slowdown that no counter inside the sandbox can see ([**@lexfrei**](https://github.com/lexfrei) in #3886).
* **test(e2e): capture worker block IO and volume state on node-join failure**: The node-join failure path already records what a stuck tenant worker was allowed and what it got ([**@lexfrei**](https://github.com/lexfrei) in #3888).
* **test(e2e): record the runner VM shape and region in the e2e report**: Every e2e run now records the runner VM it landed on (OCI shape, both region keys, availability and fault domain, shape sizing numbers, kernel release, CPU count and processor model) in the job log and in cozyreport.tgz, so red and green runs can be compared by hardware ([**@lexfrei**](https://github.com/lexfrei) in #3889).
* **test(e2e): price the layers above the sandbox on the node-join window**: A tenant node-join failure now records the runner kernel's own CPU time and the sandbox VMs' per-thread QEMU CPU time on either side of the join wait, so steal on the layer above the sandbox and an uneven split across the three nodes are both visible in cozyreport.tgz ([**@lexfrei**](https://github.com/lexfrei) in #3890).
* **chore(e2e): ignore the artifacts the sandbox bringup writes to the repo root**: Adds `.gitignore` entries for the Talos secrets bundle, rendered machine configs, kubeconfigs, boot image and per-VM directories a local e2e sandbox bringup writes to the repo root, closing a path where a broad `git add` could have staged cluster private keys ([**@lexfrei**](https://github.com/lexfrei) in #3891, shipped in v1.6.4).
* **test(e2e): anchor the phase-warning guard on its own unique phrase**: The guard in the CPU throttle suite finds the missing-timeout phase warning by the first match of a phrase that stopped being unique when the bracket-pair captures started writing their own [bounds] notes with the same words ([**@lexfrei**](https://github.com/lexfrei) in #3897).
* **docs(release): add changelog for v1.6.2**: This PR adds the changelog for release `v1.6.2` ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in #3907).
* **fix(release): compare host-less image refs on digest alone**: Release verification compares host-less image references on digest alone, so mirrored images are no longer reported as mismatching ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3908).
* **docs(release): add the v1.5.4 changelog, and fix v1.6.0's fail-open audit note**: Adds the changelog for `v1.5.4`, and corrects one still-live instruction in v1.6.0's ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in #3910).
* **chore(ci): add area/cilium label and map the cilium PR scope to it**: Adds an `area/cilium` label and routes the `cilium` PR scope to it ([**@lexfrei**](https://github.com/lexfrei) in #3914).
* **test(e2e): add a runner-layer fixed-work canary around the node-join wait**: The node-join diagnostics now time two fixed amounts of work on the runner before and after the wait, so a run that spent its wall clock without completing work is visible in the report. No user-facing change ([**@lexfrei**](https://github.com/lexfrei) in #3919).
* **fix(ci): resolve fork PRs by open-PR head SHA, not commit association**: Fork PR e2e lane never ran. `resolve` looks up PR for head sha with `listPullRequestsAssociatedWithCommit` on base repo, and this endpoint returns nothing for fork PR head, because fork commit reaches base repo only as `refs/pull/<n>/head` and association index doesn't walk `refs/pull/*` ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3922).
* **test(e2e): capture CertificateSigningRequests in cozyreport**: Hack/cozyreport.sh never captured CertificateSigningRequests, so the hypothesis that a node join hangs on kubelet certificate issuance (bootstrap token, CSR, approval) had no artifact to check it against ([**@lexfrei**](https://github.com/lexfrei) in #3925).
* **ci(workflows): make the E2E breakpoint fully label opt-in**: The SSH debug breakpoint that opens on a failed end-to-end run used to open automatically for pull requests authored by any project maintainer, even when nobody intended to use it, in addition to opening for pull requests carrying an explicit label ([**@lexfrei**](https://github.com/lexfrei) in #3926).
* **test(e2e): stop blocking merges on the tenant node-join deadline**: The tenant node-join deadline still fails its suite and still collects its diagnostics, but no longer blocks a merge in the pull-request lanes; nightly and the release-tag lane stay hard ([**@lexfrei**](https://github.com/lexfrei) in #3932).
* **test(e2e): record the runner's SVM feature flags in the identity capture**: Records the runner's SVM CPU feature flags in the e2e identity capture to help diagnose virtualization differences between CI runners ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3945).
* **test(tests): cut the noise in unit and e2e job logs**: Bats unit test trace is now opt-in via COZYTEST_TRACE and quiet by default in `make unit-tests`, and the e2e previous-instance log capture now picks the most recently restarted containers first ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3973).
* **test(clickhouse): add backup API render assertions**: This adds unit tests for how the ClickHouse chart wires up the backup API that the default backup strategy calls. The Pod of the `cozy-default-altinity` strategy reaches the clickhouse-backup sidecar on the named port `ch-backup-api`. It authenticates with the credentials from the `<release>-backup-api-auth` Secret ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4011).
* **test(tests): add a render smoke check over every app chart**: I added an offline render check for application charts because helm-unittest suites can leave templates outside their `templates:` lists unchecked. It runs in `make unit-tests` with fixtures for fresh, configured and wildcard certificate states ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4018).
* **test(e2e): remove unused monitoring suites and audit BATS runners**: Removes two unused monitoring OIDC BATS suites left without a runner after the Chainsaw migration and audits the remaining BATS runners ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4019).
* **test(e2e): run the merge-gating lanes on Talos containers**: Moves the merge-gating E2E lanes onto Talos containers, fixing tenant node-join failures that were happening in 8 of the last 11 runs because tenant workers sat one virtualization level too deep on the shared runners and had their CSR signed after the readiness window closed ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4020, shipped in v1.6.4).
* **ci(workflows): build Talos only when its sources changed**: CI builds the Talos image only when its sources changed, shortening the pull-request pipeline ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4021).
* **feat(platform, tests): select E2E packages and declare controller dependencies**: E2E installs only the packages the selected suites need. MongoDB now depends on its operator, and SeaweedFS on the object storage controller ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4022).
* **test(e2e): stop bringing up the monitoring instance nothing looks at**: This PR turns off monitoring instance of root tenant in e2e install and removes waits on it ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4023).
* **docs(agents): require the neutral Assisted-by: LLM trailer**: AI-assisted commits carry a single `Assisted-by: LLM` trailer; model names, `Claude-Session:` trailers and session links are not accepted in commits, PR descriptions or comments; message, trailer and comment hygiene rules are documented as review blockers ([**@lexfrei**](https://github.com/lexfrei) in #4041).
* **chore(labels): add triage/needs-proposal for requests that go through cozystack/community**: Add triage/needs-proposal for feature requests that need a design proposal in cozystack/community before work starts here ([**@lexfrei**](https://github.com/lexfrei) in #4042).
* **docs(release): add changelog for v1.6.3**: This PR adds the changelog for release `v1.6.3` ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in #4077).
* **docs(agents): correct the labeler's trigger list**: Corrects the agent documentation's list of events that trigger the pull-request labeler workflow ([**@themoriarti**](https://github.com/themoriarti) in #4090).
* **ci(agents): check commit trailers on every pull request**: A pull request now fails when one of its commits carries an attribution trailer whose value is not `LLM`, a second attribution trailer, a session trailer or transcript link, or a `Generated with <tool>` byline. The check reads only the commits the pull request adds; backport branches are exempt ([**@lexfrei**](https://github.com/lexfrei) in #4103).
* **test(tests): wire the gpu-operator suite into the unit sweep**: Run the gpu-operator helm-unittest suite, which the sweep had been skipping because the package declared no `test:` target ([**@lexfrei**](https://github.com/lexfrei) in #4110).
* **fix(ci): map the recurring app scopes to their area labels**: More of the app scopes used in PR titles now map to an `area/*` label instead of falling back to `area/uncategorized` ([**@lexfrei**](https://github.com/lexfrei) in #4121).
* **chore(codeowners): rebuild ownership by area, five owners minimum**: Code ownership is grouped by area instead of by package and every rule names at least five owners, so no path requires a review from one of two people ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4129).
* **chore(ci): add area/openbao to the label set and the scope mapping**: Adds the `area/openbao` label and its scope mapping so OpenBao pull requests are no longer labeled `area/uncategorized` ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4182).
* **test(console): add coverage for enrichSchemaWithEnums**: Backup-utils.ts (in the in-tree console dashboard) had no test file at all, despite sibling files in the same directory (dynamic-options.test.ts, crd-option-sources.test.ts, immutable-paths.test.ts) establishing that convention ([**@magic-peach**](https://github.com/magic-peach) in #4186).
* **fix(tests): reject unreadable RD preset schemas**: Fail the RD preset check when a schema cannot be read, with the schema path and jq diagnostic ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4191).
* **fix(hack): avoid a SIGPIPE false-negative in the resourcesPreset check**: `hack/check-rd-presets.sh` intermittently fails the "Unit & controller tests" CI job with a false `resourcesPreset enum missing: <preset>` for a preset that is actually present — most visibly on `http-cache-rd`, and on PRs that have nothing to do with that package ([**@scooby87**](https://github.com/scooby87) in #4204).
* **fix(ci): keep the controller Go tests running when a unit test fails first**: The `Unit & controller tests` PR check no longer silently skips the Go controller test suite when a Helm-unittest or BATS unit test fails first; both suites now always run and report ([**@pujitha24**](https://github.com/pujitha24) in #4233).
* **fix(e2e): keep backup preflight failure evidence across attempts**: A backup access preflight that times out now prints the port-forward log and the outcome of every attempt, not just the tail of the last one ([**@lexfrei**](https://github.com/lexfrei) in #4242).
* **fix(e2e): wait for the redis replica link instead of sampling it once**: The redis TLS suite waits for the replica link to come up instead of reading it once, so a link that is still establishing no longer fails the run ([**@lexfrei**](https://github.com/lexfrei) in #4252).
* **ci(e2e): surface overlay drift and remove both causes of it**: Six commits against one script: one reports a silent degradation, three remove causes of it, two repair the tests and the reasons recorded beside them ([**@IvanHunters**](https://github.com/IvanHunters) in #4261).
* **test(ci): isolate the commit-trailer fixture from global tag signing**: `make_repo` in `hack/check-commit-trailers.bats` silences `commit.gpgsign` in the fixture repository but leaves `tag.gpgsign` alone. Anyone who signs tags has that setting globally, it reaches the fixture, `git tag start` becomes an annotated tag, and git refuses it for want of a message ([**@IvanHunters**](https://github.com/IvanHunters) in #4271).
* **docs(e2e): correct the Chainsaw Test-ordering claim in etcd and clickhouse suites**: `hack/e2e-chainsaw/etcd/chainsaw-test.yaml` explained its layout with a comment claiming Chainsaw executes Tests in **name order, not file order**, and used that claim to justify why teardown lives inside each scenario. The clickhouse suite had picked up the same claim ([**@scooby87**](https://github.com/scooby87) in #4277).
* **fix(ci): fail the trailer check on an empty commit range**: Local runs of `hack/check-commit-trailers.sh` now fail on an empty commit range instead of reporting OK, so the documented pre-push check no longer passes silently when it examined nothing ([**@scooby87**](https://github.com/scooby87) in #4278).
* **test(opensearch): separate the mount-plan rewrite from the roll onto it**: Makes the failing OpenSearch e2e step say which of its two candidate causes actually happened, instead of changing its behavior ([**@scooby87**](https://github.com/scooby87) in #4279).
* **fix(e2e): cap the bucket S3 reads so the verify step honors its timeout**: Caps the bucket S3 reads in the e2e verify step so it honors its declared timeout ([**@scooby87**](https://github.com/scooby87) in #4281).
* **fix(hack): make bats unit tests pass on a BSD/macOS checkout**: Make the bats unit-test runner parse non-ASCII `@test` titles under BSD awk, and route the in-place edits in `nightly-mirror.sh` and `promote-rewrite-tags.sh` through a temp file so they work under BSD sed — closing two of the host-tool gaps that make `make unit-tests` red on a clean macOS checkout ([**@scooby87**](https://github.com/scooby87) in #4284).
* **build(images): run the tests of the packages the patches touch**: Twelve first-party images that are built from an upstream tree with patches on top now run `go test` on the packages those patches touch. The step sits in the builder stage, right after the patches are applied ([**@lexfrei**](https://github.com/lexfrei) in #4326).
* **test(vpn): add default render helm unit tests**: Add Helm unit test coverage for the default render ([**@magic-peach**](https://github.com/magic-peach) in #4347).
* **fix(ci): teach the overlay to repair an unstamped images/*.tag ref**: `hack/overlay-main-images.sh` walks `images/*.tag`, but every branch of its line classifier needs either a YAML key or an `@sha256:`. A `.tag` file holds one bare reference and nothing else, so until a build stamps it the line has neither, the drift guard reads it as a non-ref change, and the committed reference survives ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4362).
* **docs(hack): drop a false universal from the nats-rd secrets check**: Remove a false claim about -rd package tests from the nats-rd secrets check comment ([**@lexfrei**](https://github.com/lexfrei) in #4390).
* **fix(e2e): collect the CDI controller log and CDI pods in cozyreport**: The e2e diagnostics report now includes the CDI controller log and CDI pods in every phase ([**@lexfrei**](https://github.com/lexfrei) in #4414).
* **ci(workflows): pin flux version across CI toolchain steps**: Pin the flux CLI version across CI toolchain steps and the e2e-sandbox image so the flux CLI is fixed and reproducible instead of following the installer's floating latest ([**@IvanHunters**](https://github.com/IvanHunters) in #4442).
* **chore: clean up stray console file and test fixtures**: Remove a stray kubeconfig from the console sources and use synthetic values in vSphere migration test fixtures ([**@lexfrei**](https://github.com/lexfrei) in #4455).
* **docs(e2e): document jq requirement for bucket suite**: Document jq as a host prerequisite for the bucket Chainsaw suite ([**@nightcityblade**](https://github.com/nightcityblade) in #4457).
* **fix(ci): overlay the right line's refs for a PR on a feature branch**: E2E for a PR stacked on a feature branch overlays the image refs of the line the stack grew from, instead of falling back to the committed release refs ([**@lexfrei**](https://github.com/lexfrei) in #4460).
* **chore: drop private cluster and host names from the tree**: I remove internal cluster names, a real hostname with its IP addresses, internal mirror hostnames and an internal product name from comments, an example file, test fixtures and a dead commented-out config block. Where a stand-in is needed I use neutral test-cluster wording, RFC 2606 domains or RFC 5737 documentation addresses ([**@lexfrei**](https://github.com/lexfrei) in #4461).
* **test(e2e): wait for the opensearch handoff before the first API call**: The opensearch suite sends its first OpenSearch API call, the index create in `make-the-cluster-permanently-yellow`, as soon as the nodes StatefulSet reports one ready replica ([**@lexfrei**](https://github.com/lexfrei) in #4462).
* **test(e2e): hold the wait-budget guard to a start margin across four suites**: The wait-budget guard now requires a 60s margin over each floor and covers the mongodb, rabbitmq and clickhouse backup walkthroughs alongside mariadb ([**@lexfrei**](https://github.com/lexfrei) in #4468).
* **docs(ci): say why the overlay's ancestry tie goes to main**: Explain why the base-branch overlay sends an ancestry tie to main ([**@lexfrei**](https://github.com/lexfrei) in #4469).
* **test(e2e): let HR-Ready waits outlast the install timeout of the release they wait on**: HelmRelease readiness waits in the E2E suites and backup walkthroughs now outlast the install timeout of the release they wait on, so a failed install reports its own reason instead of a bare timeout ([**@lexfrei**](https://github.com/lexfrei) in #4471).
* **test(e2e): fail the suite whose pods outlive its cleanup**: The E2E app suite now fails the suite whose pods outlive its own cleanup by more than 300s, naming each leftover pod and its owner; suites with a filed leak are listed in hack/e2e-chainsaw/_lib/leak-allowlist.txt and only warn ([**@lexfrei**](https://github.com/lexfrei) in #4492).
* **ci(pre-commit): detect schema drift across all packages**: Extends the `run-make-generate` pre-commit hook to discover package Makefiles with an actual `generate` target across every package category ([**@opsmon**](https://github.com/opsmon) in #4497).
* **ci(release): build pre-release images for amd64 and arm64 and gate releases on it**: Pre-release builds publish every image as an amd64 and arm64 multi-arch index, and a release fails if any image it ships or pulls in e2e lacks either architecture. Stable releases inherit the indexes through promotion ([**@lexfrei**](https://github.com/lexfrei) in #4505).
* **fix(ci): let the fork e2e lane check out the pull request again**: Run the fork e2e lane again after the actions/checkout v4.4.0 update ([**@lexfrei**](https://github.com/lexfrei) in #4514).
* **docs(release): add changelog for v1.6.4**: This PR adds the changelog for release `v1.6.4` ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in #4534).
* **test(vminstance): dry-run evict the VM launcher pod**: The e2e suite now dry-run evicts the VM launcher pod and checks the KubeVirt eviction decision against the VMI's LiveMigratable condition ([**@lexfrei**](https://github.com/lexfrei) in #4585).
* **test(vminstance): cover VMDisk clones from an image and from a disk**: E2e now covers VMDisk clones from a golden image (source.image) and from another disk (source.disk) ([**@lexfrei**](https://github.com/lexfrei) in #4586).
* **fix(e2e): bound post-install-prep calls and make its failures name their cause**: Bound every call in the post-install-prep step and make its failures name their cause, so a hung call fails the install instead of holding it until the job timeout ([**@lexfrei**](https://github.com/lexfrei) in #4632).
* **fix(ci): run every bats unit file after a failure and cover build-*.sh in the cross-compile check**: `make bats-unit-tests` runs every bats file after a failing one and still fails at the end, and the image cross-compile check now covers build scripts such as `build-go.sh` ([**@lexfrei**](https://github.com/lexfrei) in #4633).
* **fix(ci): close e2e suite selection gaps in select-e2e and select-install**: E2e test selection no longer drops an escalation for a non-suite path under hack/e2e-chainsaw/, selects the suites that use seaweedfs instead of the full run, sees suites written as chainsaw-test.yml, and escalates edits to examples/backups/_lib/ ([**@lexfrei**](https://github.com/lexfrei) in #4634).
* **fix(e2e): bound the dataplane capture collector by wall clock**: Bound the failure-path data-plane collector by a wall-clock budget, keep its per-pod and per-LoadBalancer captures from overwriting each other, and capture the cilium service map for ClusterIP failures ([**@lexfrei**](https://github.com/lexfrei) in #4640).
* **test(e2e): keep gateway and redis-tls op ceilings above their own waits**: The gateway listener-set and redis-tls projection steps now fail on their own errors instead of being killed by the op timeout, and redis-tls reports the real cause of a failed read or exec ([**@lexfrei**](https://github.com/lexfrei) in #4643).
* **ci(workflows): pin flux, yq and helm-unittest installs**: Install flux through the commit-pinned upstream setup action instead of running fluxcd.io/install.sh as root, pin yq with checksum verification in the nightly and fork e2e workflows, and pin the helm-unittest plugin version ([**@lexfrei**](https://github.com/lexfrei) in #4645).
* **fix(e2e): hold replicated-lane LINSTOR cleanup to a pre-suite baseline and delete kafka suite PVCs**: The kubernetes suites' LINSTOR cleanup compares each node's ZFS datasets against a pre-suite baseline instead of cached free capacity, the replicated lane checks its 90 GiB floor before the suite starts, and the kafka suite deletes the PVCs its clusters keep ([**@lexfrei**](https://github.com/lexfrei) in #4651).
* **test(e2e): drop readiness asserts that HelmRelease Ready already covers**: The qdrant, external-dns and vminstance suites no longer repeat readiness checks that the HelmRelease Ready assert already covers ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4652).
* **test(e2e): record ZFS datasets and LINSTOR volumes in cozyreport**: Cozyreport records the ZFS datasets of each node's LINSTOR pools in exact bytes, and the LINSTOR volume list, on every e2e run ([**@lexfrei**](https://github.com/lexfrei) in #4657).
* **fix(ci): close three blind spots in the e2e test selectors**: The e2e test selectors now see Chainsaw suites made of numbered step files or nested below a suite directory, and run the full suite for an edit to a file placed directly in examples/backups/ ([**@lexfrei**](https://github.com/lexfrei) in #4658).
* **ci(build): fall back to direct, retry and cache Go module downloads**: Go module downloads in image builds and CI jobs now fall back to direct fetching and retry on transient proxy.golang.org errors. The unit test job reuses a module cache written from main ([**@lexfrei**](https://github.com/lexfrei) in #4660).
* **test(tests): make absence-only helm-unittest cases fail when they check nothing**: Twenty-one helm-unittest cases asserted only that something is absent from a document, and could pass without ever looking at the document they name. This PR makes each of them fail when that happens ([**@lexfrei**](https://github.com/lexfrei) in #4664).
* **fix(ci): harden the PR labeler's title parsing and map recurring scopes**: The PR labeler no longer crashes on titles scoped like Object.prototype members, handles nested backport prefixes, and maps more recurring scopes, including a new area/dx ([**@lexfrei**](https://github.com/lexfrei) in #4666).
* **test(e2e): cover monitoring OIDC modes with a Chainsaw suite**: Monitoring OIDC gets live-cluster e2e coverage again. It had none since the two pre-Chainsaw BATS suites were removed without a port. This adds a Chainsaw suite in `hack/e2e-chainsaw/monitoring/` that ports what they asserted, plus the in-place switch from an inline config to a `secretRef` that nothing covered ([**@lexfrei**](https://github.com/lexfrei) in #4678).
* **test(e2e): read the opensearch release history through the pinned name**: The opensearch suite reads HelmRelease history through the shared jsonpath, which is now pinned against the upstream Flux type for every Chainsaw suite that runs the remediation guard ([**@lexfrei**](https://github.com/lexfrei) in #4689).
* **ci(dashboard): run eslint on the console and clear its errors**: The console's eslint now runs in the UI test job, and the console passes it with no errors ([**@lexfrei**](https://github.com/lexfrei) in #4692).
* **test(e2e): wait for the S3 gateway to accept a key before using it**: The bucket suite waits for the S3 gateway to accept a new key before writing with it, instead of trusting BucketAccess readiness ([**@lexfrei**](https://github.com/lexfrei) in #4694).
* **fix(release): collect refs stored as registry/image/tag/digest keys**: The image-ref enumeration used by promotion, the nightly mirror and the multi-arch gate now reads refs stored as separate registry/image/tag/digest keys, the layout of the ingress-nginx controller and default backend ([**@lexfrei**](https://github.com/lexfrei) in #4697).
* **test(qdrant): exercise the tenant trust anchor end to end**: E2e now runs qdrant with TLS on and checks that a client trusting only the tenant-published CA reaches it, while a wrong or missing anchor and plaintext HTTP are refused ([**@lexfrei**](https://github.com/lexfrei) in #4701).
* **test(seaweedfs): lower the volume size limit on a live instance and test the chart hand-off**: Add a seaweedfs suite that lowers the volume size limit on a live instance and checks that the bucket still takes writes, and a unit test for the hand-off between the SeaweedFS app and data-plane charts ([**@lexfrei**](https://github.com/lexfrei) in #4707).
* **ci(e2e): add a daily isp-hosted install lane on kind**: Add a daily lane that installs the isp-hosted platform variant on kind ([**@lexfrei**](https://github.com/lexfrei) in #4712).
* **test(e2e): cover the computeplane module end to end**: Add an end-to-end suite for the ComputePlane tenant module ([**@lexfrei**](https://github.com/lexfrei) in #4713).
* **test(controller): seed the DataVolume informer for controller-runtime 0.25**: Seeds the DataVolume informer in the controller tests for controller-runtime 0.25, restoring the red `Unit & controller tests` job ([**@lexfrei**](https://github.com/lexfrei) in #4729).
* **fix(ci): reconcile three unit-test contracts broken by adjacent merges**: Fix three unit-test contracts that went red on main after adjacent merges ([**@lexfrei**](https://github.com/lexfrei) in #4751).
* **ci(e2e): refuse a new fork image before the build starts**: A fork PR that adds a new first-party image now fails in the plan job, naming the image, instead of after the whole build at the image push ([**@lexfrei**](https://github.com/lexfrei) in #4767).
* **test(tests): stop unit tests from reaching a cluster**: Unit and controller test targets run with no kubeconfig and no in-cluster service host, so a unit test can no longer reach a cluster ([**@lexfrei**](https://github.com/lexfrei) in #4804).
* **test(hack): load the strict-mode helper in fork-new-images.bats**: Fork-new-images.bats loads the strict-mode helper, so the strict-mode audit in the bats unit lane passes again ([**@lexfrei**](https://github.com/lexfrei) in #4805).
* **ci(e2e): never tear down the Proxmox tenant on the runner's default context**: The Proxmox e2e teardown always uses the staged kubeconfig, so it can no longer delete from the runner's default cluster. It skips only when Stage did not succeed and fails when the kubeconfig breaks after Stage. A failed delete no longer skips the tenant delete ([**@lexfrei**](https://github.com/lexfrei) in #4807).
* **fix(hack): keep root monitoring redirects off in the E2E install**: The E2E install keeps root monitoring redirects off, so runs that select only a few suites no longer fail on a missing cozy-monitoring namespace ([**@lexfrei**](https://github.com/lexfrei) in #4811).
* **test(monitoring): stop the alert severity guard from missing rule sources**: The alert severity check now also catches rule files with a quoted or commented kind, and fails on a rule group it cannot see into instead of passing it unchecked ([**@yankawai**](https://github.com/yankawai) in #4812).
* **ci(retention): sweep packages named by the published nightly tree**: The GHCR retention workflow takes package names from the published nightly tree instead of the org package listing, which rejects the app token, and fails the run whenever names, versions or deletions cannot be resolved ([**@lexfrei**](https://github.com/lexfrei) in #4826).
* **ci(e2e): run nightly and release-tag e2e on the container lane**: Nightly and release-tag e2e run on Talos containers instead of QEMU guests. DRBD, the replicated StorageClass and the Cozystack Talos node image are no longer covered by any e2e lane ([**@lexfrei**](https://github.com/lexfrei) in #4828, shipped in v1.6.5).
## Documentation
* **docs: add public ROADMAP.md (2026-2028)**: This PR adds a public `ROADMAP.md` covering the project's direction over the two-year window May 2026 – May 2028, aligned with the CNCF Sandbox → Incubating → Graduated trajectory ([**@tym83**](https://github.com/tym83) in #2666).
* **docs(governance): document the Reviewer role and align the maintainer roster**: Makes the documented roster match who actually holds write access, which CNCF Incubation checks directly through the API ("Code and Doc ownership in GitHub matches documented governance roles", Required) ([**@kvaps**](https://github.com/kvaps) in #3462).
* **docs(postgres): name the trust anchor as the tenant actually sees it**: The tenant CA trust anchor is `postgres-<name>.tenant-ca`, not `<release>.tenant-ca` ([**@lexfrei**](https://github.com/lexfrei) in #3463).
* **docs(monitoring): correct the release name the OIDC comment, guide and test fixture encode**: Correct the release name (`monitoring-system`) and platform namespace (`tenant-root`) encoded in the Grafana OIDC helper comment, the OIDC guide, and the chart test fixture ([**@youdie006**](https://github.com/youdie006) in #3468).
* **docs(platform): explain system component memory limits**: Document why system components carry memory limits, how the Talos userspace OOM handler selects victims, how to tune or disable the system-namespace LimitRange, and why PriorityClass does not influence any of it ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #3496).
* **docs: add CNCF General Technical Review (GTR) questionnaire**: Add CNCF General Technical Review (GTR) questionnaire for the Sandbox → Incubation review ([**@tym83**](https://github.com/tym83) in #3572).
* **docs(observability): add a distributed tracing guide**: Add a distributed tracing guide covering the per-tenant collector, client-side instrumentation per managed engine, and shared-central tracing in tenant-root ([**@scooby87**](https://github.com/scooby87) in #3772).
* **docs(governance): describe sub-projects and third-party dependencies**: Brings `GOVERNANCE.md` in line with the organisation it describes: what the project governs, and what it ships but does not ([**@kvaps**](https://github.com/kvaps) in #3784).
* **docs(etcd): note that affinity is not applied to existing members**: Documents that the etcd `affinity` value only applies to member Pods the operator creates or replaces and is never used to reschedule running members ([**@scooby87**](https://github.com/scooby87) in #3964).
* **docs(gpu): drop two stale references in the vGPU page**: Removes two stale references from the vGPU documentation page ([**@lexfrei**](https://github.com/lexfrei) in #4096).
* **docs(rabbitmq): say that deleting the application leaves the TLS Secrets behind**: `packages/apps/rabbitmq/README.md` documented the key material that survives turning TLS off, but not the same outcome on deleting the application ([**@lexfrei**](https://github.com/lexfrei) in #4143).
* **docs(gtr): finalize editorial markers and pin submission version**: Finalize editorial GTR answers, correct tenant-isolation and alpha/beta wording, pin submission version to v1.6.3, assign Security Champion ([**@tym83**](https://github.com/tym83) in #4221).
* **docs(readme): stop the logo filling the page and thin the badge wall**: The README opens with the logo rendered at **734×246** on a desktop viewport — a wall of logo before a single word of text — followed by eight badges of ragged widths ([**@tym83**](https://github.com/tym83) in #4222).
* **docs(security): document Security Champion rotation and response team**: Document the Security Champion rotation policy and security response team in SECURITY.md ([**@tym83**](https://github.com/tym83) in #4223).
* **docs(governance): remove the unused Directors role**: Remove the unused Directors role from GOVERNANCE.md ([**@tym83**](https://github.com/tym83) in #4296).
* **docs(kubevirt): name every payload line the tail checks do not cover**: The sentence that PR left in `packages/system/kubevirt/Makefile` says the device and CPU payload lines stay outside the marker checks, which reads as if the rest are handled. The `- {{ . }}` entries inside the `extraFeatureGates` and `disabledFeatureGates` ranges are neither device nor CPU and are not covered either ([**@yankawai**](https://github.com/yankawai) in #4340).
* **docs(api): stop listing helm-install-timeout carriers in rest.go**: Drop a stale list of helm-install-timeout carriers from a code comment ([**@lexfrei**](https://github.com/lexfrei) in #4381).
* **docs(api): separate the healthCheckExprs failure modes**: Describe each healthCheckExprs failure mode separately in the ApplicationDefinition and PackageSource field docs ([**@lexfrei**](https://github.com/lexfrei) in #4387).
* **docs(apps): correct the shared resources and preset paragraphs**: Corrects two sentences of the boilerplate shared by the application READMEs about resource presets ([**@yankawai**](https://github.com/yankawai) in #4394).
* **docs(governance): nominate @themoriarti as Reviewer for the Proxmox integration**: Add @themoriarti as Reviewer for the Proxmox integration ([**@tym83**](https://github.com/tym83) in #4439).
* **docs(kubevirt): state the preconditions of the eviction comments**: The KubeVirt CR comments now say that a VMI created before the evictionStrategy change keeps LiveMigrate until its VM restarts, and that the MachineHealthCheck replaces a non-migratable kubernetes-nodes worker when the workload updater evicts it ([**@lexfrei**](https://github.com/lexfrei) in #4441).
* **docs(api): clarify kind-wide Helm timeout overrides**: Clarifies that Helm install and upgrade timeout annotations are configured on an ApplicationDefinition and apply to every Application of that kind ([**@opsmon**](https://github.com/opsmon) in #4493).
* **docs(virtual-machine): correct persistent EFI migration documentation**: Removes the outdated warning that persistent EFI state on default RWO storage makes a virtual machine node-pinned and prevents live migration. The warning no longer reflects the supported migration behavior and could discourage users from persisting Secure Boot keys ([**@opsmon**](https://github.com/opsmon) in #4494).
* **docs(kubernetes): clarify node pool Helm timeout semantics**: Corrects the explanation for the Kubernetes node-pool Helm timeout ([**@opsmon**](https://github.com/opsmon) in #4496).
* **docs(security): add an OpenSSF Security Insights manifest**: Publish an OpenSSF Security Insights manifest (security-insights.yml) describing the project's vulnerability reporting, security tooling and dependency policy ([**@themoriarti**](https://github.com/themoriarti) in #4757).
* **docs: link the documentation of the current release**: Link the v1.6 documentation from the README and GOVERNANCE.md instead of the v0 pages the unversioned cozystack.io paths redirect to, and describe the kubernetes-distribution use case as Build Your Own Platform, which is what that page documents in v1 ([**@themoriarti**](https://github.com/themoriarti) in #4761).
### Website (cozystack/website)
* **[website] docs(kubernetes): add OIDC authentication guide**: Adds a guide for configuring OIDC authentication on managed Kubernetes clusters ([**@IvanHunters**](https://github.com/IvanHunters) in cozystack/website#596).
* **[website] docs(monitoring): add OIDC authentication guide for Grafana**: Adds a guide for configuring OIDC single sign-on authentication for per-instance Grafana, complementing v1.6.0's tenant OIDC support ([**@IvanHunters**](https://github.com/IvanHunters) in cozystack/website#597).
* **[website] fix(docs): refresh trunk version pins to Cozystack v1.6.3**: Updates the documentation's trunk version references to Cozystack v1.6.3 ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#618).
* **[website] feat(i18n): automated translation pipeline with machine review gate**: Adds an automated pipeline that discovers changed English pages, translates them with a machine review gate, and opens a weekly pull request that a maintainer must merge before anything reaches production ([**@tym83**](https://github.com/tym83) in cozystack/website#623).
* **[website] feat(blog): how LINSTOR and DRBD keep your data alive when a server dies**: New blog post about replicated block storage in Cozystack: LINSTOR + DRBD ([**@tym83**](https://github.com/tym83) in cozystack/website#626).
* **[website] [docs] Update managed apps reference for v1.6.0**: Refreshes the managed applications reference pages to match what shipped in v1.6.0 ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#628).
* **[website] docs(backups): document PostgreSQL point-in-time recovery (PITR)**: Adds documentation for PostgreSQL point-in-time recovery, covering how to configure and perform a PITR restore for managed PostgreSQL clusters ([**@androndo**](https://github.com/androndo) in cozystack/website#629).
* **[website] feat(i18n): deterministic integrity/typography checks and real per-language style guides**: Quality improvements to the translation pipeline. The glossary, style-guide, back-translation, two-reviewer and digest design is kept and extended ([**@tym83**](https://github.com/tym83) in cozystack/website#630).
* **[website] feat(hack): generate docs from a pre-tag ref and fail loudly on fetch errors**: Hardens the docs-generation tooling to build from a pinned pre-tag ref and to fail loudly instead of silently on a fetch error, reducing the chance of publishing docs generated from the wrong ref ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#631).
* **[website] chore(codeowners): add the site owners**: Adds <@tym83> and <@myasnikovdaniil> to the owners of this repository, so review reaches the people carrying the site day to day. @kvaps and @lllamnyp stay on the list — this is an addition, not a replacement ([**@kvaps**](https://github.com/kvaps) in cozystack/website#632).
* **[website] docs(talm): describe the preset value knobs for network and registries**: Documents the new `talm` preset value knobs for time servers, control-plane extra args, registry mirrors and TLS, VIPs, and network links, so operators can find the declarative equivalents to a template fork ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#633).
* **[website] [oss-health] Update monthly OSS health snapshot**: Updates the monthly OSS health snapshot published on the site ([**@app/github-actions**](https://github.com/apps/github-actions) in cozystack/website#634).
* **[website] chore(blog): make the two taxonomy axes disjoint**: Cleans up the blog's tagging so its two taxonomy axes no longer overlap, making blog post categorization and filtering more consistent ([**@tym83**](https://github.com/tym83) in cozystack/website#635).
* **[website] chore: upgrade Hugo to 0.164.0 and convert HTML content to markdown**: Upgrades the site generator to Hugo 0.164.0, requiring Node 22 under its new node permission model, and converts remaining raw-HTML content to markdown along the way, keeping the site buildable on current tooling ([**@tym83**](https://github.com/tym83) in cozystack/website#636).
* **[website] feat(community): add a Community page and link it from the main menu**: Adds a Community page linked from the site's main menu, giving visitors a single place to find how to get in touch with and contribute to the Cozystack community ([**@tym83**](https://github.com/tym83) in cozystack/website#637).
* **[website] feat(blog): Cozystack 1.6 release and Blockstor announcement**: Publishes the v1.6.0 release-announcement blog post — covering Talos Linux tenant workers, tenant-controlled OIDC, the `SecurityGroup` API, hierarchical quotas, and in-place etcd-operator adoption — alongside a companion post announcing the open-sourcing of Blockstor, the LINSTOR-compatible storage control plane ([**@tym83**](https://github.com/tym83) in cozystack/website#641).
* **[website] chore(blog): repair front matter, links and bundle names**: Brings every blog post in line with the publishing checks: 83 posts now pass with zero errors ([**@tym83**](https://github.com/tym83) in cozystack/website#642).
* **[website] chore(telemetry): publish July 2026 and explain how the figures are derived**: Publishes the July 2026 telemetry figures and documents how those figures are derived, giving the community visibility into adoption trends and how the numbers are calculated ([**@tym83**](https://github.com/tym83) in cozystack/website#644).
* **[website] ci(telemetry): put the monthly refresh back on a schedule**: Restores the scheduled monthly refresh of the telemetry dashboard data ([**@tym83**](https://github.com/tym83) in cozystack/website#645).
* **[website] feat(blog): new Blockstor banner**: Adds a new banner promoting Blockstor to the blog, improving the visibility of the storage control plane's announcement ([**@tym83**](https://github.com/tym83) in cozystack/website#646).
* **[website] docs: import the operator guides that lived in the cozystack repo**: Moves the operator-facing guides that used to live in the `cozystack` repo over to the documentation site, so operators find them alongside the rest of the docs instead of scattered across two repositories ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#648).
* **[website] docs(licenses): list kuberture and ouroboros**: Adds `kuberture` and `ouroboros` — both shipped with the platform and maintained by a Cozystack maintainer — to the Licenses page, which otherwise only enumerates upstream components ([**@kvaps**](https://github.com/kvaps) in cozystack/website#649).
* **[website] docs(oidc): document private CA and staging trust**: Documents how to configure tenant OIDC to trust a private certificate authority and staging certificates, closing a gap for operators running their own CA or testing with a staging issuer ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#650).
* **[website] fix(telemetry): document the opt-out for both collectors**: Documents both opt-out steps now that the fix above threads `telemetry.disabled` through the platform, replacing the previously incomplete single-step procedure ([**@kvaps**](https://github.com/kvaps) in cozystack/website#651).
* **[website] docs(networking): GatewayClass selection, edge TLS mode, and the Cloudflare Tunnel package**: Documents GatewayClass selection, the edge TLS certificate mode, and the Cloudflare Tunnel Gateway API package ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#655).
* **[website] docs(compliance): add a Compliance section with a PCI DSS page**: Adds a `Compliance` section to the main menu and its first page: a requirement-by-requirement account of what Cozystack provides towards PCI DSS ([**@tym83**](https://github.com/tym83) in cozystack/website#656).
* **[website] docs(compliance): add CIS Benchmark, GDPR and DORA pages**: Three more frameworks under `Compliance`, written the same way as the PCI DSS page: measured on a live cluster where measurable, honest about the boundary between what the platform supplies and what the organization holds ([**@tym83**](https://github.com/tym83) in cozystack/website#657).
* **[website] docs(compliance): add Kubernetes conformance results page**: Adds a Kubernetes conformance page documenting the platform's CNCF conformance runs, later extended with the v1.31 result and corrected submission scope ([**@tym83**](https://github.com/tym83) in cozystack/website#658).
* **[website] fix(blog): restore Community Meeting Calendar link in Wizard post**: Fixes an empty link in the Wizard announcement blog post ([**@tym83**](https://github.com/tym83) in cozystack/website#659).
* **[website] docs(compliance): add v1.31 conformance result, correct submitted versions**: Adds the Kubernetes v1.31 conformance result and corrects the submitted versions listed on the conformance page ([**@tym83**](https://github.com/tym83) in cozystack/website#660).
* **[website] docs(faq): drop the retired GitOps example**: Replaces a FAQ pointer to an archived, pre-1.0 GitOps example repository with a direct answer explaining that Cozystack applications are ordinary Kubernetes objects, so no platform-specific GitOps tooling is needed ([**@kvaps**](https://github.com/kvaps) in cozystack/website#661).
* **[website] docs(home): refresh the screenshot gallery for the reworked console**: Retakes the home-page screenshot gallery against the reworked console (Marketplace, Modules, Tenants, Admin), replacing slides that showed a header and sidebar layout that no longer exist ([**@kvaps**](https://github.com/kvaps) in cozystack/website#662).
* **[website] docs(images): refresh the product screenshot and its dark variant**: Retakes the standalone product screenshot referenced by the announcement post and outside links, which still showed a console two redesigns old ([**@kvaps**](https://github.com/kvaps) in cozystack/website#663).
* **[website] docs(virtualization): guide for building a Windows golden image with Packer**: Adds a guide for building a Windows golden image with Packer on KubeVirt, covering the CDI clone flow and the published HashiCorp KubeVirt Packer plugin ([**@scooby87**](https://github.com/scooby87) in cozystack/website#666).
* **[website] fix(page): render page title and fix top spacing on type:page pages**: Fixes page titles not rendering and content sitting flush against the navbar on `type: page` pages, most visible on the Compliance section ([**@tym83**](https://github.com/tym83) in cozystack/website#668).
* **[website] feat(site): Live Demo — the real console, mocked, in the browser**: Adds a `/demo` page running the real `cozystack-ui` console with a mocked Kubernetes API, so a visitor can explore the actual marketplace and console without a cluster, signup, or install ([**@tym83**](https://github.com/tym83) in cozystack/website#669).
* **[website] feat(demo): self-refreshing console demo + Resources nav grouping**: Adds the demo's source as an overlay with a weekly-refresh workflow so the mocked bundle tracks upstream `cozystack-ui` automatically, and groups secondary navigation links under a Resources menu ([**@tym83**](https://github.com/tym83) in cozystack/website#670).
* **[website] docs(marketplace): add Application Marketplace section**: Adds an Application Marketplace section to the next docs covering the PackageSource-based marketplace and the `cozypkg` CLI ([**@IvanHunters**](https://github.com/IvanHunters) in cozystack/website#672).
* **[website] docs(virtualization): add a VMware vSphere migration guide**: Adds a guide for migrating VMware vSphere VMs into Cozystack, including the VDDK image build ([**@kvaps**](https://github.com/kvaps) in cozystack/website#673).
* **[website] fix(demo): embed console under site nav, fix refresh + deep create**: Fixes the live demo breaking on a hard refresh of any deep route by serving the SPA at a separate path and embedding it in an iframe under a real Hugo page, and keeps the site's own navigation visible above the demo ([**@tym83**](https://github.com/tym83) in cozystack/website#676).
* **[website] fix(demo): create works for every catalog type**: Fixes demo creation hanging on an endless spinner for most marketplace types (Kubernetes cluster, FoundationDB, Harbor, MariaDB, and others) by resolving the mocked kind from the application catalog instead of a hardcoded list of seven types ([**@tym83**](https://github.com/tym83) in cozystack/website#677).
* **[website] docs(platform-package): document kubevirt.disabledFeatureGates**: Documents the `kubevirt.disabledFeatureGates` platform value ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#678).
* **[website] docs(platform-package): sync the publishing.* table with the platform values**: Keeps the platform values reference's `publishing.*` table in sync with the actual platform values ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#679).
* **[website] [docs] Update managed apps reference for v1.6.3**: Refreshes the managed applications reference for the v1.6.3 release ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in cozystack/website#681).
* **[website] fix(networking): correct what disabledPackages does to ouroboros**: Corrects the documented effect of `disabledPackages` on the `ouroboros` package ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#683).
* **[website] fix(operations): replace the component removal recipe that undoes itself**: `kubectl delete hr -n <namespace> <component>` is documented as the way to remove an installed component. It removes nothing, and it is not harmless ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#684).
* **[website] fix(site): replace vendor social accounts in the footer with project channels**: Replaces vendor-owned social accounts in the site footer with the project's own channels ([**@tym83**](https://github.com/tym83) in cozystack/website#690).
* **[website] feat(site): verify the @cozystack.io Bluesky handle over HTTPS**: Serves the Bluesky handle DID at `/.well-known/atproto-did` to verify the `@cozystack.io` Bluesky handle over HTTPS ([**@kvaps**](https://github.com/kvaps) in cozystack/website#691).
* **[website] fix(site): give the live-demo band its own layout**: Gives the live-demo band on the home page its own layout instead of sharing one with unrelated content ([**@tym83**](https://github.com/tym83) in cozystack/website#692).
* **[website] docs(compliance): add AI Conformance page, update Kubernetes Conformance**: Adds the initial AI Conformance page and updates the Kubernetes Conformance page ([**@tym83**](https://github.com/tym83) in cozystack/website#693).
* **[website] feat(home): add a "where Cozystack fits" section**: Adds a "where Cozystack fits" section to the home page ([**@tym83**](https://github.com/tym83) in cozystack/website#694).
* **[website] fix(site): resolve /roadmap and move the vendor logo off a project path**: Fixes the `/roadmap` link and moves a vendor logo off a project-owned path ([**@tym83**](https://github.com/tym83) in cozystack/website#695).
* **[website] docs(blog): route independent-adopter replies to the project**: Redirects independent-adopter reply channels on the blog to the project instead of an individual ([**@tym83**](https://github.com/tym83) in cozystack/website#696).
* **[website] docs(platform-package): document KubeVirt migration settings**: Documents the new `kubevirt.migrations` platform value from the corresponding fix above ([**@yankawai**](https://github.com/yankawai) in cozystack/website#697).
* **[website] chore(demo): refresh the console demo from the cozystack monorepo**: Refreshes the live demo's bundled `cozystack-ui` console from the monorepo ([**@kvaps**](https://github.com/kvaps) in cozystack/website#698).
* **[website] fix(site): point the Certified Kubernetes badge at the live landscape entry**: Fixes the Certified Kubernetes badge linking to a stale landscape entry ([**@tym83**](https://github.com/tym83) in cozystack/website#699).
* **[website] docs(marketplace): point the Repositories view to Admin**: The connect guide only said to open the view "from the sidebar", and the page sits in the Marketplace docs, so after that change readers would look in the wrong place. Only `next/` is touched, the view is not in a release yet ([**@kvaps**](https://github.com/kvaps) in cozystack/website#700).
* **[website] fix(demo): keep the screen when the demo is reloaded**: Fixes the live console demo losing its current screen on a browser reload ([**@kvaps**](https://github.com/kvaps) in cozystack/website#701).
* **[website] docs(install): note the talm version pins on the update path**: Talm v0.35.0 no longer substitutes a built-in Kubernetes version when `templateOptions.kubernetesVersion` is empty. This PR documents what that means for projects created with the cozystack preset and how to pin the version ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#703).
* **[website] docs(platform-package): document kubevirt.migrations for v1.6**: Document kubevirt.migrations, available since v1.6.4, on the v1.6 platform values reference ([**@yankawai**](https://github.com/yankawai) in cozystack/website#705).
* **[website] feat(compliance): surface AI Conformance page, mark submission accepted**: Publishes the AI Conformance results page and marks the platform's submission as accepted ([**@tym83**](https://github.com/tym83) in cozystack/website#706).
* **[website] fix(docs): correct stale hairpin-proxy uninstall claim and package names**: Two docs corrections, both checked against source ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#709).
* **[website] fix(versions): pin the trunk to published releases and fix released pin headers**: Two fixes in the pipeline behind `data/versions/*.yaml` ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#710).
* **[website] docs(development): describe the multi-arch default of make image**: States in the development guide that `make image` now builds for linux/amd64 and linux/arm64 by default ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#711).
* **[website] [docs] Update managed apps reference for v1.6.4**: Automated docs update for release `v1.6.4`, generated at PROMOTE time from the `release-1.6.4` staging branch (fetched via FETCH_REF; the stable tag `v1.6.4` does not exist yet) ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in cozystack/website#712).
* **[website] docs(gpu): select the container variant from bundle values**: Documents setting up the container GPU variant through the bundle values ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#714).
* **[website] docs(variants): document the slim platform variants**: Documents the new `isp-slim`, `isp-slim-generic` and `isp-hosted-slim` platform variants ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#718).
* **[website] docs: move the next docs to Talos v1.14.2**: Moves the next docs to Talos v1.14.2, matching the Talos image Cozystack now ships ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#720).
* **[website] docs(talm): document the ZFS export-on-shutdown knob**: The talm page gets a section on the new cozystack preset setting that stops the ZFS extension from exporting the pool on shutdown. Without it, a node with LINSTOR on ZFS hangs in `rebooting` on every reboot, upgrade and reset while DRBD holds its zvols ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#721).
* **[website] feat(blog): compliance results news and GTR page**: Short news posts for the conformance and review results, plus a Compliance page for the CNCF General Technical Review ([**@tym83**](https://github.com/tym83) in cozystack/website#722).
* **[website] blog: Oberon RISC5 machine as a Cozystack catalog app (news)**: Community news post: a pluggable Cozystack catalog that runs Niklaus Wirth's RISC5 Oberon workstation as an ordinary VM, using the `cozymarketplace` mechanism and a KubeVirt sidecar handler — with no fork of Cozystack or KubeVirt ([**@tym83**](https://github.com/tym83) in cozystack/website#723).
* **[website] docs(platform-package): add publishing.serviceDomain**: Documents the new `publishing.serviceDomain` platform value in the publishing reference table ([**@lexfrei**](https://github.com/lexfrei) in cozystack/website#724).
* **[website] fix(footer): use the LF Projects copyright notice on every page**: Makes the footer of every page carry the LF Projects notice required by the CNCF website guidelines ([**@themoriarti**](https://github.com/themoriarti) in cozystack/website#736).
* **[website] fix(llms): point llms.txt docs links at the current release**: The short `/docs/v1/...` path is not an option here: only `layouts/404.html` redirects it, in JavaScript, after an HTTP 404. So llms.txt needs full version paths ([**@themoriarti**](https://github.com/themoriarti) in cozystack/website#737).
* **[website] docs(platform): document the monitoring platform values**: Adds a Monitoring section to the next Platform Package reference documenting the `monitoring.*` platform values ([**@scooby87**](https://github.com/scooby87) in cozystack/website#738).
* **[website] docs(platform-package): note the /21 the kube-ovn variants keep for cilium**: Notes the /21 that the kube-ovn variants now reserve for Cilium's router and Gateway API ingress addresses ([**@yankawai**](https://github.com/yankawai) in cozystack/website#739).
* **[website] docs: add v1.6 to v1.7 upgrade notes**: Adds the v1.6 to v1.7 upgrade notes and brings the Gateway API, Backup Classes and Platform Package pages up to 1.7 ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#757).
* **[website] build(docs): generate reference pages for more managed apps**: The generated reference now covers Valkey, OpenSearch, Bucket, KubernetesNodes, ComputePlane and Gateway ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#756).
* **[website] docs(networking): drop HTTPCache and hold back the Cloudflare Tunnel page**: Removes the HTTPCache page and holds the Cloudflare Tunnel page back for 1.7 ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#755).
* **[website] docs: carry the operator guide edits over to next/**: Carries the operator guide edits made after the import over to the `next/` docs ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#754).
* **[website] docs(next): show flux-plunger recording the processed version**: The flux-plunger docs now show it recording the processed version before deleting the Secret ([**@scooby87**](https://github.com/scooby87) in cozystack/website#752).
* **[website] fix(monitoring): publish the Grafana OIDC guide in v1.6**: Publishes the Grafana OIDC guide in the v1.6 docs ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#647).
* **[website] docs: update managed apps reference for v1.6.5**: Refreshes the managed apps reference for v1.6.5 ([**@app/cozystack-ci**](https://github.com/apps/cozystack-ci) in cozystack/website#758).
* **[website] docs(community): drop the Russian-language Telegram channel**: Removes the Russian-language Telegram channel from the community page ([**@tym83**](https://github.com/tym83) in cozystack/website#744).
* **[website] Fix mobile overflow and broken images; host Hidora logo locally**: Fixes horizontal scrolling on phones on the home and Resources pages, wraps long links in docs and blog posts, repairs the image paths of the Google IdP guide in every doc version, and serves the Hidora logo on the support pages from the site itself ([**@kvapsova**](https://github.com/kvapsova) in cozystack/website#769).
* **[website] Redesign: monochrome theme, Nunito Sans, circuit artwork**: Restyles the site to a new design: a reworked home page in all five languages, a self-hosted Nunito Sans font with no Google Fonts request, a white navbar, a new footer, and restyled docs, blog, support, compliance and Resources pages ([**@kvapsova**](https://github.com/kvapsova) in cozystack/website#771).
## Related repositories
Changes from the repositories that ship alongside Cozystack during this release cycle: `talm` (v0.34.0, v0.35.0, v0.36.0), `cozy-proxy` (v0.4.0), `cozyhr`, `ansible-cozystack` (v1.6.0–v1.6.5) and `external-apps-example`. `boot-to-talos` had no tags or merged changes in the period. Entries note the release that carries them, or that they are merged but not yet in a release of their repository.
* **[talm] chore(deps): bump Talos to v1.14.0**: Moves talm to Talos v1.14.0 for the main module and `pkg/machinery`, and fixes what the bump broke ([**@sircthulhu**](https://github.com/sircthulhu) in cozystack/talm#223).
* **[talm] chore(deps): migrate to Helm 4 and drop the cozystack/talos fork**: Migrates talm off the end-of-life Helm 3 library to Helm 4 and drops the `cozystack/talos` fork in favor of stock upstream Talos ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#231).
* **[talm] feat(charts): add preset value knobs**: Adds preset values for time servers, control-plane component extra args, registry mirrors and TLS, multiple Layer2 VIPs and network settings, so a node's Talos config can come from `values.yaml` instead of a template fork ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#232).
* **[talm] Documentation site at talm.cozystack.io with a generated CLI reference**: Publishes the talm manual as a documentation site at talm.cozystack.io with a CLI reference generated from the command tree ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#235).
* **[talm] Unbreak the lint job under Go 1.27**: Fixes the failing lint job under Go 1.27 so CI is green again ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#237).
* **[talm] feat(upgrade): make the install image resolution visible and guarded**: `talm upgrade` now makes it visible which install image it resolved and guards against an unintended default, so an upgrade with `-f` no longer silently targets the wrong image ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#243).
* **[talm] fix(cozystack): skip ZFS pool export on shutdown by default**: Skips the ZFS pool export on shutdown by default, so a Talos node with LINSTOR on ZFS no longer hangs in `rebooting` on every reboot, upgrade and reset ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#251).
* **[talm] fix(cozystack): load drbd_transport_tcp explicitly**: The cozystack preset now loads `drbd_transport_tcp` next to `drbd`, which DRBD needs to connect to any peer on Talos v1.14 (released in talm v0.36.0) ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#252).
* **[talm] fix(charts): render the Talos v1.14 config contract**: Both presets now render the Talos v1.14 typed-document config shape and pin it by default, since from v1.14 Talos rejects a config that also sets the old v1alpha1 fields (released in talm v0.36.0) ([**@lexfrei**](https://github.com/lexfrei) in cozystack/talm#259).
* **[cozyhr] Canonicalize manifests before diffing to remove serialization noise**: `cozyhr diff` now canonicalizes manifests before comparing, removing false 'has changed' reports caused by YAML serialization differences (merged after cozyhr v1.6.1, not yet in a cozyhr release) ([**@lllamnyp**](https://github.com/lllamnyp) in cozystack/cozyhr#16).
* **[cozyhr] fix: stop treating condition messages as format strings**: Stops passing already-built condition messages as format strings, so messages containing `%` are no longer garbled (merged after cozyhr v1.6.1, not yet in a cozyhr release) ([**@magic-peach**](https://github.com/magic-peach) in cozystack/cozyhr#17).
* **[cozy-proxy] chore(codeowners): declare repository ownership**: Declares repository ownership so reviews route to the right people ([**@kvaps**](https://github.com/kvaps) in cozystack/cozy-proxy#17).
* **[cozy-proxy] proxy: scope the datapath per node, keep the egress SNAT cluster-wide**: Each node now programs the datapath maps only for services whose backends it hosts while the egress SNAT stays cluster-wide, reducing per-node state and churn (merged after cozy-proxy v0.4.0, not yet in a cozy-proxy release) ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in cozystack/cozy-proxy#18).
* **[cozy-proxy] feat(image): publish a linux/amd64 and linux/arm64 index**: Publishes the cozy-proxy image as a linux/amd64 and linux/arm64 index so the DaemonSet no longer crashes on arm64 nodes ([**@lexfrei**](https://github.com/lexfrei) in cozystack/cozy-proxy#22).
* **[cozy-proxy] Prepare release v0.4.0**: Releases cozy-proxy v0.4.0 with the multi-arch image ([**@lexfrei**](https://github.com/lexfrei) in cozystack/cozy-proxy#23).
* **[ansible-cozystack] chore(deps): update ghcr.io/cozystack/cozystack/cozy-installer docker tag to v1.6.0**: Tracks the Cozystack installer image at v1.6.0 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#63).
* **[ansible-cozystack] chore(deps): update ghcr.io/cozystack/cozystack/cozy-installer docker tag to v1.6.1**: Tracks the Cozystack installer image at v1.6.1 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#65).
* **[ansible-cozystack] chore(deps): update dependency k3s-io/k3s to v1.36.3+k3s1**: Bumps the k3s version used by the management-cluster bootstrap role ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#66).
* **[ansible-cozystack] chore(deps): update ghcr.io/cozystack/cozystack/cozy-installer docker tag to v1.6.2**: Tracks the Cozystack installer image at v1.6.2 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#68).
* **[ansible-cozystack] chore(deps): update dependency k3s-io/k3s to v1.36.4+k3s1**: Bumps the k3s version used by the management-cluster bootstrap role ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#69).
* **[ansible-cozystack] feat(roles): restore NVIDIA vGPU host state across reboots**: Adds a role that restores NVIDIA vGPU host state after a reboot instead of requiring manual reconfiguration ([**@lexfrei**](https://github.com/lexfrei) in cozystack/ansible-cozystack#70).
* **[ansible-cozystack] chore(deps): update ghcr.io/cozystack/cozystack/cozy-installer docker tag to v1.6.3**: Tracks the Cozystack installer image at v1.6.3 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#71).
* **[ansible-cozystack] chore(deps): update dependency k3s-io/k3s to v1.37.0+k3s1**: Bumps the k3s version used by the management-cluster bootstrap role ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#72).
* **[ansible-cozystack] fix(roles): ask the driver before writing a vGPU profile**: The vGPU boot unit now asks the driver before writing a profile, so cards that do not report SR-IOV host mode are no longer misconfigured ([**@lexfrei**](https://github.com/lexfrei) in cozystack/ansible-cozystack#76).
* **[ansible-cozystack] chore(deps): update ghcr.io/cozystack/cozystack/cozy-installer docker tag to v1.6.4**: Tracks the Cozystack installer image at v1.6.4 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#78).
* **[ansible-cozystack] docs(roles): list the slim platform variants**: Documents the new slim platform variants in the role ([**@lexfrei**](https://github.com/lexfrei) in cozystack/ansible-cozystack#79).
* **[ansible-cozystack] chore(deps): update dependency k3s-io/k3s to v1.37.1+k3s1**: Bumps the k3s version used by the management-cluster bootstrap role ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#81).
* **[ansible-cozystack] chore(deps): update ghcr.io/cozystack/cozystack/cozy-installer docker tag to v1.6.5**: Tracks the Cozystack installer image at v1.6.5 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#87).
* **[ansible-cozystack] chore(deps): update actions/setup-python action to v7.1.0**: Moves the repository's CI to actions/setup-python v7.1.0; no change to the roles (merged after ansible-cozystack v1.6.5, not yet in a release) ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#88).
* **[external-apps-example] fix(scripts): make .PHONY in package.mk an actual declaration**: Fixes `scripts/package.mk` so `.PHONY` is a real declaration rather than a variable assignment (the repository has no releases) ([**@lexfrei**](https://github.com/lexfrei) in cozystack/external-apps-example#4).
* **[external-apps-example] docs: add the Apache-2.0 LICENSE file**: Adds the Apache-2.0 LICENSE file so GitHub reports the repository's license (the repository has no releases) ([**@kvaps**](https://github.com/kvaps) in cozystack/external-apps-example#5).
## Contributors
We'd like to thank all contributors who made this release possible:
* [**@2mind**](https://github.com/2mind)
* [**@androndo**](https://github.com/androndo)
* [**@Arsolitt**](https://github.com/Arsolitt)
* [**@batonogov**](https://github.com/batonogov)
* [**@chathurabuddi**](https://github.com/chathurabuddi)
* [**@crossRT**](https://github.com/crossRT)
* [**@gireassen**](https://github.com/gireassen)
* [**@harshitrwt**](https://github.com/harshitrwt)
* [**@IvanHunters**](https://github.com/IvanHunters)
* [**@kvaps**](https://github.com/kvaps)
* [**@kvapsova**](https://github.com/kvapsova)
* [**@lexfrei**](https://github.com/lexfrei)
* [**@lfinmauritius**](https://github.com/lfinmauritius)
* [**@lllamnyp**](https://github.com/lllamnyp)
* [**@magic-peach**](https://github.com/magic-peach)
* [**@matthieu-robin**](https://github.com/matthieu-robin)
* [**@mattia-eleuteri**](https://github.com/mattia-eleuteri)
* [**@myasnikovdaniil**](https://github.com/myasnikovdaniil)
* [**@nightcityblade**](https://github.com/nightcityblade)
* [**@opsmon**](https://github.com/opsmon)
* [**@pujitha24**](https://github.com/pujitha24)
* [**@scooby87**](https://github.com/scooby87)
* [**@shaurya703**](https://github.com/shaurya703)
* [**@sircthulhu**](https://github.com/sircthulhu)
* [**@themoriarti**](https://github.com/themoriarti)
* [**@tym83**](https://github.com/tym83)
* [**@yankawai**](https://github.com/yankawai)
* [**@youdie006**](https://github.com/youdie006)
* [**@yrahman82**](https://github.com/yrahman82)
### New Contributors
We're excited to welcome our first-time contributors:
* [**@2mind**](https://github.com/2mind) - First contribution!
* [**@batonogov**](https://github.com/batonogov) - First contribution!
* [**@chathurabuddi**](https://github.com/chathurabuddi) - First contribution!
* [**@crossRT**](https://github.com/crossRT) - First contribution!
* [**@gireassen**](https://github.com/gireassen) - First contribution!
* [**@harshitrwt**](https://github.com/harshitrwt) - First contribution!
* [**@lfinmauritius**](https://github.com/lfinmauritius) - First contribution!
* [**@magic-peach**](https://github.com/magic-peach) - First contribution!
* [**@nightcityblade**](https://github.com/nightcityblade) - First contribution!
* [**@opsmon**](https://github.com/opsmon) - First contribution!
* [**@pujitha24**](https://github.com/pujitha24) - First contribution!
* [**@shaurya703**](https://github.com/shaurya703) - First contribution!
* [**@youdie006**](https://github.com/youdie006) - First contribution!
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.6.0...v1.7.0
+89
View File
@@ -0,0 +1,89 @@
<!--
https://github.com/cozystack/cozystack/releases/tag/v1.7.0
-->
# Cozystack v1.7.0
Cozystack v1.7.0 adds **Proxmox VE as a substrate for tenant Kubernetes workers**, turns worker pools into first-class **`KubernetesNodes` resources**, introduces a **community Marketplace for External-Apps repositories** together with a reworked dashboard and Admin portal, extends the unified backup interface to **MongoDB, Redis, Valkey, RabbitMQ and Kafka** (plus PostgreSQL point-in-time recovery), and brings **TLS and admission-time hardening to the managed databases**. It also adds **managed Valkey**, **distributed tracing**, a **ComputePlane tenant module**, **arm64 image builds** and **slim installation variants**.
This page lists only what has to be known before upgrading; every change in the release is in the [full changelog](https://github.com/cozystack/cozystack/blob/v1.7.0/docs/changelogs/v1.7.0.md).
## ⚠️ Breaking Changes and Required Actions
Several changes in this release alter behavior of running clusters. Read this section before applying the v1.7.0 Platform Package.
1. **Talos v1.14.2 needs `drbd_transport_tcp` (#4598).** Before upgrading a node, add `drbd_transport_tcp` to its Talos `machine.kernel.modules` next to `drbd`; without it DRBD cannot connect to any peer. ([details][pre-upgrade])
2. **Tenant Kubernetes worker pools move to `KubernetesNodes` (#3315, #3379).** Existing pools are adopted on upgrade and roll their worker VMs once; a pool that is not adopted is listed in ConfigMap `cozy-system/cozystack-migration-56-unadopted` and needs to be adopted or recreated by hand, a cluster that uses the `ingressNginx` addon needs a pool with `roles: [ingress-nginx]`, and the `addons.fluxcd` addon is removed. ([details][pre-upgrade])
3. **Kamaji rolls every tenant control plane (#4398).** Every tenant kube-apiserver rolls on the first reconcile and again when the tenant's `kubernetes` release is upgraded; in between, a tenant that sets `--enable-admission-plugins=` in `controlPlane.apiServer.extraArgs` runs with Kamaji's default admission plugins, which do not include NodeRestriction. Plan the upgrade window accordingly. ([details][pre-upgrade])
4. **Cilium takes its addresses from a reserved block on Kube-OVN variants (#4778).** `networking.podCIDR` must be `/20` or wider on these variants, and each Cilium agent restarts twice when upgrading from v1.6.4 or earlier. ([details][pre-upgrade])
5. **KubeVirt eviction and hotplug changes (#4403, #4453).** VMs that cannot live-migrate are shut down and restarted instead of blocking a drain, but a running VM keeps the old strategy until it restarts, so restart such VMs once after upgrading if their nodes need to drain. On a `VMInstance` the upgrade unplugs a volume hot-plugged only on the VMI: before upgrading, detach it inside the guest, remove it with `virtctl removevolume`, and add the disk to the VMInstance's `spec.disks`, where at v1.6.5 it attaches only at the VM's next restart. ([details][pre-upgrade])
6. **External exposure now requires TLS (#4639, #4847, #2682).** Redis exposure needs `tls.enabled: true`, external PostgreSQL refuses plaintext sessions unless `tls.required: false` is set, and external OpenSearch moves to the chart's cert-manager CA; clients that connected with `sslmode=disable` or skipped certificate verification must be updated. ([details][pre-upgrade])
7. **Tenant route hostnames are mandatory (#3475, #3342, #4638, #3872).** An `HTTPRoute`, `TLSRoute` or `GRPCRoute` in a `tenant-*` namespace must declare a non-empty `spec.hostnames`; an existing route without them keeps serving, but its next update, a Helm upgrade included, is refused. ([details][pre-upgrade])
8. **Managed PostgreSQL and MariaDB user passwords (#4078, #4574).** `users[].password` is no longer honored and passwords are always auto-generated; roles left behind by users removed from `users` on v1.6.x are dropped on the first reconcile, and clients still logging in with such a role lose access. ([details][pre-upgrade])
9. **Smaller breaking changes (#4405, #4569, #3411, #3195, #4512, #4200).** Fix `vm-instance` resources that cannot size a VM, keep `ApplicationDefinition.spec.release.prefix` within `^[a-z0-9-]*$` and upgrade FoundationDB 7.1.x clusters to 7.3 first; multus now stages the reference CNI plugins into `/opt/cni/bin`, and HTTPCache is removed from the catalog. ([details][pre-upgrade])
10. **A platform variant switch on a live cluster is refused (#4790, backport #5021).** A switch goes through only with `networking.acknowledgeVariantChange` set to the new variant, and a cluster whose `cozystack.networking` variant was set by hand needs `cozystack.networking` in `bundles.disabledPackages` before the upgrade. ([details][pre-upgrade])
11. **A golden-image disk into an unsuitable LINSTOR class is refused (#4796, backport #5022).** Existing disks are exempt; for a new one use the golden image's StorageClass, or have the platform administrator set the CDI `cloneStrategyOverride` to `copy`. ([details][pre-upgrade])
12. **Pod rolls on upgrade.** Kafka clusters, SeaweedFS components, `kube-multus`, the LINSTOR pods and the Redis Sentinel and data pods roll once (#5019, #5028, #5018, #5020, #4831), and every PostgreSQL instance restarts at least once (#3951); a cluster with one instance is unavailable while its pod restarts. ([details][pre-upgrade])
13. **FoundationDB selects a release line, not a version (#5085, backport #5108).** `version: v7.3` or `v7.4` deploys 7.3.63 or 7.4.1, a cluster on another patch of its line is moved to the pinned one, downwards included, and a release on 7.1 or on mismatched lines is refused; check the running patch of every FoundationDB cluster before upgrading and read the `cozystack.io/migration-60-needs-attention` annotation afterwards. ([details][pre-upgrade])
14. **Image overrides of Kubernetes, KubernetesNodes and OpenSearch are no longer read (#5085, backport #5108).** Stored `images.*` values are ignored, with no API warning, and the workloads move to the platform's images; check custom OpenSearch images before upgrading, set `version` on an OpenSearch release that stores `version: null`, and mirror `docker.io` and `ghcr.io` on air-gapped management-cluster nodes. ([details][pre-upgrade])
15. **A disk still importing holds its release during the upgrade (#5087, backport #5126).** Such a `vm-disk` release can fail its upgrade and occupy a helm-controller worker until it reconciles on its own, about 20 minutes on a test cluster; finish running imports before upgrading (`kubectl get dv -A`). ([details][pre-upgrade])
16. **Read the migration log right after the upgrade.** Read it with `kubectl logs -n cozy-system job/cozystack-migration-hook` once the migrations finish; on a test cluster it could no longer be read about 20 minutes after the run. ([details][pre-upgrade])
17. **Free disk space on the nodes.** Observed on a test cluster: nodes reported `DiskPressure` during the platform upgrade and the kubelet evicted pods, so make sure the nodes have free disk space before upgrading. ([details][pre-upgrade])
[pre-upgrade]: https://github.com/cozystack/cozystack/blob/v1.7.0/docs/changelogs/v1.7.0.md#pre-upgrade-checks-and-required-actions
Every breaking change, one entry per pull request: [Breaking changes in this release](https://github.com/cozystack/cozystack/blob/v1.7.0/docs/changelogs/v1.7.0.md#breaking-changes-in-this-release).
## Contributors
We'd like to thank all contributors who made this release possible:
* [**@2mind**](https://github.com/2mind)
* [**@androndo**](https://github.com/androndo)
* [**@Arsolitt**](https://github.com/Arsolitt)
* [**@batonogov**](https://github.com/batonogov)
* [**@chathurabuddi**](https://github.com/chathurabuddi)
* [**@crossRT**](https://github.com/crossRT)
* [**@gireassen**](https://github.com/gireassen)
* [**@harshitrwt**](https://github.com/harshitrwt)
* [**@IvanHunters**](https://github.com/IvanHunters)
* [**@kvaps**](https://github.com/kvaps)
* [**@kvapsova**](https://github.com/kvapsova)
* [**@lexfrei**](https://github.com/lexfrei)
* [**@lfinmauritius**](https://github.com/lfinmauritius)
* [**@lllamnyp**](https://github.com/lllamnyp)
* [**@magic-peach**](https://github.com/magic-peach)
* [**@matthieu-robin**](https://github.com/matthieu-robin)
* [**@mattia-eleuteri**](https://github.com/mattia-eleuteri)
* [**@myasnikovdaniil**](https://github.com/myasnikovdaniil)
* [**@nightcityblade**](https://github.com/nightcityblade)
* [**@opsmon**](https://github.com/opsmon)
* [**@pujitha24**](https://github.com/pujitha24)
* [**@scooby87**](https://github.com/scooby87)
* [**@shaurya703**](https://github.com/shaurya703)
* [**@sircthulhu**](https://github.com/sircthulhu)
* [**@themoriarti**](https://github.com/themoriarti)
* [**@tym83**](https://github.com/tym83)
* [**@yankawai**](https://github.com/yankawai)
* [**@youdie006**](https://github.com/youdie006)
* [**@yrahman82**](https://github.com/yrahman82)
### New Contributors
We're excited to welcome our first-time contributors:
* [**@2mind**](https://github.com/2mind) - First contribution!
* [**@batonogov**](https://github.com/batonogov) - First contribution!
* [**@chathurabuddi**](https://github.com/chathurabuddi) - First contribution!
* [**@crossRT**](https://github.com/crossRT) - First contribution!
* [**@gireassen**](https://github.com/gireassen) - First contribution!
* [**@harshitrwt**](https://github.com/harshitrwt) - First contribution!
* [**@lfinmauritius**](https://github.com/lfinmauritius) - First contribution!
* [**@magic-peach**](https://github.com/magic-peach) - First contribution!
* [**@nightcityblade**](https://github.com/nightcityblade) - First contribution!
* [**@opsmon**](https://github.com/opsmon) - First contribution!
* [**@pujitha24**](https://github.com/pujitha24) - First contribution!
* [**@shaurya703**](https://github.com/shaurya703) - First contribution!
* [**@youdie006**](https://github.com/youdie006) - First contribution!
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.6.0...v1.7.0
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/altinity-clickhouse-backup:v1.6.0@sha256:e890d78cd83968bdff9a36c9ca4312afc05e138af486e476da0a935f81f9b7a7
ghcr.io/cozystack/cozystack/altinity-clickhouse-backup:v1.7.0@sha256:b69e3407612331ed7ddfe42f1e9ab699213fe5f1400d17a5f562d3e78b655f6a
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/clickhouse-backup:v1.6.0@sha256:8efef62d59fb82d39544823f81a797c2b9ac130a50241fee48e701161ce2670d
ghcr.io/cozystack/cozystack/clickhouse-backup:v1.7.0@sha256:037e7cec9c7868587cf53159061f4b151fb7b3330346f184977fadb0f80e7c76
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/cluster-autoscaler:v1.6.0@sha256:d188f395a31e37ea09bfa91e45d7ce1606b7459192a437fa66bb760ffb5f17d2
ghcr.io/cozystack/cozystack/cluster-autoscaler:v1.7.0@sha256:6991b8f49c3d1dc7d75ee361adc5dd43c36790172d4212f82bafef094d54bff2
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/kubevirt-cloud-provider:v1.6.0@sha256:f005b99041d191eed9dde98f9f6691ba2d0cbfa1b3b0b2fe4b32cc0fdba74acf
ghcr.io/cozystack/cozystack/kubevirt-cloud-provider:v1.7.0@sha256:7f32eddd46edb85c728d824d681bff242e572b63619fa82ae5e6647fcd9acea4
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/kubevirt-csi-driver:v1.6.0@sha256:d6326e343c7932ce3d375c6742b9b9745090a4a26bf5bbc5ad19f5504ea93d65
ghcr.io/cozystack/cozystack/kubevirt-csi-driver:v1.7.0@sha256:95d2a631c93eceb9b003d57c406315e684138a6365afe1d385ed1acb99bf9730
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/talos-csr-signer:v1.6.0@sha256:85b41e16e5e40398b1272fe0a69de72e205532c8034d0e6a9fc4a1d510f17818
ghcr.io/cozystack/cozystack/talos-csr-signer:v1.7.0@sha256:4235082e93a8cd665f2af2948a0ad085fdabf29854340e6e4aba59f1b008d383
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/mariadb-backup:v1.6.0@sha256:1de944b5c4fbeef94004b8620b0865aa7886a692dacb23d50c8f9b31fbeece07
ghcr.io/cozystack/cozystack/mariadb-backup:v1.7.0@sha256:839af624d9410bd2c27cb838db2233e5c55ae92919f6c6420107d782e14bea2e
+1 -1
View File
@@ -1 +1 @@
quay.io/outline/shadowbox:v1.12.3@sha256:545c6f7c7261bb30ae1dffe24a6fca5f8512f5d17c72cfb9e410e7e655444e62
ghcr.io/cozystack/cozystack/shadowbox:v1.7.0@sha256:e0ef5583279ab27d9a7711a20bd443b3e2b1c3dfa0a53552c9e54a05b9d38781
+2 -2
View File
@@ -21,9 +21,9 @@ cozystackOperator:
# well, or the opt-out is partial. Declared here rather than left implicit
# because the documented opt-out is a `--set` against this key.
disableTelemetry: false
image: ghcr.io/cozystack/cozystack/cozystack-operator:v1.6.0@sha256:93b305f78823b070c59ca631b213e39bd9d8a256d5c520f156a03f5f72f57a5d
image: ghcr.io/cozystack/cozystack/cozystack-operator:v1.7.0@sha256:0d32c1688f4e9085bc40ffe2294ad1ce6804aca4d20216f19d322f7fd22563a1
platformSourceUrl: 'oci://ghcr.io/cozystack/cozystack/cozystack-packages'
platformSourceRef: 'digest=sha256:bf68208730860fa8e47f378a0260e79a0262e0783fa63ba4f7f97f57a48b2d23'
platformSourceRef: 'digest=sha256:d085adfd0b32dafd5ba3b7d844bf19718d8a445e1afaf12618a52d4bb51b6989'
# When non-empty, overrides the operator's --helmrelease-interval flag
# (operator default: 5m). E2E sets this to 30s; production should leave empty.
helmReleaseInterval: ""
+1 -4
View File
@@ -13,7 +13,7 @@ migrations:
# below therefore lags targetVersion in-tree by design; run-migrations.sh
# refuses to advance past a migration file missing from the image (exit 1), so
# a stale pin fails loudly rather than silently skipping the etcd adoption.
image: ghcr.io/cozystack/cozystack/platform-migrations:v1.6.0@sha256:6777042e3e9c6bad76e7d96fb64baa115061975f39061cb3ede84e21d0e2213f
image: ghcr.io/cozystack/cozystack/platform-migrations:v1.7.0@sha256:9018fba0873094431cf130cc949d4651a8d971d02e5925b70e6f22648818a4c5
targetVersion: 61
# Adopt legacy etcd clusters onto the v1alpha2 operator WITHOUT the mandatory
# pre-adoption safety snapshot (migration 50). Leave false unless migration 50
@@ -186,7 +186,6 @@ networking:
# resources:
# limits:
# memory: 8Gi
# Service publishing and ingress configuration
publishing:
host: "example.org"
@@ -715,7 +714,6 @@ monitoring:
# cozystack.cozystack-platform Package — see docs/operations/backup-classes.md.
# Empty (the default) forwards nothing and keeps the child chart defaults.
backupStorage: {}
# backupStrategyController overrides for the platform-managed strategy CRs,
# forwarded verbatim into the same Package as
# components.backupstrategy-controller.values.backupStrategyController. The
@@ -917,7 +915,6 @@ gpu:
# missing entries).
vgpuDeviceManager:
enabled: false
# VM import from external virtualization platforms (opt-in; see the iaas bundle).
vmImport:
# Container image carrying the VMware Virtual Disk Development Kit, used by
+1 -1
View File
@@ -1,2 +1,2 @@
e2e:
image: ghcr.io/cozystack/cozystack/e2e-sandbox:v1.6.0@sha256:a9980e8c48d6e50ed2b5d8245a4e0aae00970777b6e2255577fdb66035a44c68
image: ghcr.io/cozystack/cozystack/e2e-sandbox:v1.7.0@sha256:f6a1cfc0fa5b09dcc60071aee43a1f3a09d350e11681a693805118d289218153
+1 -1
View File
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/matchbox:v1.6.0@sha256:7805917462ad1288272a6324c0f32a4f9a250e1723a2759f7afebce181400f96
ghcr.io/cozystack/cozystack/matchbox:v1.7.0@sha256:d4fa0aa0578be7c358bcffd9f7c0a7ca7b9a7a8ba63df1ed80aa63832825443e
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/objectstorage-sidecar:v1.6.0@sha256:4d665becd3399c2fe1e8a66e15d14d16e4a5a75395fb0ae65cff46679a8989f1
ghcr.io/cozystack/cozystack/objectstorage-sidecar:v1.7.0@sha256:b7b0d339e0e2c5043eb03b6d78e05437daae1116c0e1a6a1d88ae4f70b296534
@@ -1,5 +1,5 @@
backupController:
image: "ghcr.io/cozystack/cozystack/backup-controller:v1.6.0@sha256:fc4571354413d86ed11fa38dee8fd3d2d879e11c48d7d92c28630b2d54a2bcbb"
image: "ghcr.io/cozystack/cozystack/backup-controller:v1.7.0@sha256:65cb050d1f240caaf35b5e35c5aed73ab197666e930146a418565b08f9bbe34c"
replicas: 2
debug: false
metrics:
@@ -1,5 +1,5 @@
backupStrategyController:
image: "ghcr.io/cozystack/cozystack/backupstrategy-controller:v1.6.0@sha256:e8bf0521d1d1d8b90ebe66e9013010887033ba2653901799d57f2df47c9d879e"
image: "ghcr.io/cozystack/cozystack/backupstrategy-controller:v1.7.0@sha256:990c8ab93b9f14ee7f70473d199558c859fa5e9532d6b1f47a74698f34c07c1b"
# chBackupClientImage is the image rendered into the Altinity strategy
# Pod: it drives clickhouse-backup's HTTP API via curl + jq.
#
@@ -35,7 +35,7 @@ backupStrategyController:
# retag both to the same stable tag, and the write-once guard in
# hack/promote-retag.sh fails the release. hack/image-pin-consistency.bats
# asserts no repository is pinned at more than one digest.
chBackupClientImage: "ghcr.io/cozystack/cozystack/platform-migrations:v1.6.0@sha256:6777042e3e9c6bad76e7d96fb64baa115061975f39061cb3ede84e21d0e2213f"
chBackupClientImage: "ghcr.io/cozystack/cozystack/platform-migrations:v1.7.0@sha256:9018fba0873094431cf130cc949d4651a8d971d02e5925b70e6f22648818a4c5"
# rabbitmqBackupClientImage is the image rendered into the RabbitMQ Job
# strategy (strategy-rabbitmq-default.yaml): its script needs only curl +
# ca-certificates to call the RabbitMQ management API and push the
@@ -48,13 +48,13 @@ backupStrategyController:
# dedicated alpine + curl + ca-certificates image built by this package's
# Makefile, which also lets this pin move on its own.
# DO NOT hand-edit: stamped by packages/core/platform/Makefile.
rabbitmqBackupClientImage: "ghcr.io/cozystack/cozystack/platform-migrations:v1.6.0@sha256:6777042e3e9c6bad76e7d96fb64baa115061975f39061cb3ede84e21d0e2213f"
rabbitmqBackupClientImage: "ghcr.io/cozystack/cozystack/platform-migrations:v1.7.0@sha256:9018fba0873094431cf130cc949d4651a8d971d02e5925b70e6f22648818a4c5"
# redisBackupClientImage is rendered into the Redis strategy Pod. The
# spotahome operator ships no backup CRD, so the strategy runs a one-shot Job
# that dumps an RDB snapshot to S3 and, on restore, loads it into a throwaway
# redis to emit a faithful replay stream. It ships its own image (redis 8 +
# curl) built from images/redis-backup/; `make image` re-digests this value.
redisBackupClientImage: "ghcr.io/cozystack/cozystack/redis-backup:v1.6.0"
redisBackupClientImage: "ghcr.io/cozystack/cozystack/redis-backup:v1.7.0@sha256:11492ac0b1899fc02aea9ff40ea72bab006b8816e426b84030528fa18be45b90"
# redisBackupActiveDeadlineSeconds bounds one backup/restore Job so a run that
# wedges (e.g. a --rdb dump blocking on a stalled master) becomes a Failed Job
# the driver reports, rather than a Running BackupJob a cron Plan stacks behind.
+1 -1
View File
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/s3manager:v1.6.0@sha256:73665aaf7f5406b4a1e456d14060fe874e0ae2a436ba7fd3058fa73abff8a927
ghcr.io/cozystack/cozystack/s3manager:v1.7.0@sha256:ad438f9618253d718e1f20f0b26059a62dd5e398347b125e52c63e485e20bb49
@@ -16292,7 +16292,7 @@ spec:
- --dynamic-infrastructure-clusters=${CACPPK_INFRASTRUCTURE_CLUSTERS:= }
command:
- /manager
image: ghcr.io/cozystack/cozystack/cluster-api-control-plane-provider-kamaji:v0.19.0-cozystack.0@sha256:a0b8367f374432a5fa0b12018a35b0e4828466f53eb6de263d90c3408d9fd07d
image: ghcr.io/cozystack/cozystack/cluster-api-control-plane-provider-kamaji:v0.19.0-cozystack.0@sha256:9e5eb9a7e28ad17d9c85be3fc9aede8436ae9b7aa5497afa340b70ad4950c889
livenessProbe:
httpGet:
path: /healthz
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/cluster-api-control-plane-provider-kamaji:v0.19.0-cozystack.0@sha256:a0b8367f374432a5fa0b12018a35b0e4828466f53eb6de263d90c3408d9fd07d
ghcr.io/cozystack/cozystack/cluster-api-control-plane-provider-kamaji:v0.19.0-cozystack.0@sha256:9e5eb9a7e28ad17d9c85be3fc9aede8436ae9b7aa5497afa340b70ad4950c889
@@ -1 +1 @@
quay.io/capk/capk-manager:v0.1.10@sha256:87c2a80413a0f64f65326d379f4905d3f3f2eaf9dab8a6a67a7f7973ca85aef6
ghcr.io/cozystack/cozystack/cluster-api-provider-kubevirt:v1.7.0@sha256:e347d8f5f6258b6158afb7bf50ee5618a2f1de264d93d30b1c9402942dd4125f
+3 -3
View File
@@ -41,8 +41,8 @@ cilium:
mode: "kubernetes"
image:
repository: ghcr.io/cozystack/cozystack/cilium
tag: v1.6.0
digest: "sha256:136a7dfff4d6adcd448424c2354144160028a8e96eb6558b917fd3ee6d7b39fe"
tag: v1.7.0
digest: "sha256:16d824903e909c1aeedfa1610a8b5676edf8169f80c324a6f735c8756ac52516"
envoy:
enabled: true
# Keep the per-type xDS the 1.19 agent and envoy speak. The 1.20 chart
@@ -134,7 +134,7 @@ cilium:
# unpatched upstream operator the rebuild starts from, so a tree rendered
# without a build still pulls an image that exists.
image:
override: "quay.io/cilium/operator-generic:v1.20.2@sha256:64d8798350e8569b8e7622563fed6e44dce2625f311e4651b774816516c744fc"
override: "ghcr.io/cozystack/cozystack/cilium-operator:v1.7.0@sha256:58f2b0bcfeaf2a996a470593457d04d1ba06f08c12774a6bc0c36f1142449226"
rollOutPods: true
replicas: 1
resources:
+1 -2
View File
@@ -1,9 +1,8 @@
# Injected by the platform from the cozystack-values Secret. The empty
# default keeps the chart renderable on its own, outside a cluster.
_cluster: {}
cozystackAPI:
image: ghcr.io/cozystack/cozystack/cozystack-api:v1.6.0@sha256:c4d374a9bc5e6f6e66d24b29396f1eae9d7f06fd268ff767d05c45ca35e84e2e
image: ghcr.io/cozystack/cozystack/cozystack-api:v1.7.0@sha256:817b7e345fbebc389f53fd19c5f5c17217973bdb153b3c0afb3071f8bbe2bddc
replicas: 2
# Empty by default: no resources block is rendered, so requests and limits are
# opt-in. Idle usage (measured) is ~130-143Mi — keep a limit above that.
@@ -1,5 +1,5 @@
cozystackController:
image: ghcr.io/cozystack/cozystack/cozystack-controller:v1.6.0@sha256:b01f6fe6e8fe6426b112fbda2b4abac946b4e45cd118137c9001c9e6ee784b3d
image: ghcr.io/cozystack/cozystack/cozystack-controller:v1.7.0@sha256:c39fcff5fdb2dbc96df91d082d31e458273f20b8ed04a9f785c7675bd0bf77c7
debug: false
disableTelemetry: false
# Base URL of a Prometheus-compatible query API to fetch SeaweedFS bucket
+2 -3
View File
@@ -1,8 +1,7 @@
# Injected by the platform from the cozystack-values Secret. The empty
# default keeps the chart renderable on its own, outside a cluster.
_cluster: {}
console:
image: ghcr.io/cozystack/cozystack/cozystack-ui:v1.6.0@sha256:ad0e132ce506bbc9e0f6a5dfb94311c9bbc08da3d8eca84eb0e527ba6c1a8897
image: ghcr.io/cozystack/cozystack/cozystack-ui:v1.7.0@sha256:55605af79f1edecba274e3c137efccb7614c15128e9186f3b1560665c6b94dc0
tokenProxy:
image: ghcr.io/cozystack/cozystack/token-proxy:v1.6.0@sha256:97124c012246c33cdcac60e8f884097b48c0081624716fe8161efc2b3ceeb2a8
image: ghcr.io/cozystack/cozystack/token-proxy:v1.7.0@sha256:b5b17859ba0f69d1dac6b2cc3ed377acc821fdb0a9e83142610ac57e40b31fb0
+1 -3
View File
@@ -1,5 +1,4 @@
image: "ghcr.io/cozystack/cozystack/flux-plunger:latest@sha256:6a6ec938973e6583c5e1573f7d25beddc1852a6699a089116b2407e29a564a72"
image: "ghcr.io/cozystack/cozystack/flux-plunger:v1.7.0@sha256:f09439183b70fd1a267f0bdbdc5d01e1bc75ad5559f75ccf216a07f277b76050"
# How long flux-plunger is given to stop, between 40 and 140. On SIGTERM it stops
# its manager (up to 10 seconds), then gives up the uninstall fences it holds,
# waiting for helm-controller to uninstall a release being deleted, under a
@@ -8,7 +7,6 @@ image: "ghcr.io/cozystack/cozystack/flux-plunger:latest@sha256:6a6ec938973e6583c
# acts. The pre-delete hook waits for the pod a little longer than this before it
# releases what is left.
shutdownGracePeriodSeconds: 90
# Image for the pre-delete uninstall-fence cleanup hook Job (see
# templates/uninstall-fence-cleanup.yaml). Only needs a kubectl binary. The tag
# is digest-pinned so an upstream retag does not change what runs across the
@@ -1,5 +1,5 @@
fluxShardOperator:
image: ghcr.io/cozystack/cozystack/flux-shard-operator:v1.6.0@sha256:daf002af34671371646185d4135c5096f8aba9518f132089bd5d6fbe7d9d06e7
image: ghcr.io/cozystack/cozystack/flux-shard-operator:v1.7.0@sha256:69e157898201e2a2b6b3404083a7e2a7499e7710ddb7f2935dd2dc9edfb649f9
debug: false
replicas: 2
## Number of helm-controller shards to provision and distribute tenants
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/grafana-dashboards:v1.6.0@sha256:cb22416b34d4385e4df58314de390920af805c03358fe00734c8fa44ba505fe9
ghcr.io/cozystack/cozystack/grafana-dashboards:v1.7.0@sha256:b6500cfe6d36e31ba9f24d94809071513b81fdcb977f7919f8bd734209275c78
+16 -16
View File
@@ -3,37 +3,37 @@ harbor:
# with a multi-arch rebuild from images/.
core:
image:
repository: docker.io/goharbor/harbor-core
tag: v2.15.1@sha256:887a85b8ea98b76bfc9f715f1a0785bb99f9a1034241513902dd6e95be922a83
repository: ghcr.io/cozystack/cozystack/harbor-core
tag: v1.7.0@sha256:b645c46af670bbff36313466ffe955f8c0c4ea81cd8eb20f6d82a904be4be934
jobservice:
image:
repository: docker.io/goharbor/harbor-jobservice
tag: v2.15.1@sha256:0de4fd2ce3a02d3e6591b439e4674ea085885ddf43652b44004cc67eb19dba12
repository: ghcr.io/cozystack/cozystack/harbor-jobservice
tag: v1.7.0@sha256:b1988c0fc72d4710d11f7f19ebc0a15b506d986606ce84ab5627861a69b206f0
portal:
image:
repository: docker.io/goharbor/harbor-portal
tag: v2.15.1@sha256:ac55161c57a8351807adf8f8def264bdd52667c371d0436beefebdac4341c9e2
repository: ghcr.io/cozystack/cozystack/harbor-portal
tag: v1.7.0@sha256:0974f606b8f6aaed7abd400c421e8e0e175d277394623f53bffd873479cdeec5
registry:
registry:
image:
repository: docker.io/goharbor/registry-photon
tag: v2.15.1@sha256:ebf0325c2661729dbb317cbf839608eb8b15cfa158911a94976f2c21563c466e
repository: ghcr.io/cozystack/cozystack/registry-photon
tag: v1.7.0@sha256:bb7103f8ea3784b7924da12404806f1b2f19e2d284c54f2049766aaaf687054b
controller:
image:
repository: docker.io/goharbor/harbor-registryctl
tag: v2.15.1@sha256:554147a956989175f63f8d41573d716c6ddf6052acd1749c88c0f99ce6ee2bff
repository: ghcr.io/cozystack/cozystack/harbor-registryctl
tag: v1.7.0@sha256:223af937d92110210ee0dbdcd19c8d2c365918236dc41111393acd1885303217
exporter:
image:
repository: docker.io/goharbor/harbor-exporter
tag: v2.15.1@sha256:372275b6b6abeb571fa7bd5de458e3d75f1a1636503b3c0abe02938f2b285d75
repository: ghcr.io/cozystack/cozystack/harbor-exporter
tag: v1.7.0@sha256:b437212b0206bc39e2f930b3b8113d0af0b00ee6108db7e76850f916bb698a7c
nginx:
image:
repository: docker.io/goharbor/nginx-photon
tag: v2.15.1@sha256:1f9d06d040f2478f6cf7f6da0091877bf987a641810f4bfaadb2a87d73dbf4fb
repository: ghcr.io/cozystack/cozystack/nginx-photon
tag: v1.7.0@sha256:0b424a225d291786b1c15d7eb7ad4ac4a33c9a0a845a8487747f17afe4c0f5f0
trivy:
image:
repository: docker.io/goharbor/trivy-adapter-photon
tag: v2.15.1@sha256:f0c52bed36c083ec4fded26cefab01e4cd58c9354c261cb3156c69f57bb50131
repository: ghcr.io/cozystack/cozystack/trivy-adapter-photon
tag: v1.7.0@sha256:ab6a59fac0abdc58f752d6bc50ec69c6a900f856448891b2701921533ca576b9
db:
replicas: 2
size: 5Gi
+2 -2
View File
@@ -28,7 +28,7 @@ kamaji:
topologyKey: kubernetes.io/hostname
image:
pullPolicy: IfNotPresent
tag: v1.6.0@sha256:7f7751b3d01b256621a131c217fc90464cd51d3d07a6d870c3d8f28784745293
tag: v1.7.0@sha256:93cc08b8ffa56a84633efc7b31b7fba876e44a1e5fd0f4d953aeee899e8f312e
repository: ghcr.io/cozystack/cozystack/kamaji
resources:
limits:
@@ -61,4 +61,4 @@ kamaji:
periodSeconds: 10
timeoutSeconds: 1
extraArgs:
- --migrate-image=ghcr.io/cozystack/cozystack/kamaji:v1.6.0@sha256:7f7751b3d01b256621a131c217fc90464cd51d3d07a6d870c3d8f28784745293
- --migrate-image=ghcr.io/cozystack/cozystack/kamaji:v1.7.0@sha256:93cc08b8ffa56a84633efc7b31b7fba876e44a1e5fd0f4d953aeee899e8f312e
@@ -4,7 +4,7 @@ keycloak-operator:
# cannot reach a build registry whose `registry` key holds only part of it.
registry: ""
repository: ghcr.io/cozystack/cozystack/keycloak-operator
tag: "latest@sha256:27279247dfbd2d1e86fc4e827d34c891880ca65eefd75da1701bc0a1adf865a9"
tag: "v1.7.0@sha256:298f5b846f8c16f32bae67df4e811194029beabaf80b1d44820e428f2b5940af"
clusterReconciliationEnabled: true
resources:
limits:
+1 -1
View File
@@ -1,7 +1,7 @@
kilo:
image:
pullPolicy: IfNotPresent
tag: v1.5.0@sha256:fe837f428504f6c6e9e4582326a3bf9cd507a9cca58c975952f5cc0b34728f63
tag: v1.7.0@sha256:c6216f8b9eec0e45aa265c9df0fcaeb8a71277cd9b242c0e5e5d91cc2980286c
repository: ghcr.io/cozystack/cozystack/kilo
transitCIDR: 100.66.0.0/16
meshGranularity: location
+1 -1
View File
@@ -1,4 +1,4 @@
portSecurity: true
routes: ""
image: ghcr.io/cozystack/cozystack/kubeovn-plunger:v1.6.0@sha256:0d2a42d5aa05f73e9a5bab2ffbcf20c64e2925b7b5f50d7b39b7d366a1d8f3d6
image: ghcr.io/cozystack/cozystack/kubeovn-plunger:v1.7.0@sha256:aabaeb663b978e095fd783c5265d2b05216297c93bb949a7431f352e3c8fd03e
ovnCentralName: ovn-central
+1 -2
View File
@@ -28,8 +28,7 @@ routes: ""
# and this key pointed at the mounted path. Until then the flag exists so the
# observation above can run, not so it can be switched on.
clientCAFile: ""
image: ghcr.io/cozystack/cozystack/kubeovn-webhook:v1.6.0@sha256:593c324a38db59495497c5b68b90cad25dfe6753c73b7a287b3280e26f70a15f
image: ghcr.io/cozystack/cozystack/kubeovn-webhook:v1.7.0@sha256:4c0b4434a3d3cf525c2256acc9aa93e87ff9bf479c766258f6f9e74c51db0af0
# Restrict who may reach the webhook's serving port.
#
# The handler returns namespace-derived annotations to any caller, and only the
+1 -1
View File
@@ -79,4 +79,4 @@ global:
images:
kubeovn:
repository: kubeovn
tag: v1.15.26@sha256:741299cbd0081a786a6b60c460fa3156b3a42a38141c559dd8ac031f50c5504f
tag: v1.7.0@sha256:69b0cf6602eb2e518858d7fe66f8d2742530c209b134d48950067613431e0212
@@ -9,4 +9,4 @@ storageClasses:
infraStorageClass: replicated
default: false
csiDriver:
image: ghcr.io/cozystack/cozystack/kubevirt-csi-driver:v1.6.0@sha256:d6326e343c7932ce3d375c6742b9b9745090a4a26bf5bbc5ad19f5504ea93d65
image: ghcr.io/cozystack/cozystack/kubevirt-csi-driver:v1.7.0@sha256:95d2a631c93eceb9b003d57c406315e684138a6365afe1d385ed1acb99bf9730
@@ -1,5 +1,5 @@
lineageControllerWebhook:
image: ghcr.io/cozystack/cozystack/lineage-controller-webhook:v1.6.0@sha256:3a5b4951a78d507eb847f4f3311052810dc658fd7c85c49eeb32ac5469cb39a2
image: ghcr.io/cozystack/cozystack/lineage-controller-webhook:v1.7.0@sha256:27f616fe78eceec1195e4e54a2a354a89a83ccbf1fdf9541c1fd8bfbc6646732
debug: false
replicas: 2
# DEPRECATED. Injects KUBERNETES_SERVICE_HOST=status.hostIP and
+1 -2
View File
@@ -1,13 +1,12 @@
# Injected by the platform from the cozystack-values Secret. The empty
# default keeps the chart renderable on its own, outside a cluster.
_cluster: {}
## @section Image
## @param image.repository LINSTOR GUI container image repository
## @param image.tag LINSTOR GUI container image tag (digest recommended)
image:
repository: ghcr.io/cozystack/cozystack/linstor-gui
tag: v1.6.0@sha256:6e11829de86709f5e21636cdd1a1ada1239a7dcc1741f91e9d95d9f6ade603a2
tag: v1.7.0@sha256:899ce54eadb25dd26abdd804db3316119ac075a87c331de6adcecbbf8d33267b
## @section Deployment
## @param replicas Number of linstor-gui replicas
replicas: 1
+2 -2
View File
@@ -1,7 +1,7 @@
piraeusServer:
image:
repository: ghcr.io/cozystack/cozystack/piraeus-server
tag: v1.6.0@sha256:70ddd1a66fbc1ea58078f4645fc572ab8e2e189dbb4d9a5f9ca43d5663106483
tag: v1.7.0@sha256:ba23d02dd92839181703cdfe194d68aac8f652e7af6e82234089a81f96c50098
# Talos-specific workarounds (disable for generic Linux like Ubuntu/Debian)
talos:
enabled: true
@@ -99,7 +99,7 @@ linstorCSI:
strictTopology: true
image:
repository: ghcr.io/cozystack/cozystack/linstor-csi
tag: v1.6.0@sha256:ae854423de62e94fb89b59d92c20da8939196e643e83b2d60d9f6915d5b0e55c
tag: v1.7.0@sha256:760c0e5434927d4995061e932a123b361c974ef1dfd361a58516801120a35dd7
controller:
# Resource requests for the linstor-controller JVM. Requests (not limits) are
# set on purpose: a CPU request guarantees CPU shares so the JVM can answer its
+2 -2
View File
@@ -37,11 +37,11 @@ metallb:
controller:
image:
repository: ghcr.io/cozystack/cozystack/metallb-controller
tag: v1.6.0@sha256:cb7321c5674509048dfbea7938378b84013d1370eca5ace7f9665fc79b4a8a05
tag: v1.7.0@sha256:ad13b61e71b9332c6fad4aa78d9ed9857058eb3a165c2512301bbced6ebba764
speaker:
image:
repository: ghcr.io/cozystack/cozystack/metallb-speaker
tag: v1.6.0@sha256:61ab7e2381f0897607b15df7cb5ba4d51a947c261c1c197fcda3b79c7a280016
tag: v1.7.0@sha256:6325666b9b83d378319ac528ca28256db99dc7e21150afc92168fcd33d28adfd
# A memory limit is what sets memory.max on the pod cgroup, and the Talos
# userspace OOM handler (v1.12+) only ever kills cgroups that lack it. The
# speaker ran BestEffort, so it was picked as the victim whenever a tenant
@@ -1,5 +1,5 @@
migrationController:
image: "ghcr.io/cozystack/cozystack/migration-controller:latest"
image: "ghcr.io/cozystack/cozystack/migration-controller:v1.7.0@sha256:16493deaf49706ac3422cd771d3b023abd13ad450f876d78fa0d55ebfe06399e"
replicas: 1
debug: false
metrics:
@@ -12,7 +12,6 @@ migrationController:
limits:
cpu: 500m
memory: 256Mi
## The VMware Virtual Disk Development Kit image, delivered from the platform.
## Never a tenant value: naming an image the cluster will run is not a tenant's
## decision, and this one is built from a proprietary SDK that Cozystack can
+43 -49
View File
@@ -2,63 +2,58 @@
# defaults keep the chart renderable on its own, outside a cluster.
_cluster: {}
_namespace: {}
host: ""
metricsStorages:
## Example:
## metricsStorages:
## - name: shortterm
## retentionPeriod: "3d"
## deduplicationInterval: "15s"
## storage: 10Gi
## storageClassName: ""
## vminsert:
## minAllowed:
## cpu: 200m
## memory: 512Mi
## maxAllowed:
## cpu: 1500m
## memory: 3Gi
## vmselect:
## minAllowed:
## cpu: 300m
## memory: 1Gi
## maxAllowed:
## cpu: 3500m
## memory: 6Gi
## vmstorage:
## minAllowed:
## cpu: 500m
## memory: 2Gi
## maxAllowed:
## cpu: 4000m
## memory: 8Gi
- name: shortterm
retentionPeriod: "3d"
deduplicationInterval: "15s"
storage: 10Gi
storageClassName: ""
- name: longterm
retentionPeriod: "14d"
deduplicationInterval: "5m"
storage: 10Gi
storageClassName: ""
## Example:
## metricsStorages:
## - name: shortterm
## retentionPeriod: "3d"
## deduplicationInterval: "15s"
## storage: 10Gi
## storageClassName: ""
## vminsert:
## minAllowed:
## cpu: 200m
## memory: 512Mi
## maxAllowed:
## cpu: 1500m
## memory: 3Gi
## vmselect:
## minAllowed:
## cpu: 300m
## memory: 1Gi
## maxAllowed:
## cpu: 3500m
## memory: 6Gi
## vmstorage:
## minAllowed:
## cpu: 500m
## memory: 2Gi
## maxAllowed:
## cpu: 4000m
## memory: 8Gi
- name: shortterm
retentionPeriod: "3d"
deduplicationInterval: "15s"
storage: 10Gi
storageClassName: ""
- name: longterm
retentionPeriod: "14d"
deduplicationInterval: "5m"
storage: 10Gi
storageClassName: ""
logsStorages:
- name: generic
retentionPeriod: "1"
storage: 10Gi
storageClassName: ""
- name: generic
retentionPeriod: "1"
storage: 10Gi
storageClassName: ""
tracingStorages: []
tracingCentralHost: false
tracingCollector:
enabled: true
replicas: 1
samplingPercentage: 10
image: docker.io/otel/opentelemetry-collector-contrib:0.136.0@sha256:45392d534c1edcc809c2d112394029246bc679d2ae5ea7081414a1fc74f2c621
image: ghcr.io/cozystack/cozystack/otelcol-tracing:v0.136.0-cozystack.0@sha256:006107b8a0467d87746f873d15d032ab729babef86b1cbe1fd8a9c16562d61dd
redactAttributes: []
resources:
limits:
@@ -123,7 +118,6 @@ vmagent:
## - job_name: "custom"
## static_configs:
## - targets: ["my-service:9090"]
## OIDC authentication for the Grafana instance.
##
## Identity mode picks the auth source for `[auth.generic_oauth]`:
@@ -156,7 +156,7 @@ spec:
serviceAccountName: multus
containers:
- name: kube-multus
image: ghcr.io/cozystack/cozystack/multus-cni:v1.6.0@sha256:0a92e861ead179dd6b46f475fd8878d44b3cfa40ce4d7a8ff92f790d54250d25
image: ghcr.io/cozystack/cozystack/multus-cni:v1.7.0@sha256:f8732c27bfdfe48073e514be4eed8c6eeec3b9405b26906bc05ddc1c28606367
command: [ "/usr/src/multus-cni/bin/multus-daemon" ]
# Keep a memory limit: without one the pod cgroup has no memory.max,
# and the Talos userspace OOM handler (v1.12+) only ever kills cgroups
@@ -222,7 +222,7 @@ spec:
fieldPath: spec.nodeName
initContainers:
- name: install-multus-binary
image: ghcr.io/cozystack/cozystack/multus-cni:v1.6.0@sha256:0a92e861ead179dd6b46f475fd8878d44b3cfa40ce4d7a8ff92f790d54250d25
image: ghcr.io/cozystack/cozystack/multus-cni:v1.7.0@sha256:f8732c27bfdfe48073e514be4eed8c6eeec3b9405b26906bc05ddc1c28606367
command:
- "/usr/src/multus-cni/bin/install_multus"
- "-d"
@@ -248,7 +248,7 @@ spec:
{{- if not (has $stageCniPlugins (list "true" "yes" "on" "1" "false" "no" "off" "0")) }}{{- fail (printf "stageCniPlugins: expected a boolean, got %q" $stageCniPlugins) }}{{- end }}
{{- if has $stageCniPlugins (list "true" "yes" "on" "1") }}
- name: install-cni-plugins
image: ghcr.io/cozystack/cozystack/multus-cni:v1.6.0@sha256:0a92e861ead179dd6b46f475fd8878d44b3cfa40ce4d7a8ff92f790d54250d25
image: ghcr.io/cozystack/cozystack/multus-cni:v1.7.0@sha256:f8732c27bfdfe48073e514be4eed8c6eeec3b9405b26906bc05ddc1c28606367
command:
- "sh"
- "-c"
@@ -1,3 +1,3 @@
objectstorage:
controller:
image: "ghcr.io/cozystack/cozystack/objectstorage-controller:v1.6.0@sha256:9c284ef4f2fbbf5ee3ccd75f86ec6c993dc60aa673c6df3f9bd15d21a7e8485c"
image: "ghcr.io/cozystack/cozystack/objectstorage-controller:v1.7.0@sha256:f0c6f9c12fd3197debef467670f4793235e9057b094893abfcbb9fdf00c80ea7"
@@ -16,12 +16,11 @@ opensearch-operator:
# retag and mirror. `make image` rewrites both keys on every build.
image:
repository: ghcr.io/cozystack/cozystack/opensearch-operator
tag: v2.8.0@sha256:cbe6b50f13aa5abf338209d38dfcd80e73296e97510465b5fca73a0130bfaad7
tag: v1.7.0@sha256:c52607d8ea24cfb557968cbddc8fff26859b11c9afbc4d535408a89013297918
kubeRbacProxy:
image:
# gcr.io/kubebuilder/kube-rbac-proxy was sunset; use the original upstream
repository: "quay.io/brancz/kube-rbac-proxy"
sysctl:
# busybox carries the sysctl applet and doubles as the idle main container.
# Third-party pass-through pinned by digest, the same reference the kubernetes
+1 -1
View File
@@ -1,7 +1,7 @@
redis-operator:
image:
repository: ghcr.io/cozystack/cozystack/redis-operator
tag: v3.3.5@sha256:551f39340bee3d6ea8e1da973cfc64092b769526c4a7aea0427070cbf6702ddd
tag: v1.7.0@sha256:b86abaa7fc5dd8525f20f74b9f02c99884cbe42fd4afe62d3bd722ca9cdbf2a4
# The freshworks-oss fork requires a non-empty operator group id at startup
# (fatal "operator_group_id environment variable is required" otherwise); the
# archived spotahome operator had no such knob. A single cluster-wide operator
+2 -2
View File
@@ -31,7 +31,7 @@ seaweedfs:
# `make image` rewrites both keys on every build.
image:
repository: "ghcr.io/cozystack/cozystack/seaweedfs"
tag: "4.45@sha256:2a063deb9992ac0c06f8c09589ae063a30ac8c87af2c6aa1f4d4420cc544535c"
tag: "v1.7.0@sha256:6d4b63d9c28e7553eebc45bd3723cd0160d2fdb6b4f6fd39a869c7cab0839f71"
master:
# At the shipped replicationFactor of 2 the size limit and
# extraEnvironmentVars.WEED_MASTER_VOLUME_GROWTH_COPY_2 below are one
@@ -310,7 +310,7 @@ seaweedfs:
bucketClassName: "seaweedfs"
region: ""
sidecar:
image: "ghcr.io/cozystack/cozystack/objectstorage-sidecar:v1.6.0@sha256:4d665becd3399c2fe1e8a66e15d14d16e4a5a75395fb0ae65cff46679a8989f1"
image: "ghcr.io/cozystack/cozystack/objectstorage-sidecar:v1.7.0@sha256:b7b0d339e0e2c5043eb03b6d78e05437daae1116c0e1a6a1d88ae4f70b296534"
certificates:
commonName: "SeaweedFS CA"
ipAddresses: []
@@ -1,5 +1,5 @@
securityGroupController:
image: "ghcr.io/cozystack/cozystack/securitygroup-controller:v1.6.0@sha256:09ee1d8fbfacc5b4ac24d0efbc8313bb0ad0e5507a149235894ac7d4b5539232"
image: "ghcr.io/cozystack/cozystack/securitygroup-controller:v1.7.0@sha256:d8e73d20cc03f1d47fa904c98574595ba67582712d63839b91784005ea636c1c"
replicas: 2
debug: false
resources:
@@ -1 +1 @@
ghcr.io/cozystack/cozystack/vector:0.56.0-alpine@sha256:0eb66216f5f9322264e2ba83f4606428ef77a2cd4dea619a5bea610ac80ccc43
ghcr.io/cozystack/cozystack/vector:v1.7.0@sha256:4b5d6ee2d1802bca9b858e278978bc44131738997cbc4e0f37691da8c968f9c9
+1 -1
View File
@@ -11,7 +11,7 @@ velero:
volumeMounts:
- mountPath: /target
name: plugins
- image: quay.io/kubevirt/kubevirt-velero-plugin:v0.9.0
- image: ghcr.io/cozystack/cozystack/kubevirt-velero-plugin:v1.7.0@sha256:e2e18356449b502b3d819bb299ec642c48e260b141d38b12a36c0eab881ff32a
name: kubevirt-kubevirt-velero-plugin
imagePullPolicy: IfNotPresent
volumeMounts: