Commit Graph
389 Commits
Author SHA1 Message Date
IvanHunters 59964043e3 fix(platform): migration 49 backfills tenant ancestor labels
The tenant chart fix makes new and re-reconciled tenant namespaces
self-compute their full tenant.cozystack.io/<ancestor> chain, but existing
namespaces whose HelmRelease does not promptly re-reconcile (suspended or
failed releases) keep the incomplete labels — and with them the broken
<ancestor>-egress reachability that surfaces as upstream timeouts to nested
tenants. Backfill the labels directly so clusters heal on upgrade regardless
of per-tenant HelmRelease state.

Bump targetVersion to 50 so run-migrations.sh iterates the new script
(seq CURRENT..49); the missing-file guard otherwise never fires. Best-effort
and idempotent: kubectl label --overwrite adds only the listed labels, and
per-namespace failures are tolerated so a transient apiserver error cannot
abort the platform upgrade.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-03 09:54:12 +03:00
Алексей Артамонов a2ae1621af fix(clickhouse): surface cleanup hook failures and backfill legacy keeper PVC labels
Address review on #3072:

- Cleanup hook no longer masks kubectl delete failures: the command now
  fails (exit 1) when PVC deletion fails instead of swallowing the error
  with || echo. The hook-delete-policy drops hook-succeeded (keeps only
  before-hook-creation) so a failed cleanup Job survives for inspection.
- Add migration 45 (45 -> 46) that backfills app.kubernetes.io/instance
  onto keeper PVCs created before this change; the altinity keeper operator
  never re-stamps volumeClaimTemplate labels onto existing PVCs, so without
  this the cleanup selector misses pre-upgrade keeper PVCs and #3057
  persists for existing releases. Bump migrations.targetVersion to 46.
- Unify helm.sh/hook-weight key quoting across the four hook manifests.
- Extend the unittest to assert the new hook-delete-policy and exit-on-failure.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
2026-07-01 18:31:05 +03:00
Aleksei Sviridkin cbc314a8c5 fix(kubevirt-cdi): order behind cozystack-basics so cdi-clone-dv RoleBinding finds cozy-public
The kubevirt-cdi default variant only depended on cozystack.networking,
which reconciles long before cozystack-basics provisions the cozy-public
namespace. kubevirt-cdi ships the cdi-clone-dv RoleBinding into cozy-public,
so on a fresh install the RoleBinding can race namespace creation and fail
with `namespaces "cozy-public" not found`. Flux retries normally mask it,
but under slow reconcile it surfaces as a hard UpgradeFailed.

Add a cozystack.cozystack-basics edge so the namespace exists first,
mirroring the backupstrategy-controller ordering. No cycle: cozystack-basics'
own dependencies (tenant-application, cozystack-engine, gateway-api-crds) do
not reach kubevirt-cdi. Pin the edge with a helm-unittest regression guard.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 19:51:46 +03:00
Aleksei Sviridkin bf14a73b39 feat(api): add SecurityGroup network policy resource (sdn.cozystack.io) (#2922)
## What this PR does

Introduces a tenant-facing `SecurityGroup` resource in a new
`sdn.cozystack.io/v1alpha1` API group, served by the Cozystack
aggregated API server as a 1:1 projection of a `CiliumNetworkPolicy`. A
SecurityGroup attaches to one of the tenant's managed applications by
reference (`spec.targetRef: {apiGroup, kind, name}`) and declares
allow-list ingress/egress for that application. Tenants manage network
policy for their own apps entirely through
`securitygroups.sdn.cozystack.io`, without being granted any access to
the `cilium.io` API group.

The translation is synchronous and stateless — the API server converts
each SecurityGroup into a CiliumNetworkPolicy in the same namespace on
write and back on read; there is no controller. The backing policy's
`endpointSelector` is **derived** from `targetRef` (matched on the
application's lineage labels
`apps.cozystack.io/application.{group,kind,name}`), never copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
Policies owned by this API carry an `sdn.cozystack.io/securitygroup`
marker label that the storage always re-asserts and hides from the view,
so it only ever surfaces or mutates its own policies and never touches
platform-managed network policies.

The value is self-service network policy on a Cozystack-native,
RBAC-scoped surface, without exposing Cilium's version-coupled CRDs to
tenants. Because the selector is derived from an authorized application
reference rather than free-form tenant input, the RBAC boundary is
structural, not selector-validated: a tenant cannot express network
policy against arbitrary or platform-owned pods. The full design —
selector derivation from lineage labels, the structural boundary, the
additive-policy reality (a SecurityGroup widens a managed application's
allowed traffic but cannot sever its platform-managed management plane,
and equally cannot tighten below the platform baseline), the in-tree
CiliumNetworkPolicy mirror that avoids a Cilium Go-module dependency,
RBAC, and deferred extensions (raw-pod targeting, cluster scope,
reusable groups) — is in `pkg/apis/sdn/DESIGN.md`.

Implements the attachment model from #1614.

### Release note

```release-note
feat(api): add the SecurityGroup resource (sdn.cozystack.io/v1alpha1) — a tenant-facing, namespace-scoped firewall that attaches to a managed application by reference and projects 1:1 onto a CiliumNetworkPolicy with a machine-derived endpoint selector, letting tenants manage their application's network policy without cilium.io access.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added the `SecurityGroup` API resource (`sdn.cozystack.io` v1alpha1)
with a namespaced projection backed by Cilium network policies.
* Enabled the SDN `APIService` and wired SDN routing, scheme, and REST
support.
* Expanded RBAC to include SecurityGroup access and required Cilium
policy permissions.
* **Documentation**
* Added a design document describing projection behavior and rule
semantics.
* **Bug Fixes**
  * Improved API rule validation coverage for missing list-type fields.
* **Tests**
* Added SDN round-trip, fuzzing, RBAC, and comprehensive REST
create/get/list/update/delete/watch/resource-version tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 14:30:23 +03:00
Aleksei Sviridkin f9d1138c6e feat(keycloak): configurable login theme via platform branding (#3124)
## What this PR does

Makes a realm's Keycloak login theme configurable through platform
`branding` values, closing a gap: `branding` could set the realm display
name but not its login theme, and the keycloak chart's `themes:` list
was not reachable from platform values.

Two new keys under `branding` (propagated via `_cluster.branding`, the
same path that already feeds `displayName`):

- `branding.loginTheme` — keycloak-configure renders it as
`ClusterKeycloakRealm.spec.themes.loginTheme`, so the realm selects that
login theme.
- `branding.loginThemeImage` — when set together with `loginTheme`, the
keycloak chart appends `{name: loginTheme, image: loginThemeImage}` to
its effective theme list, so the existing init-container mounts the
theme image into `/opt/keycloak/themes/`. No StatefulSet patching or
per-package values plumbing.

`loginTheme` alone (no image) still selects the theme — for themes
already baked into the Keycloak image. Explicit chart `themes:` entries
keep working; the branding-derived entry is appended after them.

Flow: platform `branding` → `_cluster.branding` (cozystack-values
Secret) → keycloak-configure (realm loginTheme) + keycloak (theme init
container). Covered by helm-unittest in both charts.

### Screenshots

No bundled UI assets — this PR adds the mechanism only; the login theme
is operator-supplied via `loginThemeImage`, so the visual result depends
on that image.

### Release note

```release-note
feat(keycloak): select the realm login theme and ship its theme image through platform `branding` values (`branding.loginTheme`, `branding.loginThemeImage`)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for configuring a custom login theme from platform
branding, including automatic selection in the realm setup.
* Theme entries can now be combined with existing theme settings and
rendered in the expected order.

* **Documentation**
* Expanded configuration guidance for branding and theme-related
settings with clearer examples.

* **Tests**
* Added coverage for login theme rendering and fallback behavior when
theme settings are missing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 12:29:52 +03:00
Aleksei Sviridkin 31c15b282f feat(network): mechanism-agnostic LoadBalancer exposure (ExposureClass + ServiceExposure) (#3081)
## What this PR does

Lays the foundation for migrating tenant LoadBalancer exposure off the
deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off
MetalLB onto Cilium-native announcement — the mechanism-agnostic shape
prescribed when the earlier chart-side approach (#2468) was reverted in
#2603.

It adds a new `network.cozystack.io/v1alpha1` API group and a controller
that translates intent into whichever LoadBalancer backend the cluster
admin configured:

- `ExposureClass` (cluster-scoped, admin): binds a logical class name to
a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP
address scope — the StorageClass/IngressClass pattern.
- `ServiceExposure` (namespaced, tenant intent): "expose this Service
via class X"; status reports the resolved backend and assigned IPs.
- A `serviceexposure` controller renders the backend's pool + announcer
once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium
`CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to
the namespaces that opted in. Backends are pluggable behind a small Go
interface, so adding BGP or another cloud LB is one implementation, not
an API change. The controller never mutates the target Service — the
chart still owns its shape.

Platform wiring: `publishing.exposureClass.*` renders the admin
`ExposureClass` and the host ingress publishes through it. The default
(no class set, `externalIPs`) leaves every existing cluster's behaviour
byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes
a single `ExposureClass.spec.backend` change.

This is the foundation only — Part of #2586. Converting per-app
exposure, flipping Cilium `l2announcements` on, changing the default,
and removing MetalLB are deferred to later phases.

### Release note

```release-note
feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added new `ExposureClass` and `ServiceExposure` custom resources with
support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB),
including optional L2 and address configuration.
* Added Helm wiring and templates to enable service/ingress publishing
via `ServiceExposure`, selecting a configured (or default)
`ExposureClass`.
* **Bug Fixes**
* Improved reconciliation, finalizer-based cleanup, and status updates
for readiness and assigned IP reporting, including safer handling when
backends or managed resources change.
* **Tests**
* Added end-to-end and unit test coverage for backend rendering,
readiness, and cleanup flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 12:29:48 +03:00
Myasnikov Daniil b9749cec80 fix(platform): md0 migration -- fail closed and preserve the no-delete label
Migration 47 pins the implicit default md0 node group before the kubernetes
chart upgrade so the live md0 MachineDeployment is not pruned. Two correctness
fixes:

- Stamp the cozystack-version ConfigMap via the shared stamp_cozystack_version
  helper instead of `kubectl create configmap | kubectl apply`. The label-less
  apply by the same field manager stripped the platform.cozystack.io/no-delete
  label migration 42 added, dropping cozystack-version out of the
  cozystack-no-delete-guardrail policy; the helper re-emits the label and is
  enforced for every migration >= 42 by hack/cozystack-version-stamp.bats.
- Fail closed. A non-NotFound read error is now counted as a failure rather
  than silently skipped, and any failure exits 1 so the pre-upgrade hook aborts
  before Helm rolls out the new chart against clusters whose md0 is not yet
  pinned. The version stamp stays at 47 and the migration retries next upgrade.

Also narrow the merge patch to only md0 so a concurrent edit to a sibling node
group between the read and the patch cannot be reverted.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-30 12:08:17 +05:00
mattia-eleuteri 602b8d06fd fix(platform): add migration 47 to pin implicit md0 node group
After the kubernetes chart stops force-merging a default md0 node group,
clusters that relied on the implicit md0 (defined other nodeGroups without
md0) would have their live md0 MachineDeployment pruned by Helm on the next
reconcile. Migration 47 makes md0 explicit on those clusters to preserve the
current node topology; it is idempotent, treats a missing CRD as a no-op, and
skips clusters that already define md0.

Signed-off-by: mattia-eleuteri <mattia@hidora.io>
2026-06-30 12:08:17 +05:00
Aleksei Sviridkin 60c9b39578 docs(platform): document branding keys including login theme
The branding map shipped as an undocumented empty map. Describe the keys it
actually supports (brandName, brandHtmlName) and the new login-theme keys
(loginTheme, loginThemeImage) consumed by the Keycloak charts, with an example.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-28 05:08:47 +03:00
Myasnikov Daniil 70320e0abf fix(platform): stamp migrations 45/46 via the cozystack-version helper
Migrations 45 and 46 (added by #2931) stamp the cozystack-version ConfigMap
with an inline `kubectl apply` heredoc instead of sourcing
lib/cozystack-version.sh and calling stamp_cozystack_version, the way 42/43/44
do. The cozystack-version-stamp.bats guard forbids any migration >= 42 from
bypassing the helper, so `make bats-unit-tests` fails on migration 45.

This only surfaces in PR CI (make unit-tests); main-push CI does not run the
bats suite, so #2931 landed red on this test and now blocks every PR that
merges current main.

Route both stamps through the helper. The helper renders a byte-identical
labeled manifest (golden-pinned in the bats suite), so this is behaviour-
preserving — it just restores the no-label-drift guarantee the heredoc form
could silently lose.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-26 20:40:02 +05:00
myasnikovdaniil 1df8c7c557 feat(kubernetes): bootstrap tenant workers with Talos via CABPT (#2931)
## What this PR does

> **Supersedes #2610.** Same work, rebased cleanly onto current `main`
so it now sits on top of #2867 (the `kube-controller-manager` VAP
type-checker fix). That fix is what was killing the base install in
#2610's E2E *before* the `kubernetes` app test ever ran, leaving the
Talos worker-bootstrap rewrite verified only by code reading. #2610's
67-commit branch is squashed here into 8 logical commits with original
authorship preserved (@kvaps and @IvanHunters as authors); the tree is
**byte-identical** to #2610's head — no code changes, just the rebase
onto `main` (to pick up #2867) plus history cleanup. The detailed review
history lives on #2610.

Phase 1 of the Kubernetes-app split design (see cozystack/community#8):
replace the Ubuntu+kubeadm worker bootstrap path with Talos, driven by
`cluster-api-bootstrap-provider-talos` (CABPT) and a
`clastix/talos-csr-signer` sidecar embedded in the Kamaji control-plane
pod. Existing tenants keep working — old machines roll out and get
replaced by Talos workers without manual intervention.

Highlights:

- Add CABPT (v0.6.12) as a second `BootstrapProvider` alongside the
existing kubeadm one in `capi-providers-bootstrap`.
- Bump the Kamaji control-plane provider with an upstream-bound patch
that exposes `KamajiControlPlane.spec.network.additionalServicePorts`,
used to publish trustd (50001/TCP) on the apiserver service.
- Generate Talos PKI (Ed25519 CA + trustd TLS) via cert-manager and
stable random Talos secrets (`token`, `clusterId`, `clusterSecret`,
`bootstrapToken`) using the helm lookup-and-reuse pattern.
- Materialise a kubeadm-format `bootstrap-token-<id>` Secret inside the
tenant `kube-system` via a Helm post-install/upgrade Job.
- Render a `TalosConfigTemplate` (worker machineconfig, `generateType:
none`) and switch the `MachineDeployment.bootstrap` reference from
`KubeadmConfigTemplate` to `TalosConfigTemplate`, gated on Talos secrets
and the Kamaji apiserver service being ready.
- Boot workers from the Talos openstack image via a CDI `DataVolume`
(`source.http.url` → `factory.talos.dev`), expose the system disk as
virtio-blk with `blockSize.custom: logical=512, physical=4096` so
4Ki-native block storage backends (LINSTOR/DRBD) play nicely with QEMU's
O_DIRECT writes and SeaBIOS still boots, drop the separate kubelet disk
(Talos lays out EPHEMERAL itself).
- Pin pod/service CIDRs in the worker machineconfig to the tenant ranges
(`10.243.0.0/16` / `10.95.0.0/16`) to avoid Talos's address-overlap
diagnostic against the host pod CIDR.
- Set `cluster.controlPlane.endpoint` and `cilium.k8sServiceHost` to the
Kamaji apiserver Service ClusterIP (looked up at render time), so
bootstrap survives the chicken-and-egg moment where tenant DNS does not
exist yet and the host CoreDNS does not serve the tenant zone.

End-to-end on a dev cluster: a fresh tenant brings up a Talos worker,
kubelet CSRs get approved (apiserver-client by Kamaji, kubelet-serving
by the talos-csr-signer sidecar), Cilium initialises with hostNetwork
against the apiserver ClusterIP, CoreDNS comes up, and the node
transitions to `Ready`.

### Release note

```release-note
feat(kubernetes): bootstrap tenant workers with Talos Linux instead of Ubuntu+kubeadm. Existing clusters roll over to Talos workers automatically; new tenants come up on Talos from the start. Powered by cluster-api-bootstrap-provider-talos and the talos-csr-signer sidecar in the Kamaji control-plane pod. Note: the tenant `kubernetes` HelmRelease now reports Ready as soon as Helm completes (Install/Upgrade DisableWait) and no longer blocks on worker/addon readiness — worker-rollout health is tracked by WorkloadMonitor, not the HelmRelease.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

**New Features**
* Added Talos worker configuration for `talos.version` and
`talos.schematicID`
* Introduced `nodeHealthCheck` (`maxUnhealthy`, `nodeStartupTimeout`)
for worker remediation tuning
* Added image overrides for `kubectl` and `talosCsrSigner`
* Exposed additional tenant control-plane Service ports support

**Breaking Changes**
* Removed Kubernetes v1.30; supported versions start at v1.31
* Renamed `nodeGroups.ephemeralStorage` → `nodeGroups.diskSize` with
consolidated Talos system-disk semantics

**Removals**
* Removed Ubuntu container-disk images and related containerd patch
mechanisms
* Air-gapped tenant workers temporarily unsupported during the Talos
transition

**Documentation**
* Updated chart docs, parameters, and GitOps upgrade guidance for
version bumps
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-26 20:06:49 +05:00
Kirill Ilin 5cb6ef435f feat(keycloak): opt-in separate hostname and route for the admin console (#2954)
## What this PR does

Adds an opt-in way to serve the Keycloak admin console and
Administration
REST API on a separate hostname and route from the public login
endpoints,
so the admin surface can be kept off the public internet.

- New `ingress.adminHost` in the keycloak chart. When set:
  - `KC_HOSTNAME_ADMIN` is configured so the admin console generates its
    URLs on that hostname;
  - the public (login) route is narrowed to the OIDC paths only
    (`/realms`, `/resources`, `/.well-known`), so the admin console and
    Administration REST API are no longer served on the login hostname;
- a dedicated route (Ingress or HTTPRoute, depending on
`gateway-enabled`)
    serves the admin hostname.
- New platform `publishing.ingressNameAdmin`, propagated as
  `_cluster.expose-ingress-admin`. The admin route attaches to the
Gateway/ingressClass it names, defaulting to `expose-ingress`. Pointing
it
at a separate (for example, private-IP) Gateway moves admin traffic off
the public one. The separate Gateway itself is provisioned out of band;
  this only selects where the admin route attaches.

Everything is empty by default, so existing deployments are unchanged.

Note: `KC_HOSTNAME_ADMIN` on its own is not a security boundary — the
Administration REST API stays reachable on any hostname routed to
Keycloak
(per the Keycloak docs). The path and network separation introduced here
is
what restricts access; full isolation requires attaching the admin route
to
a non-public Gateway.

Covered by helm-unittest suites for the keycloak chart and the platform
value wiring.

### Screenshots

N/A — no UI changes.

### Release note

```release-note
feat(keycloak): add opt-in `ingress.adminHost` to serve the admin console and Administration REST API on a separate hostname/route, plus `publishing.ingressNameAdmin` (`_cluster.expose-ingress-admin`) to attach that route to a separate (e.g. private) Gateway/ingressClass. Disabled by default; existing deployments are unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added optional configuration to serve the Keycloak admin console on a
dedicated hostname, with separate routing and ingress/gateway attachment
so admin UI/API can be isolated from public login/OIDC paths.
* Introduced a cluster-level setting to optionally expose admin routes
via a separate ingress/gateway class/namespace.

* **Tests**
* Added new YAML test coverage to validate defaulting, hostname split
behavior, and correct routing for both gateway-enabled and
gateway-disabled modes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-26 14:08:23 +05:00
Aleksei Sviridkin 67692b4544 feat(platform,ingress): publish host ingress via ServiceExposure
Add the operator-facing knob that drives the new exposure model. When
publishing.exposureClass.name is set, the platform renders a cluster-scoped
ExposureClass binding the chosen backend (externalIPs, metallb, cilium,
robotlb) and VIP address scope, and carries the class name through the
cozystack-values Secret. The host ingress then renders a ServiceExposure
for its ingress-nginx Service, which the cozystack-controller translates
into the backend's pool and announcer.

The default (empty name) renders neither the ExposureClass nor the
ServiceExposure, so clusters on the externalIPs path are unchanged on
upgrade. The ingress Service shape is untouched: it is already published
as type LoadBalancer whenever externalIPs is empty.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-25 20:47:19 +03:00
IvanHunters e0597e3d99 fix(platform): make *-rd sources depend on cozystack-engine (#3041)
## Problem

On an unlucky Flux reconcile order, a `*-application` PackageSource that
ships a `system/*-rd` component can install before the
`applicationdefinitions.cozystack.io` CRD exists. Every `*-rd` chart
renders an `ApplicationDefinition` (`cozystack.io/v1alpha1`); that CRD
is installed by the `application-definition-crd` component of
`cozystack.cozystack-engine`. When an `*-rd` HelmRelease renders first,
it hard-fails with `no matches for kind "ApplicationDefinition" in
version "cozystack.io/v1alpha1"`, and that cascades into a broad install
timeout — surfacing even on changes that have nothing to do with
ordering, because it is a latent race.

## Fix

Add `- cozystack.cozystack-engine` to the `dependsOn` of every
`*-application` source that ships a `system/*-rd` component (29
sources). `gateway-application` already carried this edge — it was added
earlier for the TenantGateway CRD, the same bug class, but only for that
one source. A new invariant test fails if any source with a
`system/*-rd` component is missing the edge.

## Why

The ordering was previously implicit, relying on Flux happening to
reconcile `cozystack-engine` before the `*-application` sources. Making
the edge explicit removes the race deterministically. No dependency
cycle is introduced: `cozystack-engine` depends only on `networking` and
`cert-manager` (plus keycloak in its oidc variant), none of which reach
back to any `*-application`.

## Test

- New generic guard `hack/check-applicationdefinition-crd-ordering.bats`
(auto-discovered by `make bats-unit-tests`, runs in CI via `make
unit-tests`): parses every source and fails if any PackageSource variant
with a `system/*-rd` component lacks the `cozystack.cozystack-engine`
edge — so future sources are covered, not just today's set.
- `helm unittest packages/core/platform`: 13 suites / 66 tests pass; the
source edits do not disturb existing assertions.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved dependency ordering across several platform packages to
ensure related components install in the correct sequence.
* Prevented end-to-end test selection from being overly broad when only
ordering-related dependencies change.

* **Tests**
* Added checks to catch missing dependency requirements and to verify
test selection behavior stays accurate.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-25 19:21:50 +03:00
IvanHunters f6d5086190 fix(platform): order backupstrategy-controller after cozystack-basics (#3037)
## Problem

On a fresh install, the install step can fail with:

```text
Error creating resource via patch ... apps.cozystack.io/v1alpha1 Kind=Bucket
  name=cozy-backups namespace=tenant-root: namespaces "tenant-root" not found
```

`backupstrategy-controller` renders the `cozy-backups` Bucket **into**
the `tenant-root` namespace, but its `dependsOn` only guards the CRDs it
needs (backup-controller / bucket-application / objectstorage-controller
/ velero) — not the existence of the target namespace. `tenant-root` is
provisioned by `cozystack-basics` (the static tenant-root release), and
there is no ordering edge to it. On an unlucky Flux reconcile order the
Bucket patch races namespace creation, fails, and that transient failure
races the install script's `kubectl wait` deadline → the run fails.

This is latent and not introduced by any particular change — the
Bucket-into-tenant-root predates recent branches — so it flakes installs
on changes that have nothing to do with backups.

## Fix

Add `cozystack.cozystack-basics` to `backupstrategy-controller`'s
`dependsOn` so the `tenant-root` namespace exists before the Bucket is
applied. `cozystack-basics` is a dependency-graph leaf (nothing depends
on it), so no cycle is introduced.

Pinned with a helm-unittest regression guard
(`tests/sources_backupstrategy_dependson_test.yaml`) asserting the edge
— mirroring the existing `sources_basics_dependson_test.yaml` pattern.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved install ordering so the backup strategy component waits for
required foundational services before deploying, reducing race
conditions during fresh installs.
* Clarified dependency relationships for cluster-wide platform
components to better reflect safe upgrade and installation behavior.

* **Tests**
* Added regression coverage to verify the backup strategy package
declares all required dependencies, including namespace and CRD
providers.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-25 19:21:25 +03:00
Aleksei Sviridkin 69247ba80c feat(platform): deploy the securitygroup-controller
Add the securitygroup-controller Helm package (Deployment, ServiceAccount,
ClusterRole and binding, image build) and register it as a default system
package so it ships with the platform.

The ClusterRole grants cluster-wide pod patch — the controller stamps the
SecurityGroup membership label on managed-app pods across dynamically-created
tenant namespaces, so the grant cannot be namespace-scoped — plus read and
finalizer patch on the backing CiliumNetworkPolicies, and leader-election
leases. Tenants gain no new permissions; the membership label is written
exclusively by this platform controller. helm-unittest pins the pod-patch and
ciliumnetworkpolicies grants so a dropped verb fails the suite rather than
silently failing closed at runtime.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-25 17:50:33 +03:00
Kirill Ilin caeb2c466f feat(platform): add publishing.ingressNameAdmin for a separate admin gateway
Expose a new opt-in publishing.ingressNameAdmin that is propagated to
_cluster.expose-ingress-admin. System components (currently Keycloak)
use it to attach their admin routes to a separate Gateway/ingressClass,
e.g. one bound to a private IP and unreachable from the public internet.
Empty by default, so admin endpoints keep sharing expose-ingress. The
private Gateway itself is provisioned out of band; this only selects
where the admin route attaches.

Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
2026-06-25 15:28:47 +05:00
Aleksei Sviridkin fd6e121b28 fix(platform): make *-rd sources depend on cozystack-engine
Every *-application PackageSource that ships a system/*-rd component
renders an ApplicationDefinition (cozystack.io/v1alpha1). That CRD is
installed by the application-definition-crd component of
cozystack-engine. Without an explicit dependsOn edge, a fresh install
where Flux reconciles a *-application source before cozystack-engine
fails to render with `no matches for kind "ApplicationDefinition" in
version "cozystack.io/v1alpha1"`, which cascades into a broad install
timeout unrelated to whatever change triggered the reconcile.

Add `- cozystack.cozystack-engine` to the dependsOn of every source that
ships a system/*-rd component; gateway-application already carried the
edge for the TenantGateway CRD. Add a generic invariant test that fails
if any such source is missing the edge, so future sources cannot
reintroduce the race.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 22:43:56 +03:00
Aleksei Sviridkin ee853b00bd fix(platform): order backupstrategy-controller after cozystack-basics
backupstrategy-controller creates the cozy-backups Bucket into the
tenant-root namespace, which cozystack-basics provisions. On a fresh
install the Bucket patch can race namespace creation and fail with
`namespaces "tenant-root" not found`, which then races the install
wait deadline and fails the run — on changes that have nothing to do
with backup ordering. Add the cozystack-basics dependsOn edge so the
target namespace exists first, pinned with a helm-unittest regression
guard.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 22:34:09 +03:00
Myasnikov Daniil bf92f3f3b2 chore(kubernetes): make migration 45 executable and drop tracking-PR refs
Set the executable bit on migration 45 to match its siblings (44, 46). Replace
"PR #2610" references in migrations 45/46 with a description of the change.
Correct migration 45's comment: the KubeadmConfig children CAPI spawns from the
template are owned by their Machine (not Helm), so they carry no
app.kubernetes.io/managed-by=Helm label, are not matched by the selector, and
are never at risk of Helm pruning.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-24 22:19:46 +05:00
Aleksei Sviridkin de44ecf15a test(platform): enforce go-forward version stamps route through the helper
The helper header and run-migrations.sh claimed every migration-driven
stamp routes through the helper, but migrations 1-41 still stamp the
cozystack-version ConfigMap label-less and are backfilled by migration 42.
Scope the comments to go-forward stamps (migration 42 onward and the
bootstrap path), and add a unittest asserting no migration numbered >= 42
nor run-migrations.sh stamps cozystack-version without the helper, so the
no-drift invariant is enforced rather than aspirational.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 15:32:52 +03:00
Aleksei Sviridkin 985b4b0954 fix(platform): guard stamp_cozystack_version against a missing version
Mirror render_cozystack_version_manifest's argument guard in
stamp_cozystack_version so a missing version aborts before the kubectl
pipe. Without it, a render failure in a shell without pipefail would
leave the pipeline exit status to kubectl, which would apply empty input.
Add a unittest covering the guard.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 15:32:05 +03:00
Aleksei Sviridkin 3aabb0cb7c fix(platform): stamp the no-delete label on bootstrap cozystack-version
The bootstrap path that creates cozystack-version when the ConfigMap is
absent applied it without the platform.cozystack.io/no-delete label,
leaving it unguarded by the cozystack-no-delete-guardrail
ValidatingAdmissionPolicy. Route it through the shared helper so the
bootstrap ConfigMap carries the label like every migration-driven stamp.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 15:32:05 +03:00
Aleksei Sviridkin b199b5e438 refactor(platform): stamp cozystack-version via shared helper in migrations
Migrations 42, 43 and 44 each carried a copy-pasted heredoc that applied
the labeled cozystack-version ConfigMap. Source the shared helper and call
it instead, so the labeled manifest has a single definition and future
migrations cannot reintroduce a label-less apply.

Behavior-preserving: the rendered manifest is byte-identical to the
previous heredocs. Migration 42 keeps its MIGRATION_DRY_RUN path, now
building its manifest from the helper's pure renderer.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 15:32:05 +03:00
Aleksei Sviridkin 50344f1ab9 refactor(platform): add shared cozystack-version stamp helper
Extract the labeled cozystack-version ConfigMap stamp into a sourced
shell helper so migrations cannot drift back to a label-less apply that
strips the platform.cozystack.io/no-delete label.

render_cozystack_version_manifest emits the manifest on stdout (pure,
unit-testable); stamp_cozystack_version renders and applies it. The
namespace defaults to cozy-system and honors $NAMESPACE.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 15:32:04 +03:00
Aleksei Sviridkin bebf4db0df chore(build): refresh first-party Alpine runtime bases to 3.24 (#3009)
## What this PR does

Moves the pinned Alpine runtime bases of the first-party images forward
to `alpine:3.24`, clearing the published `musl` / `zlib` advisories
carried by the stale layers.

| image | base |
| --- | --- |
| mariadb-backup | 3.20 → 3.24 |
| cozystack-operator | 3.22 → 3.24 |
| platform-migrations | 3.22 → 3.24 |
| grafana-dashboards | 3.22 → 3.24 |
| kubeovn-webhook | 3.21.3 → 3.24 |
| redis-operator | latest → 3.24 |

Each runtime `apk` set was verified to resolve on `alpine:3.24`. These
images rebuild on the release asset pipeline, which re-pins the digests
in `values.yaml` on the next build — matching how the rest of the ~30
first-party images pick up the shared base (the issue's "rebuild and run
the standard test pipeline").

**Scope notes:**

- The build-only `alpine AS source` stages (keycloak-operator,
objectstorage-controller) are left as-is: their final images are
distroless, so no Alpine layer ships.
- The s3manager base is owned by the separate bucket bump (#2887), and
kilo's base moves in its own image rebuild (#2898).
- **Side effect for platform-migrations**: its `kubectl` / `helm` come
from the Alpine repo, so the base bump moves them `1.33 → 1.36` and
`3.18 → 3.19`. Both are current upstream stable and in skew with the
modern control plane for the migration scripts' operations (apply / get
/ label / helm release ops).

Closes #2902

### Release note

```release-note
chore(images): refresh first-party Alpine runtime bases to 3.24 to clear the musl/zlib advisories
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Updated Alpine Linux base images across multiple application
containers to version 3.24 (including backup utilities,
platform/operator components, monitoring dashboards, and
networking/webhook components) to improve security and build
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-23 13:14:15 +03:00
Aleksei Sviridkin 51c5273d0b feat(platform): ACME wildcard certificate on the default ingress-nginx path (#2988)
## What this PR does

On the default ingress-nginx path (`gateway.enabled=false`) every
published hostname still minted its own per-host ACME certificate via
ingress-shim, even with a DNS-01 solver configured. That hits the Let's
Encrypt rate limit (50 certs per registered domain per week) once a
deployment exceeds ~50 endpoints. The Gateway API path already issues a
single per-apex wildcard in DNS-01 mode; this brings the ingress-nginx
path to parity. It implements the remaining ingress-nginx gap that #2400
was narrowed to in its triage comment (DNS-01 is already multi-provider,
and the Gateway path already issues wildcards).

The new `publishing.certificates.wildcard` toggle is opt-in and OFF by
default. When set to `true` with `solver=dns01` and no operator-provided
wildcard Secret, the platform issues one shared wildcard `Certificate`
for `<root-host>` + `*.<root-host>` and serves the resulting Secret as
the publishing controller's `--default-ssl-certificate`. The system
service Ingresses (dashboard, grafana, keycloak, harbor, …) then stop
requesting a per-host cert.

How it works:

- `core/platform` `apps.yaml` computes the effective wildcard Secret
name and a new `_cluster.wildcard-issue` signal. An operator-provided
`wildcardSecretName` (the BYO path) always wins and never triggers
issuance; HTTP-01 cannot issue wildcards; and when Gateway API is
enabled the TenantGateway controller issues the wildcard instead — so
all three are excluded from auto-issuance.
- `extra/ingress` renders the wildcard `Certificate` only on the
publishing controller (`Release.Namespace == expose-ingress`), because
the Secret must be same-namespace for ingress-nginx to read it. The
Certificate references the DNS-01 `ClusterIssuer` that
`cert-manager-issuers` already renders in `dns01` mode, so no per-tenant
Issuer is minted.

This reuses the existing wildcard-secret consumption path, so no
per-service Ingress template changes were needed — only the issuance
side is new.

Why opt-in (default off): the chosen wildcard Secret name rides the
`_cluster` channel, which every child tenant inherits verbatim, so this
is not root-tenant scoped. A single-label wildcard `*.<root-host>`
covers `<service>.<root-host>` but not a custom service host outside it
(a keycloak `ingress.host`, harbor `host`, or grafana `host` pointed at
another domain) nor a child tenant's nested host
(`<service>.<tenant>.<root-host>`). Enabling issuance makes every such
service drop its per-host ACME cert and fall back to the default
certificate, which does not cover it. Leaving it off by default means a
dns01 cluster is never silently switched on upgrade; an operator who
enables it accepts the same coverage responsibility as the
operator-provided wildcard path, which propagates identically. The
tradeoff is pinned by contract tests (keycloak custom host, bucket
nested child-tenant host). Per-tenant wildcards remain on the Gateway
path and are tracked separately.

Docs: cozystack/website#588

Closes #2400. Part of #2811.

### Screenshots

N/A — no UI changes.

### Release note

```release-note
feat(platform): add opt-in shared wildcard certificate issuance on the default ingress-nginx path via publishing.certificates.wildcard (default false). When enabled with a DNS-01 solver, the platform issues one *.<root-host> wildcard for system services instead of a per-host ACME certificate, avoiding Let's Encrypt rate limits at scale.
```



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added automatic wildcard certificate issuance support for DNS-01 ACME
solver.
* New `certificates.wildcard` configuration option to enable shared
wildcard TLS certificates.
* Ingress rules now properly utilize wildcard certificates when enabled.

* **Tests**
* Added comprehensive test suites validating wildcard certificate
generation and ingress integration across multiple scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-23 13:13:48 +03:00
Myasnikov DaniilandClaude Opus 4.8 b5cc2ae120 fix(kubernetes): address review findings on Talos worker bootstrap
- talos-reconcile-job: emit a `---` separator inside the nodeGroups
  range so a tenant with multiple node groups renders valid
  multi-document YAML instead of two Jobs concatenated into one doc
- talos-reconcile + bootstrap-token jobs: add a restricted
  securityContext (drop ALL caps, no privilege escalation, read-only
  root fs) and CPU/memory requests+limits; drop the unused management
  SA token mount on the bootstrap-token job, which only talks to the
  tenant apiserver via the mounted admin kubeconfig
- migration 44: list Helm-managed resources by the
  app.kubernetes.io/managed-by=Helm label, not meta.helm.sh/release-name
  (an annotation, so it never matched as a --selector — the keep
  annotation was silently never applied)
- cluster.yaml: fail fast on a malformed nodeHealthCheck.maxUnhealthy
  rather than silently coercing it to 0 via `| int`
- config: make helm-install-disable-wait parsing genuinely
  case-insensitive (EqualFold + trim) to match its documented contract
- capi-providers bootstrap/cpprovider Makefiles: guard the
  patches/*.diff loop against an empty glob

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-23 14:45:53 +05:00
IvanHuntersandMyasnikov Daniil cc9485e425 fix(platform): pre-Talos-rollover migrations to protect existing tenants
Migration 44 pins Helm-managed KubeadmConfigTemplates with resource-policy=keep so Helm cannot prune them mid-rollover; migration 45 bumps tenants on spec.version v1.30 to v1.31 ahead of the Talos support-matrix guard. Both stamp cozystack-version with its deletion-protection label. targetVersion is bumped to 46.

Co-authored-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-23 14:43:36 +05:00
Aleksei Sviridkin 381edde331 chore(build): refresh first-party Alpine runtime bases to 3.24
Moves the pinned Alpine runtime bases of the first-party images forward to
alpine:3.24 to clear the published musl / zlib advisories carried by the stale
layers:

- mariadb-backup: 3.20 -> 3.24
- cozystack-operator: 3.22 -> 3.24
- platform-migrations: 3.22 -> 3.24
- grafana-dashboards: 3.22 -> 3.24
- kubeovn-webhook: 3.21.3 -> 3.24 (also drops the over-specific patch pin in
  favour of the minor-float convention used elsewhere)
- redis-operator: latest -> 3.24 (pins the previously-floating runtime base)

These rebuild on the release asset pipeline; the digests pinned in values.yaml
are re-pinned there on the next build, matching how the rest of the ~30
first-party images pick up the shared base.

Side effect for platform-migrations: its kubectl / helm come from the Alpine
repo, so the base bump moves them 1.33 -> 1.36 and 3.18 -> 3.19. Both are
current upstream stable and compatible with the management cluster; for the
cozystack migration scripts (apply / get / label / helm release ops) the newer
kubectl is in skew with the modern control plane.

The build-only `alpine AS source` stages (keycloak-operator,
objectstorage-controller) are left as-is: their final images are distroless, so
no Alpine layer ships. The s3manager base is owned by the separate bucket bump,
and kilo's base moves in its own image rebuild.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-23 05:03:45 +03:00
Aleksei Sviridkin 34563ce8c7 feat(platform): add opt-in shared wildcard issuance for ingress path
On the default ingress-nginx path (gateway.enabled=false) a DNS-01 solver
still minted a per-host ACME certificate for every system service, hitting
Let's Encrypt rate limits once a deployment exceeds ~50 endpoints. Add an
opt-in publishing.certificates.wildcard toggle (default false) and compute,
in apps.yaml, an effective wildcard secret name plus a wildcard-issue
signal: when enabled with solver=dns01 and no operator-provided
wildcardSecretName, the platform targets one shared wildcard cert
(*.<root-host> + <root-host>) instead of per-host certs.

Issuance is off by default because a single-label wildcard does not cover
a custom service host outside *.<root-host> (e.g. a keycloak ingress.host,
harbor host, or grafana host pointed at another domain); enabling it would
otherwise switch such a host to the controller's default certificate and
serve the wrong certificate. The operator-provided (BYO) name always wins
and never triggers issuance; http01 cannot issue wildcards; and the
Gateway API path issues its own wildcard via the controller, so all three
are excluded. This reuses the existing wildcard-secret-name consumption
path, so no per-service ingress template changes are needed.

Part of #2811.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-22 20:01:40 +03:00
Aleksei Sviridkin e9aaee286e fix(mongodb-operator): apply CRD updates on upgrade via CreateReplace
psmdb-operator evolves its CRD schema between minor versions; 1.22.0 adds
spec.storageScaling/storageAutoscaling, vault and other fields. Helm never
upgrades CRDs shipped in crds/, so without an explicit policy the operator
binary upgrades on an existing cluster while the live CRD stays behind and
the apiserver rejects the new fields. Opt the mongodb-operator PackageSource
into CreateReplace, consistent with etcd-operator, and add a platform test
pinning the policy.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-22 19:05:03 +03:00
cozystack-ci[bot] e1aaa90ed3 Prepare release v1.5.0
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-06-22 10:55:37 +00:00
Myasnikov DaniilandClaude dc9427285d Merge branch 'main' into feat/flux-shard-operator-impl
Resolve go.mod conflict: keep main's go 1.26.4 and golang.org/x/* bumps, and keep gomodules.xyz/jsonpatch/v2 v2.4.0 as a direct dependency (imported by the flux-shard-operator admission webhook) instead of main's indirect entry. go mod tidy leaves go.sum consistent.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-22 10:19:07 +05:00
myasnikovdaniil aeaedca4a4 ci: consolidated platform & e2e stabilization batch (#2948)
## What

Consolidated landing branch for a set of interdependent platform/CI
stabilization fixes. It began as the source PRs in the table below (in
dependency order) and has since grown with review-driven fixes and a few
production-behavior changes — the full current inventory is under
**Scope** below. Landing them as one unit lets CI run against the real
combined state instead of a fragile bottom-up merge train.

## Source PRs (dependency order)

| # | Commit | Source PR | Addresses |
|---|--------|-----------|-----------|
| 1 | fix(capi): startupProbe on capi-controller-manager | #2946 | capi
crashloop during cert provisioning |
| 2 | test(e2e): bucket.bats port-forward + S3 client reliability |
#2944 | flaky bucket test |
| 3 | fix(seaweedfs): restore -lock BucketClass, s3 svc name, drop
volumeSizeLimitMB | #2943 | missing bucket/access classes |
| 4 | test(e2e): in-cluster Cilium endpoint-leak healer (install + apps)
| #2874 | cilium "IP already in use" leak |
| 5 | refactor(build): mode=max registry cache + main-only warmer |
#2938 | build cache |
| 6 | ci(build): isolate each PR build on its own ephemeral runner VM |
#2939 | shared-buildkit contention |
| 7 | fix(e2e): LINSTOR post-install waits on a single 15m deadline |
#2928 | LINSTOR provisioning timeouts |
| 8 | test(metallb): assert digest-pinned image form, not version
literal | #2873 | brittle metallb assertion |

## Why consolidated

These form a dependency DAG (verified from CI logs): e.g. 2943 needs
2946, 2938 needs 2874, 2928 needs 2938/2939, 2873 needs 2928. Tested
bottom-up, each lower PR runs with none of the fixes above it and can't
go green alone. This branch carries all of them, so CI runs against the
real combined state.

## Verification status

CI is green on the latest head (`5653c30`): full-suite E2E passes
end-to-end and is reproducible (2 of 3 runs on this SHA green). The one
red run was a LINSTOR tie-breaker / DRBD-metadata infra flake on the
sandbox, unrelated to the diff — a different environmental subsystem
failed each run (details in the comments below). The earlier 3-hour
crust-gather snapshot hang is fixed in `5653c30`.

## Scope beyond the original 8

The branch has grown past the 8 source PRs above with review-driven
fixes and a few production-behavior changes. For reviewer transparency,
the full set:

**Review fixes (@lexfrei review):**
- **B1 (blocker)** — `8f41910` converts the SeaweedFS
`-lock`/`-readonly` BucketClasses and the s3 service-name override into
`patches/`, wired into `make update`, so a re-vendor no longer drops
them.
- **FU1** — `cc99230` grants the cilium leak-healer `delete` on pods
(both delete remedies were RBAC-forbidden).
- **FU3** — `014de3f` adds a unit test for the HelmRelease update
conflict-retry path.
- **FU4** — `6deddc6` makes the leak-healer refuse a disconnect for any
non-terminal owner phase, not just `Running`.
- **FU7** — `f3914d2` sets `persist-credentials: false` on the
build-cache checkout (SHA-pinning is handled repo-wide in #2849).

**Production-behavior changes (not in the table above):**
- VPA `updateMode: Auto → Initial` for etcd (`6e9ff90`) and monitoring
(`0997105`), plus `vmselect`/`vmstorage` `minAllowed` floors — stops
install-time eviction churn.
- cozystack-api: `retry.RetryOnConflict` on the Application→HelmRelease
update path (`728f676`).

**E2E robustness follow-ups:** per-test crust-gather snapshots
(`50e5f94`), EtcdBackupSchedule wait (`763b85e`), tenant-node
single-deadline wait (`9e43d45`), harbor BucketClaim 10m budget
(`d7f244f`), tenant API via LoadBalancer (`9beda4e`), crust-gather pin
(`95f42aa`).

**Not addressed (by decision):** FU2 — the s3 Service keeps
`seaweedfs.name`-based naming via patch (rendered output unchanged;
cozystack renders `fullname == seaweedfs`, so it matches the
`componentName`/`fullname` siblings in practice). FU6 — fork PR build
push is handled by the existing mirror-to-same-repo-branch workflow.

## Relates to

#2946 #2944 #2943 #2874 #2938 #2939 #2928 #2873 — if this lands, those
can be closed; otherwise they remain the granular per-PR review path.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added CI build cache warming for main branch builds.
* Added a best-effort in-cluster Cilium endpoint self-heal watchdog for
e2e installs.
* Extended SeaweedFS COSI with object-lock support plus separate
readonly access.

* **Improvements**
* Reduced e2e install flakiness with a shared readiness deadline, safer
waits, and improved cleanup/diagnostics capture.
* Prevented install-time churn by switching VPA update mode to
**Initial**.
  * Added container startup probes and updated SeaweedFS volume sizing.

* **Tests**
* Improved e2e robustness (etcd backup schedule waiting, digest-pinned
image checks, and corrected S3 port-forwarding).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-22 09:19:22 +05:00
Myasnikov DaniilandClaude Opus 4.8 d3958c5bcb refactor(build): mode=max registry cache with a main-only warmer (#2938)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-18 10:40:05 +05:00
Andrei KvapilandClaude 7f994f2d75 fix(platform): avoid SIGPIPE in migration 44 drain check
head -n 1 closing the pipe early under pipefail aborts the migration
once the HelmRelease name list outgrows the pipe buffer; count with
wc -l instead.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-06-11 13:56:42 +03:00
Aleksei Sviridkin 6e219c241b feat(platform,gateway): distribute operator wildcard secret name
Add publishing.certificates.wildcardSecretName to the platform chart
and propagate it through the cozystack-values channel as
_cluster.wildcard-secret-name. Only the Secret NAME travels — the
cert/key material is never broadcast, unlike the public kube-root-ca
that already rides this channel.

When the key is set, the gateway chart switches the rendered
TenantGateway to certMode=existingSecret with a wildcardSecretRef
pointing at the operator-supplied Secret, and skips the ACME
solver/provider/issuer validations that no longer apply. Wildcard mode
takes precedence over solver=dns01. TLS-passthrough services and
attached namespaces continue to render unchanged.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-10 03:08:07 +03:00
Aleksei Sviridkin 1d747b3d8d feat(platform): auto-wire KubeVirt permittedHostDevices and HostDevices feature gate when gpu-operator is enabled (#2768)
Closes #2765.

## What this PR does

Makes the passthrough and vGPU variants of `cozystack.gpu-operator`
produce a KubeVirt CR that is immediately usable, without an out-of-band
`kubectl patch`. Today the gpu-operator stack reports green,
`nvidia.com/<model>` shows up under node `.status.allocatable`, but
every `VirtualMachine` referencing the resource via
`domain.devices.hostDevices` is rejected by the admission webhook
because `HostDevices` is not in `developerConfiguration.featureGates`
and the resource is not listed under `permittedHostDevices`. That manual
reconciliation is now driven by the platform.

The change has three parts.

`packages/system/kubevirt` now parameterizes the KubeVirt CR template.
`developerConfiguration.featureGates` keeps its existing static defaults
and appends `.Values.extraFeatureGates`;
`spec.configuration.permittedHostDevices` renders verbatim from
`.Values.permittedHostDevices` when set;
`spec.configuration.mediatedDevicesConfiguration` does the same.
Defaults stay empty so the rendered CR is byte-identical to the previous
template when no overrides are supplied. The chart's `make update`
target re-applies the four sed transforms after fetching the upstream
CR, so refreshing the KubeVirt release version stays a single command.

`packages/core/platform` adds two defaults tables under `files/`.
`gpu-passthrough-defaults.yaml` covers Hopper (H100/H200), Ada Lovelace
(L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10), Turing (T4) and
Volta (V100/V100S); PCI vendor:device pairs are stable, `resourceName`
slugs follow the `<arch>_<model>_<form>_<mem>` convention
`nvidia-sandbox-device-plugin` generates as of v25.x, and
`externalResourceProvider: true` is set everywhere because those
resources are advertised by the sandbox plugin rather than KubeVirt's
in-tree device plugin. `gpu-vgpu-defaults.yaml` covers the Pascal–Ampere
mdev profile range (A100-40C/80C, A40-24Q/48Q, A30-24C, A10-24Q,
V100D-32C, T4-16Q) — the same family span the upstream
`vgpu-device-manager` walks `/sys/class/mdev_bus/` for. Ada Lovelace and
Blackwell SR-IOV vGPU is intentionally out of scope for the default
table and is expected to be advertised via the user-override hook
described below.

`templates/bundles/iaas.yaml` now selects between the two tables via a
new `bundles.iaas.gpuOperatorVariant` knob (`default` for passthrough,
`vgpu` for mediated; default is passthrough). The same variant flows
through to the gpu-operator Package CR, replacing the previous
always-default call. When the package is enabled, the platform injects
`HostDevices` into `extraFeatureGates` plus the chosen defaults into
`permittedHostDevices` for the kubevirt component's values. Operators
extend or replace via top-level platform values:

```yaml
gpu:
  replaceDefaults: false              # append (default) vs. wipe-and-replace
  permittedHostDevices:
    pciHostDevices: []                # cluster-specific PCI passthrough entries
    mediatedDevices: []               # cluster-specific vGPU entries
  mediatedDevicesConfiguration: {}    # extra mediated-device types / per-node mappings
```

`docs/gpu-vgpu.md` is updated to drop the manual `kubectl patch` step
and describe the auto-wiring contract, the per-variant defaults, the
escape hatches, and the sandbox-device-plugin `resourceName` slug
convention so an operator whose node publishes a different name knows
where to override.

## Test coverage

`helm-unittest` covers eight branches in
`packages/core/platform/tests/bundles_gpu_kubevirt_wiring_test.yaml`:
gpu-operator disabled emits no extras; gpu-operator enabled at default
variant emits the HostDevices gate plus `pciHostDevices`; vgpu variant
switches to `mediatedDevices` plus `mediatedDevicesConfiguration`; user
overrides append by default; `replaceDefaults: true` wipes the platform
table; an unknown variant fails fast with a descriptive error.
`packages/system/kubevirt/tests/kubevirt_cr_test.yaml` pins the
empty-values render and the three populated shapes at the CR template
level. `make helm-unit-tests` passes locally on the full repo.

### Release note

```release-note
feat(platform): the KubeVirt CR is now auto-configured for GPU host-device passthrough (HostDevices feature gate plus permittedHostDevices populated from NVIDIA defaults) whenever cozystack.gpu-operator is in bundles.enabledPackages. Operators no longer need a manual kubectl patch to make VMs schedule against a GPU. The variant is selected via bundles.iaas.gpuOperatorVariant (default: passthrough; vgpu: mediated). Extend or replace the NVIDIA defaults via .gpu.permittedHostDevices and .gpu.replaceDefaults.

**Upgrade note:** the bundle now OWNS spec.configuration.permittedHostDevices on the KubeVirt CR. If you previously hand-edited it (kubectl edit kubevirt), the first reconcile after upgrade overwrites your manual entries with the rendered NVIDIA default table. Before upgrading, move any custom entries into .gpu.permittedHostDevices (set .gpu.replaceDefaults: true to keep only your list) and verify each resourceName against what your nodes advertise (kubectl describe node <node> | grep nvidia.com/) — the default table now carries the exact slug the sandbox device plugin generates, which upper-cases the card's PCI-IDs name and keeps the Tesla/GL tokens (e.g. a Tesla T4 is nvidia.com/TU104GL_TESLA_T4), but a different plugin build can publish a different string; a mismatch makes the admission webhook reject GPU VMs on their next restart or migration.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Bundle-managed GPU Operator with selectable passthrough vs vGPU
variants; platform now injects GPU settings into KubeVirt and forwards
vGPU manager overrides when enabled.
* Default platform mappings for PCI passthrough and vGPU mediated-device
resources added.

* **Documentation**
* Expanded GPU/vGPU deployment guide with bundle-managed vs manual
guidance, wiring/precedence rules, resource-name conventions, and
verification steps.

* **Tests**
* New and expanded tests for bundle→KubeVirt wiring, variant behaviors,
default/override merging, and idempotent template parameterization.

* **Chores**
* Added Helm test target and hardened KubeVirt template
update/idempotency checks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-10 02:02:01 +03:00
Aleksei Sviridkin b6108039b7 feat(gpu-operator): add container variant for preinstalled host driver (#2766)
## What this PR does

Add a third architectural variant to `cozystack.gpu-operator` for hosts
where the NVIDIA driver and `nvidia-container-toolkit` are already
installed by the distro package manager (the typical Ubuntu/Debian GPU
node shape: apt-installed driver + apt-installed CT).

The existing variants do not produce a working setup in that shape.
`default` (passthrough) runs `vfioManager`, which unbinds the host
driver to bind `vfio-pci` — with the host driver active,
`k8s-driver-manager` early-exits and the bind never happens, so the GPU
never reaches a pod. `vgpu` requires the proprietary NVIDIA vGPU host
driver and a license server; the user-installed open or proprietary
driver does not satisfy it.

The new `container` variant maps onto upstream gpu-operator's third mode
(`sandboxWorkloads.enabled=false` + `devicePlugin.enabled=true`) with
every component that would conflict with the host install switched off:

- `driver.enabled=false` — use the host driver.
- `toolkit.enabled=false` — host already has `nvidia-container-toolkit`.
Enabling the operator's toolkit DaemonSet would overwrite the
`/etc/containerd/config.toml` the host package configured (via
`nvidia-ctk runtime configure`), breaking the host runtime wiring.
Matches NVIDIA's documented guidance and the existing
`examples/values-native-talos.yaml` reference, which pins the same flag
for the same reason.
- `devicePlugin.enabled=true` — publish `nvidia.com/gpu` to the kubelet.
- `vfioManager.enabled=false` — do not unbind the host driver. Upstream
renders this `true` unconditionally; it only stays at zero pods because
no node carries a `vm-passthrough` sandbox assignment when
`sandboxWorkloads` is off — pinned off so the rendered policy matches
intent.
- `cdi.enabled=false` — upstream defaults this `true`, but CDI spec
generation is serviced by the operator-managed toolkit DaemonSet this
variant disables. An apt host doesn't auto-generate CDI specs
(`nvidia-ctk cdi generate` is a manual step) and uses the classic
`nvidia-container-runtime` hook, so leaving CDI on would point the
device plugin at CDI injection with no specs on the host and silently
break allocation.
- `sandboxWorkloads.enabled=false` — regular pod workloads, not VMs.
With this off, upstream `defaultWorkload` is already `container`, so no
override is needed.

Host prerequisite: because the operator's toolkit component is disabled,
the node's containerd must already have the `nvidia` runtime registered
(`nvidia-ctk runtime configure --runtime=containerd`); `apt install
nvidia-container-toolkit` installs binaries only and does not wire
containerd. With `driver.enabled=false` the operator uses the
pre-installed host driver at its standard location, so a stock apt
install needs no `hostPaths.driverInstallDir` override (Talos lands the
driver under a non-standard prefix — see
`examples/values-native-talos.yaml`).

The bundle wiring (`packages/core/platform/templates/bundles/iaas.yaml`)
already enables `cozystack.gpu-operator` as an optional package — no
bundle changes required. Users opt into this variant by setting
`variant: container` in their `Package` CR.

Verified locally with `helm template` against `values.yaml` +
`values-container.yaml`: ClusterPolicy renders with
`driver.enabled=false`, `toolkit.enabled=false`,
`devicePlugin.enabled=true`, `vfioManager.enabled=false`,
`cdi.enabled=false`, `sandboxWorkloads.enabled=false`. Platform `helm
unittest` and the `gpu-operator` variant bats suite are green. Not yet
smoke-tested on a real Ubuntu GPU node — the config matches the
documented apt-host wiring (legacy `nvidia-container-runtime` hook, CDI
off), but an end-to-end CUDA-pod run on hardware is still owed.

Closes #2764

### Release note

```release-note
feat(gpu-operator): add `container` variant for hosts with a pre-installed NVIDIA driver and nvidia-container-toolkit, exposing containerized GPU workloads via the standard NVIDIA device plugin without VFIO passthrough or vGPU.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a "container" GPU operator variant to support containerized GPU
workloads on hosts with the NVIDIA driver and nvidia-container-toolkit.

* **Documentation**
* Updated variants documentation and OS support table to include the
container variant and per-host requirements (including Ubuntu and Talos
notes).
  * Clarified Talos native pod example wording.

* **Tests**
* Added regression tests to verify container-variant configuration and
component enablement/disablement.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-10 02:00:59 +03:00
Aleksei Sviridkin 931f286b9f fix(ingress,platform): deliver publishing.proxyProtocol to host ingress-nginx (#2799)
## What this PR does

`publishing.proxyProtocol: true` is documented to enable PROXY-protocol
on the host ingress-nginx, but the three nginx config keys
(`use-proxy-protocol`, `real-ip-header`, `enable-real-ip`) never reached
the controller. They were written into the `ingress-system` component
values of the `cozystack.ingress-application` Package, but that
component has no `install:` block, so the package controller never
builds a HelmRelease for it and the values are dropped. External traffic
behind a PROXY-protocol L4 load balancer kept breaking.

This routes the switch through `_cluster.proxy-protocol` in the
`cozystack-values` Secret — the same path `expose-external-ips` already
travels to reach the host ingress release — and the `extra/ingress`
chart applies it to the external-facing ingress (the namespace named by
`expose-ingress`). The dead component-values injection is removed.

The `extra/ingress` chart also gains a first-class `proxyProtocol`
value, so PROXY-protocol can be enabled per Ingress application (a
per-app boolean knob like `cloudflareProxy`).

Two `fail` guards keep the new knobs safe: the per-app `proxyProtocol`
value is rejected on the external-facing (host) ingress — that path must
go through `publishing.proxyProtocol`, which also installs ouroboros
(hairpin handling) and arms the disable-hazard guard — and
`cloudflareProxy` + `proxyProtocol` together are refused, since
Cloudflare's HTTP proxy does not speak PROXY-protocol to origins and its
`real_ip_header` server-snippet would override `real-ip-header:
proxy_protocol`.

`use-forwarded-headers` / `compute-full-forwarded-for` are deliberately
not set: without a paired `proxy-real-ip-cidr` they would let clients
spoof `X-Forwarded-For`, and the PROXY-protocol frame is already the
source of truth for the client address.

helm-unittest coverage pins runtime delivery on the host path, the
per-app path, and that non-host tenants stay untouched; the previous
test that only checked template emission into the (non-consumed) Package
component is replaced.

Affected versions: v1.4.0–v1.4.2 and current main (introduced in #2561).

**Upgrade note (action required).** Because the flag was a silent no-op
before this fix, a v1.4.x cluster that set `publishing.proxyProtocol:
true` aspirationally — without its L4 load balancer actually injecting
PROXY frames — kept working by accident. After this fix the flag really
enables `use-proxy-protocol`, which makes ingress-nginx require a PROXY
frame on every connection; if the LB does not send one, all external
ingress breaks. Before upgrading, confirm the upstream LB injects
PROXY-protocol v1 frames, or unset `publishing.proxyProtocol`.

Closes #2796

### Release note

```release-note
fix(ingress): publishing.proxyProtocol now enables PROXY-protocol on the host ingress-nginx (the config was silently dropped before reaching the controller); the Ingress application also gains a proxyProtocol value to enable PROXY-protocol per-app. ACTION REQUIRED on upgrade: the flag was previously a no-op, so a cluster that set it without the upstream LB injecting PROXY-protocol v1 frames will lose all external ingress once it takes effect — verify the LB, or unset publishing.proxyProtocol before upgrading.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Tenant ingress support for PROXY-protocol via a new proxyProtocol
option (default: false).
* Platform-level publishing.proxyProtocol now controls external/host
ingress behavior.

* **Bug Fixes / Validation**
* Rejects enabling per-app proxyProtocol on external-facing host
ingresses.
* Rejects using proxyProtocol together with cloudflareProxy to prevent
conflicting real‑IP handling.

* **Documentation**
* Updated docs describing proxyProtocol usage, defaults, and safety
requirements.

* **Tests**
* Added tests covering proxyProtocol wiring, config emission, and
failure cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-09 17:04:14 +03:00
Aleksei Sviridkin 479929ebfb ci(e2e): drop 3x retry on Run E2E + Install Cozystack, wait out gateway.bats tenant teardown (#2558)
## What this PR does

Drops the 3× retry loop on `Run E2E tests` and `Install Cozystack into
sandbox`. `Prepare environment` keeps its 3× retry — that step is pure
infrastructure (Talos image download, sandbox VM boot, network) where
transient runner hiccups warrant a retry.

On failure, the test step now captures `kubectl get hr -A -o wide` and
`kubectl get events -A` under a collapsible group so triage starts with
the actual broken-state snapshot.

> [!NOTE]
> An earlier revision of this PR also doubled every bats timeout. That
commit was dropped in a rebase and is intentionally **not restored**:
the timeout class that actually matters (per-app HR-Ready waits) has
since been standardized at 5m on `main` (7b9f286fa), making a blanket 2×
redundant.

**Fixes gateway.bats teardown leakage.** The nested-tenant tests deleted
tenants fire-and-forget, parent and child back-to-back. The leftover
uninstalls (each blocked on a cleanup Job, parents wedged on
still-terminating child namespaces) plus one mid-install child HR
occupied exactly 5 workers on the `--concurrent=5` tenants
helm-controller shard, starving whichever app test ran next — observed
as the harbor HR sitting unreconciled for its whole 5m HR-Ready budget
in [run
27020081550](https://github.com/cozystack/cozystack/actions/runs/27020081550),
surfaced by this PR's own retry removal + diagnostics dump. Teardown now
deletes child→parent with hard `wait hr --for=delete` between, so a
wedged tenant uninstall fails gateway.bats itself, not an innocent
neighbor.

## Why

Audit of 30 successful PR runs found that across 5 sampled failure
attempts, **25/25 retries** on `Run E2E tests` failed — the retry loop
never recovered a flake, only stretched deterministic failures and
tripled diagnostic wall-time. Same data shape on `Install Cozystack`.

Beyond wasted CI time, the retry was hiding ~10 deterministic bugs (Helm
namespace-ownership conflict, seaweedfs HR timeout, harbor BucketInfo
wiring, vminstance disk race, etc.). Each failure looked like a "flake"
because the retry sometimes coincided with whatever transient state had
cleared — the retry never fixed the bug, just delayed surfacing.

## Dependencies

The deterministic bugs the retry was masking are now fixed on `main`:
- ✅ **#2508** — installer namespace bootstrap (Helm namespace-ownership
conflict on cold install) — merged
- ✅ **#2509** — operator HelmRelease config knobs (`seaweedfs-system`
2-min wait race within Flux's 5-min reconcile windows) — merged
- ✅ **#2528** — harbor bucket-secret + BucketInfo gating (harbor
`ValuesError` on first install) — merged
- ✅ **#2529** — objectstorage-controller BucketAccess conflict retry —
merged

Companion PRs in the #2619 split (independent of this PR,
ordering-wise):
- **#2602** — Flux v2.8.0 + chart fixes
- **#2601** — seaweedfs-system split

This PR does NOT depend on #2602/#2601 — it now touches only the
workflow file and gateway.bats teardown, both on top of fresh `main`.

Surfaced from #2500.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* CI prepare-environment step now reports plain attempt counts with
clear success/failure messages.
* Install and per-app test steps no longer retry; each runs once and
fails immediately on error. Failed apps log diagnostics and job proceeds
to remaining apps while overall job fails.
* **Tests**
* End-to-end tests and install/prepare flows use longer, more tolerant
timeouts and added existence polling to reduce flakiness and improve
diagnostics.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2558?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-09 16:21:15 +03:00
Aleksei Sviridkin c32812ccf9 fix(platform): deliver publishing.proxyProtocol to host ingress-nginx
The host PROXY-protocol switch wrote the nginx config keys into the
ingress-system component values of the cozystack.ingress-application
Package. That component has no install block, so the package controller
never built a HelmRelease for it and the values were dropped — the host
ingress-nginx never received use-proxy-protocol and external traffic
behind a PROXY-protocol L4 load balancer kept breaking.

Route the switch through _cluster.proxy-protocol in the cozystack-values
Secret instead, the same path expose-external-ips travels to reach the
host ingress release, and drop the dead component-values injection.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-09 12:50:53 +03:00
Myasnikov Daniil 9868c43e53 fix(build): read --cache-from from ghcr :latest so PR builds warm-start
PR #2711 switched PR builds to push only :pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to stop concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry (iad.ocir.io) and never publish :latest there, so every PR build's registry cache lookup 404s and rebuilds cold.

Add a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and point every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so the #2711 anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-09 14:24:36 +05:00
Aleksei Sviridkin 117d317fd8 fix(platform): drop fragile numeric vGPU mediatedDeviceTypes defaults
The vgpu defaults shipped a mediatedDevicesConfiguration block whose
mediatedDeviceTypes were hardcoded nvidia-NNN ids. Those ids are not
stable identifiers — they are the per-(SKU, driver) indices under which a
host enumerates mdev types in /sys/.../mdev_supported_types/, so the same
profile gets a different number on a different card or driver branch.
Auto-injecting them made KubeVirt try to create mdev types that usually
do not exist on the operator's nodes, and duplicated work the GPU
Operator's vGPU Device Manager already does.

Delegate mdev creation to the vGPU Device Manager, which is name-based and
stable, and keep only the name-based permittedHostDevices.mediatedDevices
exposure list (mdevNameSelector) in the defaults. KubeVirt-driven creation
stays available through the .gpu.mediatedDevicesConfiguration override for
operators who supply host-verified type ids; document the lookup recipe.

Update the bundle wiring, the gpu/vgpu values comments, the vGPU guide and
the tests accordingly (the default vgpu path now emits no
mediatedDevicesConfiguration; the override path is covered explicitly).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-08 18:14:59 +03:00
Aleksei Sviridkin 919073601d fix(platform): match NVIDIA passthrough defaults to the resourceName the plugin advertises
The auto-injected permittedHostDevices only works if each resourceName
equals what nvidia-sandbox-device-plugin advertises on the node; otherwise
the admission webhook rejects the VM — the very manual step this bundle
removes. Several defaults did not match the slug the plugin
(nvcr.io/nvidia/kubevirt-gpu-device-plugin) derives from the PCI-IDs
database name, which uppercases the name and keeps the die suffix, the
Tesla brand and the form-factor/memory tokens:

  10DE:1EB8  TU104GL_T4          -> TU104GL_TESLA_T4
  10DE:20B7  GA100_A30           -> GA100GL_A30_PCIE
  10DE:2331  GH100_H100_PCIE_80GB-> GH100_H100_PCIE
  10DE:2321  GH100_H100_NVL_94GB -> GH100_H100L_94GB
  10DE:1DB1/1DB4/1DF6 + new 1DB5 -> GV100GL_TESLA_V100_* family

Device 10DE:1DB1 was also mislabelled: it is the SXM2 16GB part, not the
32GB one — the 32GB SXM2 V100 is a distinct device (10DE:1DB5) that was
missing entirely. Relabel 1DB1 as 16GB and add 1DB5.

Restate the header comment as the actual generation rule and pin the
corrected slugs (T4, A30, both V100 SXM2 parts) with a regression test
that also asserts the tidy/short forms never reappear.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-08 18:06:31 +03:00
Aleksei Sviridkin da0dcf9edf test(platform): pin nil-safe GPU bundle wiring against null gpu values
The .Values.gpu accesses in the iaas bundle are fully parenthesized
((( .Values.gpu ).x).y) so the field walk stays nil-safe when an operator
nulls the whole gpu block or leaves a sub-key blank (an empty YAML key
parses to null, which Helm then drops from .Values). A bare dotted walk
would panic with "nil pointer evaluating interface {}.<field>" on the
first nil hop.

Add regression cases covering gpu: null in both passthrough and vgpu
variants plus blank sub-keys, and a comment warning against flattening
the parenthesized form. A mutation check confirms the cases fail with the
nil-pointer panic the moment the parentheses are removed.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-08 17:08:21 +03:00
Aleksei Sviridkin 91efa1f4eb feat(platform): auto-wire KubeVirt host-device support when gpu-operator is enabled
When cozystack.gpu-operator is in bundles.enabledPackages, the platform
now mirrors the chosen GPU variant into the KubeVirt CR — the HostDevices
feature gate is appended to developerConfiguration.featureGates, and a
starter permittedHostDevices block is filled from a per-variant defaults
table. Without this, a passthrough-variant GPU shows up under
.status.allocatable on the node but the KubeVirt admission webhook
rejects every VM that references it as a hostDevices.deviceName.

Two defaults tables ship under packages/core/platform/files/:

  - gpu-passthrough-defaults.yaml covers Hopper (H100/H200), Ada
    Lovelace (L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10),
    Turing (T4), Volta (V100/V100S). PCI vendor:device pairs are
    stable; resourceName slugs mirror nvidia-sandbox-device-plugin's
    <arch>_<model>_<form>_<mem> convention. externalResourceProvider
    is set on every entry because the resources are advertised by
    the sandbox-device-plugin DaemonSet, not by KubeVirt itself.

  - gpu-vgpu-defaults.yaml covers the Pascal-Ampere mdev profile
    range the upstream vgpu-device-manager walks /sys/class/mdev_bus/
    for. Ada Lovelace and Blackwell SR-IOV vGPU is out of scope for
    the chart's default list (NVIDIA driver path differs) and is
    expected to be advertised via the user-override hook below.

bundles.iaas.gpuOperatorVariant picks which table applies (default =
passthrough, vgpu = mediated). The same variant flows through to the
gpu-operator Package CR, replacing the previous always-default call.

Operators extend or replace the defaults via .gpu in platform values:

  gpu:
    replaceDefaults: false
    permittedHostDevices:
      pciHostDevices: [...]
      mediatedDevices: [...]
    mediatedDevicesConfiguration: {...}

replaceDefaults: false (the default) appends the user list to the
NVIDIA table; true drops the table entirely for clusters with only
non-NVIDIA accelerators or a strict allowlist requirement.

Companion helm-unittest suite covers the eight branches: no gpu
package emits no extras; gpu-operator at default variant emits the
HostDevices gate plus pciHostDevices; vgpu variant emits the
HostDevices gate plus mediatedDevices and mediatedDevicesConfiguration;
user overrides append by default; replaceDefaults wipes; unknown
variant fails fast.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-08 17:01:51 +03:00
Aleksei Sviridkin 9352d3ee0e feat(gpu-operator): add container variant for preinstalled host driver
The two existing variants assume the host has neither the NVIDIA driver
nor the container toolkit: default unbinds the host driver to bind
vfio-pci for KubeVirt VMs, vgpu requires the proprietary vGPU host
driver and a license server. Neither path produces a working setup on a
node that already ships the driver and nvidia-container-toolkit via the
distro package manager — the most common shape of a Linux GPU node.

The new container variant surfaces the third architectural mode that
upstream NVIDIA gpu-operator already supports: sandboxWorkloads
disabled, devicePlugin enabled, with every component that would
conflict with the host install switched off.

- driver.enabled=false        use the host driver
- toolkit.enabled=false       host already has nvidia-container-toolkit;
                              enabling overwrites the containerd
                              config.toml the host configured via
                              nvidia-ctk runtime configure
- devicePlugin.enabled=true   publish nvidia.com/gpu to the kubelet
- vfioManager.enabled=false   do not unbind the host driver
- cdi.enabled=false           CDI spec generation is serviced by the
                              disabled toolkit; an apt host uses the
                              legacy nvidia-container-runtime hook, so
                              pin off to match the host runtime wiring
- sandboxWorkloads.enabled=false   regular pod workloads, not VMs

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-08 16:58:51 +03:00
Aleksei Sviridkin 80fbfa1ed8 fix(networking): point host ouroboros proxy at the root-tenant ingress (#2800)
## What this PR does

With `publishing.proxyProtocol: true`, the host emits a
`cozystack.ouroboros` Package using the no-override Package macro, so
the proxy inherits the wrapper chart default backend
`ingress-nginx-controller.cozy-ingress-nginx`. That layout describes a
managed Kubernetes tenant cluster, not the host: on the host,
`extra/ingress` deploys ingress-nginx into the root tenant namespace as
`root-ingress-controller` in `tenant-root`. The composed backend FQDN
never resolves, so `ouroboros-proxy` never reaches Ready.

This emits the host ouroboros Package with a `proxy.target` override
derived from `publishing.ingressName` — namespace plus
`<short>-ingress-controller` service name — mirroring the `trimPrefix
"tenant-"` transform `extra/ingress` applies to its release namespace.
The derivation reuses the existing platform invariant that
`publishing.ingressName` is the namespace where the host ingress
controller runs.

Adds bundle helm-unittest coverage for the default ingress name and a
non-default one, and corrects the misleading wrapper `values.yaml`
comment that claimed the host uses `cozy-ingress-nginx`.

Scope: this fixes the ouroboros readiness defect only. The separate
defect where the injected nginx PROXY-protocol config keys are not
consumed by the host ingress is tracked independently and not touched
here.

Closes #2797

### Release note

```release-note
fix(networking): point the host ouroboros PROXY-protocol proxy at the root-tenant ingress so it becomes Ready when publishing.proxyProtocol is enabled
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Enhanced proxy protocol configuration to automatically route through
the appropriate ingress-nginx service based on your ingress name
settings.

* **Tests**
* Added test coverage for proxy protocol bundle wiring and ingress
service routing.

* **Documentation**
* Clarified proxy target configuration behavior and fallback mechanisms.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-08 16:41:15 +03:00