Commit Graph
4489 Commits
Author SHA1 Message Date
Andrei KvapilandClaude 844deb170f feat(controller): enforce recursive tenant quotas at runtime
Add a tenant quota reconciler to cozystack-controller that realises the
hierarchical (sub-tree) quota model at runtime. Each tenant's declared
quota is the budget for its whole sub-tree: a child that declares its own
quota carves a fixed slice out of the parent budget, while a child without
a quota shares the parent's remaining pool.

The controller maintains a per-namespace 'tenant-quota-allocated'
ResourceQuota (Kubernetes enforces the most restrictive quota, so it binds
without fighting Flux over the chart-rendered 'tenant-quota'), aggregating
sub-tree usage so a pool's members collectively stay within the budget. A
--tenant-quota-buffer-percent flag temporarily inflates pool budgets so
workloads already over a freshly-introduced quota keep running during
rollout; overcommitted pools emit a QuotaOvercommitted event.

The model and the selector-spans-namespaces/aggregated-usage approach are
adapted from OpenShift's ClusterResourceQuota controllers (openshift/api,
library-go, cluster-policy-controller, apiserver-library-go), Apache-2.0,
Copyright Red Hat, Inc. and the OpenShift Authors.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-06-30 16:12:17 +02:00
Andrei KvapilandClaude c87612b538 feat(apiserver): enforce hierarchical tenant quota allocation at admission
Add a declaration-time gate in the aggregated apiserver: a child tenant's
declared resourceQuotas are carved out of its parent tenant's remaining
quota and may not exceed it. A child without a quota shares the parent
pool and is allowed; a parent without a quota does not constrain children.

This blocks quota escalation via sub-tenant creation (declaring a bigger
or unbounded child) without re-implementing pod-level admission.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-06-30 16:11:17 +02:00
Aleksei Sviridkin eab9b034f6 fix(seaweedfs): enforce read-only bucket access by bumping cosi-driver to v0.3.1 (#3136)
## What this PR does

The SeaweedFS COSI driver shipped at `v0.1.2` hardcodes read-write S3
actions (`Read,Write,List,Tagging`) for every BucketAccess and never
reads the `accessPolicy` BucketAccessClass parameter. The chart already
defines a `-readonly` BucketAccessClass carrying
`parameters.accessPolicy: readonly`, and the `bucket` app routes
`readonly: true` users to it — but `v0.1.2` ignores the parameter, so a
credential meant to be read-only is silently issued with full read-write
access and can upload and delete objects. This is an access-control bug.

Upstream `seaweedfs/seaweedfs-cosi-driver` added `accessPolicy` handling
in `v0.3.0`: `readonly` maps to `Read,List`, while `readwrite` and the
empty default stay `Read,Write,List,Tagging`. This bumps the driver to
`v0.3.1` via a `seaweedfs.cosi.image` override in the package
`values.yaml`, so the override survives `make update` and the vendored
chart is left untouched. The default read-write action set is unchanged,
so existing read-write credentials are unaffected; only read-only grants
are tightened to actually be read-only.

The `extra/seaweedfs` chart already runs `v0.3.1` with the same
objectstorage-sidecar, so the driver/sidecar combination is already
proven in this repo. A chart unit test pins the rendered driver image so
this fix cannot silently regress on a future re-vendor.

This does not close #1921. That issue bundles two features — read-only
bucket access (fixed here) and Object Lock / WORM (the `-lock`
BucketClass), which is independent and out of scope here. Relates to
#1921.

### Release note

```release-note
Fix SeaweedFS read-only bucket credentials being issued with read-write access; the COSI driver now honours the read-only access policy.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Pin the SeaweedFS COSI driver image to
`ghcr.io/seaweedfs/seaweedfs-cosi-driver:v0.3.1` to keep access behavior
consistent and avoid relying on a vendored default.
* Ensure the rendered SeaweedFS COSI deployment uses the expected
container name and the pinned driver image.
* **Tests**
* Add a YAML test suite to validate the generated COSI deployment
template matches the expected container name and pinned image.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 16:04:00 +03:00
scooby87 bef42229d4 fix(harbor): clean up orphaned jobservice PVC and ACME solver resources on delete (#3061) (#3077)
## What this PR does

After deleting a `Harbor` app CR, two things were left orphaned in the
tenant
namespace:

- the **jobservice / trivy PVCs** (`<release>-jobservice`, and
`<release>-trivy`
when Trivy is enabled), because
`packages/apps/harbor/templates/harbor.yaml`
sets `persistence.resourcePolicy: "keep"`, which annotates the upstream
Harbor
subchart's PVCs with `helm.sh/resource-policy: keep` so Helm/Flux
deliberately
  never deletes them;
- cert-manager's **ACME http01-solver** resources
(`cm-acme-http-solver-*`), via
the same stuck-challenge mechanism as the Bucket issue (#3060) when
Harbor
  issues a non-wildcard cert.

This adds a single `post-delete` cleanup hook that:

- **deletes the upstream Harbor subchart's kept PVCs** — selecting by
`app.kubernetes.io/instance=<release>-system` (the subchart runs as the
  `<release>-system` HelmRelease, so its PVCs carry that label, **not**
`<release>`). A hook is required precisely because `resourcePolicy:
keep`
blocks normal reclaim. This intentionally covers both the jobservice and
trivy
  PVCs: both hold only regenerable data (jobservice file logs; trivy's
  vulnerability-DB cache), and registry images live in S3.
- **deletes this release's `Certificate`** (deterministic name
`<release>-ingress-tls`); cert-manager then cascade-deletes the `Order`,
`Challenge` and solver `Pod`/`Service`/`Ingress` objects it owns via
owner
  references.
- **host-scoped belt-and-braces sweep** for any ACME http01 solver
objects
cert-manager may have orphaned (e.g. the Certificate was already gone,
so
deleting it did not cascade). The sweep is scoped **strictly to this
release's
Harbor host** — Challenges filtered by `.spec.dnsName`, solver Ingresses
by
  `.spec.rules[0].host` — **not** a namespace-wide
`acme.cert-manager.io/http01-solver=true` label sweep, because a
Cozystack
tenant namespace can host many apps and a blanket label sweep would
delete
  sibling apps' in-flight challenge solvers.

The Job is hardened: non-root (uid 65534), read-only root fs, all caps
dropped,
`seccompProfile: RuntimeDefault`, with CPU/memory requests and limits
set. RBAC
is scoped to exactly the resources it deletes (`persistentvolumeclaims`,
`certificates`, `orders`, `challenges`, solver `ingresses`) and verified
against
the vendored cert-manager CRDs. Failures are logged but never block
teardown.

## Cleanup scope — what is and isn't removed

**Removed:** the upstream Harbor subchart's kept PVCs (jobservice +
trivy) and
this release's cert-manager ACME artifacts (Certificate, and host-scoped
Order/Challenge/solver-Ingress).

**Not touched (by design):**

- **CNPG database PVCs** (`<release>-db-N`) — owned by the CNPG
`Cluster` and
  governed by CNPG's own reclaim policy; they carry CNPG labels, not
`app.kubernetes.io/instance=<release>-system`. Re-installing Harbor with
a
different DB size into the same namespace can therefore leak the old DB
PVCs;
  this is left to CNPG's lifecycle on purpose (the data is the user's).
- **Redis PVCs** — managed by the redis-operator with their own labels,
likewise
  untouched.
- **Solver `Pod`/`Service`** — not deleted directly; they are removed by
cert-manager's owner-ref cascade when the Certificate/Challenge is
deleted, so
  the hook grants no RBAC for them.

> [!NOTE]
> The PVC sweep is eventually-consistent: while the inner
`<release>-system`
> HelmRelease is still being uninstalled by Flux, a PVC may briefly stay
in
> `Terminating` until its consumer Pods (jobservice/trivy) exit. The
Job's
> deletion request always succeeds; reclaim completes once the pods are
gone.

> [!WARNING]
> **This overrides the intentional `persistence.resourcePolicy: "keep"`
setting
> and force-deletes the jobservice/trivy PVCs on app deletion.** Those
PVCs hold
> only jobservice file logs and the trivy vuln-DB cache (registry images
live in
> S3), so data-loss risk is low — but if retaining them across
re-installs is
> desired, this is a conscious product decision to revisit. The ACME
cleanup
> affects only cert-manager's transient artifacts.

Fixes #3061.

> Follow-up tracked in #3088: digest-pin the cleanup `kubectl` image and
route it
> through `cozy-lib.images-registry` (pre-existing debt shared with the
MariaDB
> cleanup hook), out of scope for this PR.

### Release note

```release-note
fix(harbor): clean up the orphaned jobservice/trivy PVCs and cert-manager ACME http01-solver resources left behind after a Harbor app is deleted. NOTE: this force-deletes the jobservice/trivy PVCs, overriding persistence.resourcePolicy=keep (jobservice file logs and trivy vuln-DB cache only; registry images are in S3). CNPG database PVCs are not touched.
```
2026-06-30 15:55:52 +03:00
myasnikovdaniil 3e20a0cd41 fix(kubernetes): make node group resources optional when instanceType is set (#3121)
## What this PR does

A tenant Kubernetes node group can be sized two ways: by `instanceType`
or by an explicit `resources` block. The generated values schema,
however, marked `resources` as required on every node group, so a node
group sized purely by `instanceType` was rejected at install time:

```text
at '/nodeGroups/<name>': missing property 'resources'
```

The same failure occurs through the typed `Kubernetes` CR: an empty
`resources: {}` is dropped during the CR-to-HelmRelease translation, so
by validation time the key is absent and the required check fails — even
though sizing was already fully specified by `instanceType`.

This marks `resources` optional in the node group definition and
regenerates the dependent artifacts (`values.schema.json`, API types,
README, and the `ApplicationDefinition`). No change is needed to emit
the VM: the chart already renders `domain.cpu`/`domain.memory` only when
`resources` is present and `instancetype` only when set, so a node group
sized purely by `instanceType` now validates and installs. Node groups
that specify `resources` are unaffected.

`instanceType` and `resources` are alternative, mutually exclusive
sizing sources — KubeVirt rejects a VirtualMachine that references an
instancetype and also sets explicit `domain.cpu`/`domain.memory`, since
instancetype CPU/memory cannot be overridden. To avoid emitting a VM
that the instancetype admission webhook would later reject, the chart
now fails fast at render time with a clear message when a node group
sets both a non-empty `instanceType` and explicit `resources`.

Tests: helm-unittest coverage proves a node group with `instanceType`
and no `resources` passes schema validation and renders without explicit
`domain.cpu`/`memory`; a complementary case confirms explicit
`resources` are still honoured; and a `failedTemplate` case pins the
render-time rejection of the conflicting both-set combination. The
omit-resources case fails against the pre-fix schema with the exact
error above, so it is a real regression guard.

Addresses #3046.

### Screenshots

N/A — schema and chart change, no UI impact.

### Release note

```release-note
fix(kubernetes): allow a node group to omit `resources` when it is sized by `instanceType` (previously failed schema validation with "missing property 'resources'"); reject the unsupported combination of `instanceType` and explicit `resources` at render time
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Worker node groups can optionally size CPU/memory via `resources`, and
can omit `resources` when using `instanceType`.
* **Bug Fixes**
* Render now validates CPU/memory rules: `instanceType` with only one of
`resources.cpu`/`resources.memory` fails, and `instancetype` is omitted
when both `resources.cpu` and `resources.memory` are set (with
`instanceType` ignored).
* **Documentation**
* Updated API/Helm docs and JSON schema to clarify `resources` vs
`instanceType` precedence and the “set both CPU and memory (or neither)”
requirement.
* **Tests**
* Added helm-unittest scenarios covering omission, precedence, and
render-time failure cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 17:46:54 +05:00
Aleksei Sviridkin 0892c5a38c fix(qdrant)!: clean up orphaned data PVC on delete (#3059) (#3075)
## What this PR does

After deleting a `Qdrant` app CR, the StatefulSet data PVC
(`qdrant-storage-<release>-0`) was left orphaned in the tenant
namespace,
consuming storage permanently. StatefulSet-templated PVCs are never
garbage-collected on StatefulSet deletion by default.

The vendored upstream Qdrant chart does **not** support
`persistentVolumeClaimRetentionPolicy` (verified: its `statefulset.yaml`
never
renders that field), so the OpenBAO-style value passthrough is not
possible
here. Instead this adds a `post-delete` PVC cleanup hook (modeled on the
MariaDB
hook) that deletes the release's PVCs by label.

The chart's volumeClaimTemplate only stamps `app: qdrant` (shared by
every
Qdrant in a namespace), so to select PVCs safely per release this also
passes
`persistence.additionalLabels.app.kubernetes.io/instance: <release>`
through the
HelmRelease — giving the data PVC a release-unique label the hook
selects on.

The cleanup Job is hardened: runs as non-root (uid 65534), read-only
root fs,
all capabilities dropped, `seccompProfile: RuntimeDefault`; RBAC scoped
to
`persistentvolumeclaims`. A real cleanup failure is logged (not silently
swallowed) but never blocks namespace teardown.

> [!WARNING]
> **Deleting a Qdrant app now permanently removes its data PVC and the
vector data it holds.**
> This is intentional and fixes the storage leak; the hook only fires on
full
> release deletion (not on scale-down). Back up before deleting.

Fixes #3059.

### Release note

```release-note
fix(qdrant): garbage-collect the StatefulSet data PVC when a Qdrant app is deleted, instead of leaving it orphaned. WARNING: deleting a Qdrant app now permanently removes its data PVC and the data it holds.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a post-delete cleanup hook for Qdrant to remove leftover
StatefulSet data PVCs after deletion.
* Included hardened Kubernetes settings for the cleanup job to improve
runtime safety.
* **Tests**
* Added Helm chart tests that validate the post-delete cleanup job and
its RBAC permissions.
* Enhanced end-to-end verification to confirm the data PVC is fully
removed after deleting a Qdrant instance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 14:53:11 +03:00
Aleksei Sviridkin bf14a73b39 feat(api): add SecurityGroup network policy resource (sdn.cozystack.io) (#2922)
## What this PR does

Introduces a tenant-facing `SecurityGroup` resource in a new
`sdn.cozystack.io/v1alpha1` API group, served by the Cozystack
aggregated API server as a 1:1 projection of a `CiliumNetworkPolicy`. A
SecurityGroup attaches to one of the tenant's managed applications by
reference (`spec.targetRef: {apiGroup, kind, name}`) and declares
allow-list ingress/egress for that application. Tenants manage network
policy for their own apps entirely through
`securitygroups.sdn.cozystack.io`, without being granted any access to
the `cilium.io` API group.

The translation is synchronous and stateless — the API server converts
each SecurityGroup into a CiliumNetworkPolicy in the same namespace on
write and back on read; there is no controller. The backing policy's
`endpointSelector` is **derived** from `targetRef` (matched on the
application's lineage labels
`apps.cozystack.io/application.{group,kind,name}`), never copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
Policies owned by this API carry an `sdn.cozystack.io/securitygroup`
marker label that the storage always re-asserts and hides from the view,
so it only ever surfaces or mutates its own policies and never touches
platform-managed network policies.

The value is self-service network policy on a Cozystack-native,
RBAC-scoped surface, without exposing Cilium's version-coupled CRDs to
tenants. Because the selector is derived from an authorized application
reference rather than free-form tenant input, the RBAC boundary is
structural, not selector-validated: a tenant cannot express network
policy against arbitrary or platform-owned pods. The full design —
selector derivation from lineage labels, the structural boundary, the
additive-policy reality (a SecurityGroup widens a managed application's
allowed traffic but cannot sever its platform-managed management plane,
and equally cannot tighten below the platform baseline), the in-tree
CiliumNetworkPolicy mirror that avoids a Cilium Go-module dependency,
RBAC, and deferred extensions (raw-pod targeting, cluster scope,
reusable groups) — is in `pkg/apis/sdn/DESIGN.md`.

Implements the attachment model from #1614.

### Release note

```release-note
feat(api): add the SecurityGroup resource (sdn.cozystack.io/v1alpha1) — a tenant-facing, namespace-scoped firewall that attaches to a managed application by reference and projects 1:1 onto a CiliumNetworkPolicy with a machine-derived endpoint selector, letting tenants manage their application's network policy without cilium.io access.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added the `SecurityGroup` API resource (`sdn.cozystack.io` v1alpha1)
with a namespaced projection backed by Cilium network policies.
* Enabled the SDN `APIService` and wired SDN routing, scheme, and REST
support.
* Expanded RBAC to include SecurityGroup access and required Cilium
policy permissions.
* **Documentation**
* Added a design document describing projection behavior and rule
semantics.
* **Bug Fixes**
  * Improved API rule validation coverage for missing list-type fields.
* **Tests**
* Added SDN round-trip, fuzzing, RBAC, and comprehensive REST
create/get/list/update/delete/watch/resource-version tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 14:30:23 +03:00
Алексей АртамоновandClaude Opus 4.8 b25f294d46 fix(harbor): select kept PVCs by release label so the trivy PVC is also cleaned
The post-delete cleanup hook selected PVCs by
app.kubernetes.io/instance=<release>-system, a label only the jobservice PVC
carries (via the chart's "harbor.labels" helper). The trivy PVC is a
StatefulSet volumeClaimTemplate labeled via "harbor.legacy.labels" with a
selector using "harbor.matchLabels" — neither sets
app.kubernetes.io/instance — so the sweep silently missed it. Trivy is enabled
by default, so a default Harbor delete left the trivy PVC orphaned,
contradicting the PR's own claim.

Select on release=<release>-system instead: both kept PVCs carry it, while
CNPG (db) and Redis PVCs use different label schemes and the registry is on S3,
so only the two intended PVCs match. The unit test now asserts the corrected
selector and guards against regressing to the instance label.

Also drop the by-name delete of Orders/Challenges (their hash-suffixed names
make a by-name delete a no-op; the Certificate owner-ref cascade plus the
host-scoped sweep handle them), narrow the hook RBAC to challenges only, and
note the Gateway-mode no-op.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
2026-06-30 14:28:31 +03:00
Алексей Артамонов ae7c232409 fix(qdrant): anchor PVC ordinal so cleanup never deletes a sibling's data
The post-delete hook selected PVCs with an unanchored name prefix
(qdrant-storage-<release>-). Because release names are qdrant-<app>, deleting
release "qdrant-test" matched not only its own PVC
(qdrant-storage-qdrant-test-0) but also a sibling "qdrant-test-2"'s PVC
(qdrant-storage-qdrant-test-2-0) in the same namespace — irreversibly deleting
the sibling's data (or wedging it in Terminating via pvc-protection).

Anchor the ordinal with grep -E "...-[0-9]+$" so only the release's own
data PVCs match. A release name that is a prefix of a sibling no longer
collides.

- cleanup-pvc.yaml: grep -E with anchored "-[0-9]+$".
- tests: assert the anchored selector and add a notMatchRegex guard against
  regressing to the unanchored prefix.
- e2e: gate the post-delete wait on a NotFound result specifically, so a
  transient API/auth error is not misread as "PVC reclaimed".

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
2026-06-30 14:17:14 +03:00
Aleksei Sviridkin a349c753ad fix(kubernetes): make node group resources optional and prefer them over instanceType
A node group is sized either by instanceType or by explicit resources, but the
generated values schema required resources on every node group. A node group
with instanceType and no resources — or an empty resources dropped during
typed-CR to HelmRelease translation — failed schema validation with "missing
property 'resources'" even though sizing was fully specified.

Mark resources optional in the node group definition and regenerate the schema,
API types, README, and ApplicationDefinition. The chart template emits
domain.cpu/memory only when resources is present, so a node group sized purely
by instanceType now passes validation and installs.

When a node group sets both instanceType and explicit resources, explicit
resources take precedence: the instancetype is omitted from the rendered VM and
the per-group instancetype lookup is skipped, so KubeVirt accepts the explicit
domain.cpu/memory with no instancetype to conflict with. instanceType stays
required and defaulted (no schema change), so the console's required
instanceType dropdown keeps working — sending the default instanceType plus
explicit resources simply sizes the VM by resources. This replaces an earlier
render-time hard failure on the both-set case, which was unreachable to clear
from the dashboard (the required dropdown cannot be emptied).

Explicit resources are all-or-nothing alongside an instanceType: a partial spec
(only cpu, or only memory) on a group that carries a non-empty instanceType
would emit one domain override next to the instancetype — which KubeVirt rejects
— so it now fails at render time with a clear error. Partial resources on a
group without an instanceType (used to tune kubelet reservations) are unaffected.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 13:32:15 +03:00
Aleksei Sviridkin 665e7e4609 test(e2e): make etcd.bats teardown run and settle under cozytest (#3131)
## What this PR does

The e2e runner `hack/cozytest.sh` is not real bats — it converts `@test`
blocks to shell functions and recognises a `cozy_cleanup()` hook, but
never calls `setup()`/`teardown()` and never opens fd 3. So
`etcd.bats`'s `setup()` never ran (etcd leaked its HelmRelease and data
PVCs into the shared `tenant-test` namespace) and `dump_diagnostics`
wrote to an unopened fd 3, silently dropping failure diagnostics. The
teardown's `kubectl wait --for=delete` lines also passed
`--ignore-not-found`, which `wait` does not accept.

This moves teardown into an `etcd_drain()` helper, run three ways: a
pre-clean at the start of the first scenario, inline at the end of the
last scenario (under `set -e`, so a teardown that does not settle fails
the suite), and as `cozy_cleanup` (the best-effort safety net cozytest
runs at suite exit and on the first failing test). The drain reclaims
what a plain CR/HR delete leaves behind:

- the StatefulSet's retained `data-etcd-*` PVCs (not owned by the Etcd
CR) — a later install otherwise reuses the same-named volumes, boots the
previous member identity, never forms a fresh quorum, and the chart's
`helm --wait` fails; and
- the cluster-scoped Kamaji `DataStore`, whose leaked finalizer wedges
the HR uninstall — cleared each pass until the Etcd CR, HR, DataStore,
and PVCs are all gone, bounded by a 4-minute deadline that fails the
drain on timeout.

Remnants are probed by name so a transient API error is never mistaken
for a successful delete. Diagnostics now go to stdout, which cozytest
captures and prints on failure.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved end-to-end cleanup so leftover resources are removed more
reliably between runs.
  * Made failure diagnostics easier to capture when tests fail.
* Added stronger cleanup handling at the start and end of the test flow
to reduce flaky suite behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 13:23:51 +03:00
Aleksei Sviridkin 0c6426feaa fix(seaweedfs): enforce read-only bucket access by bumping cosi-driver to v0.3.1
The seaweedfs COSI driver shipped at v0.1.2 hardcodes read-write S3
actions (Read, Write, List, Tagging) for every BucketAccess and never
reads the accessPolicy parameter. The chart already defines a -readonly
BucketAccessClass carrying parameters.accessPolicy: readonly, but v0.1.2
ignores it, so a credential meant to be read-only can still write
(upload and delete) objects.

Bump the driver to v0.3.1, which honours accessPolicy: readonly by
issuing Read,List-only credentials (read-write and the empty default
stay Read,Write,List,Tagging). The override lives in the package
values.yaml rather than the vendored chart so it survives make update;
the sister extra/seaweedfs chart already runs v0.3.1.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 13:19:39 +03:00
Aleksei Sviridkin 7c67aafed5 fix(kubernetes): run the Talos reconcile Job in the main install phase (#3145)
## What this PR does

The tenant worker `MachineDeployment` `md0` intermittently never reaches
`status.replicas=2`. CAPI blocks MachineSet creation until the
`TalosConfigTemplate` that the deployment's `bootstrap.configRef` points
to exists, and on failing installs that template is never created.

### Root cause — an ordering deadlock

The `TalosConfigTemplate` is created only by the `talos-reconcile`
**post-install Helm hook** (the chart can no longer render it at
template time — three of its inputs are produced asynchronously after
apply). A post-install hook runs only after Helm has applied the
release's main resources and, when Helm waits, after they become Ready.

But the same release also emits in-tenant addon HelmReleases — notably
**cilium** — whose readiness needs tenant worker nodes, and those nodes
can only be created once this hook has produced the
`TalosConfigTemplate`. So when the main-resource wait is in effect, the
install deadlocks: it waits for cilium to be Ready, cilium waits for
worker nodes, the nodes wait for the `TalosConfigTemplate`, and the
template waits for the post-install hook — which never runs, because the
install never finishes the wait that precedes the hook phase. The
install times out and `md0` never scales.

The release sets `helm-install-disable-wait` precisely to skip that
main-resource wait so the hooks run, but its effect is unreliable —
which is what makes the failure bimodal: when the wait is skipped the
hook runs and the template lands in ~46s; when it is not, the deadlock
strands the install.

A timestamped poller over a failing run (both `kubernetes-latest` and
`kubernetes-previous`, 640 polls) captured: `TalosConfigTemplate` absent
in 640/640 polls; both post-install hook Jobs (`talos-reconcile` and
`bootstrap-token`) never created — the install never reached the
post-install hook phase; all main resources applied (MachineDeployment,
KubevirtMachineTemplate, MachineHealthCheck, WorkloadMonitor, child
HelmReleases, Talos PKI/secrets) and the four hook inputs ready early
and stable; the parent HelmRelease stuck at `Running 'install' action
20m` then looping into `upgrade`, with the cilium sub-release timing out
on `cilium-operator` (which has no worker node to schedule on) —
downstream of the missing template.

### The fix

Render the `talos-reconcile` Job and its RBAC / ServiceAccount / egress
`CiliumNetworkPolicy` as ordinary **main-phase** resources instead of
post-install hooks. The Job is then created during the apply itself —
not after the wait — and runs on the management cluster, which has nodes
(unlike the tenant `cilium-operator`). It produces the
`TalosConfigTemplate`, `md0` scales, workers join, cilium goes Ready,
and the main-resource wait (if any) closes within the install budget:
the Job completes in tens of seconds, and node boot plus cilium
readiness is a few minutes — comfortably inside the 20m install timeout.
This works whether or not `helm-install-disable-wait` takes effect, so
it does not depend on that annotation's reliability.

Because a main-phase Job is immutable, it carries a content-hash name
suffix over its whole rendered spec (mirroring the
`KubevirtMachineTemplate` idiom in `cluster.yaml`): an identical render
keeps the same name and Helm no-ops it; any spec change yields a new
name and a fresh Job, so no change ever lands on the same name and trips
an immutable-field patch error. The applied `TalosConfigTemplate` keeps
its stable `<release>-<group>` name (referenced by
`bootstrap.configRef`), so an unchanged render re-applies a
byte-identical template — a CAPI no-op that never rolls existing
workers. The Job name is truncated to stay within the 63-char DNS-label
limit while always preserving the hash. The runtime input-waits, the
single `kubectl apply` of the template with a `KamajiControlPlane`
ownerReference for GC, and the `extraHostEntries` are all unchanged.

`helm-install-disable-wait`'s unreliable effectiveness is a real latent
issue (it explains the created-vs-never bimodality), but this fix breaks
the deadlock independent of it, so that is a separate follow-up rather
than a blocker here.

Supersedes #3139 (a 10-minute wait-budget bump that treated the symptom;
the deadlock is structural and a larger budget would not have helped).

### Screenshots

Not applicable — no UI changes.

### Testing

helm-unittest extended to pin: no `helm.sh/hook` on any of the seven
talos-reconcile documents, the content-hash name,
`ttlSecondsAfterFinished`, the preserved template apply + ownerReference
+ extraHostEntries, the hash rotating on a spec change, and the
long-name truncation staying within 63 chars. Full chart suite green
(128 tests). The applied template is byte-identical to the previous hook
output (verified by diff), so existing tenants are upgrade-safe.

### Release note

```release-note
fix(kubernetes): create the tenant worker TalosConfigTemplate from a main-phase Job instead of a post-install hook, so worker MachineDeployments reliably scale during install
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved Kubernetes addon install behavior by running reconcile work
during the main chart phase, reducing readiness deadlocks and upgrade
issues.
* Added more reliable reconciliation handling with stable,
change-detected job recreation and updated network access for the
reconcile workload.
* Refined chart and test guidance to match the new install flow and Helm
behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 12:29:55 +03:00
Aleksei Sviridkin f9d1138c6e feat(keycloak): configurable login theme via platform branding (#3124)
## What this PR does

Makes a realm's Keycloak login theme configurable through platform
`branding` values, closing a gap: `branding` could set the realm display
name but not its login theme, and the keycloak chart's `themes:` list
was not reachable from platform values.

Two new keys under `branding` (propagated via `_cluster.branding`, the
same path that already feeds `displayName`):

- `branding.loginTheme` — keycloak-configure renders it as
`ClusterKeycloakRealm.spec.themes.loginTheme`, so the realm selects that
login theme.
- `branding.loginThemeImage` — when set together with `loginTheme`, the
keycloak chart appends `{name: loginTheme, image: loginThemeImage}` to
its effective theme list, so the existing init-container mounts the
theme image into `/opt/keycloak/themes/`. No StatefulSet patching or
per-package values plumbing.

`loginTheme` alone (no image) still selects the theme — for themes
already baked into the Keycloak image. Explicit chart `themes:` entries
keep working; the branding-derived entry is appended after them.

Flow: platform `branding` → `_cluster.branding` (cozystack-values
Secret) → keycloak-configure (realm loginTheme) + keycloak (theme init
container). Covered by helm-unittest in both charts.

### Screenshots

No bundled UI assets — this PR adds the mechanism only; the login theme
is operator-supplied via `loginThemeImage`, so the visual result depends
on that image.

### Release note

```release-note
feat(keycloak): select the realm login theme and ship its theme image through platform `branding` values (`branding.loginTheme`, `branding.loginThemeImage`)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for configuring a custom login theme from platform
branding, including automatic selection in the realm setup.
* Theme entries can now be combined with existing theme settings and
rendered in the expected order.

* **Documentation**
* Expanded configuration guidance for branding and theme-related
settings with clearer examples.

* **Tests**
* Added coverage for login theme rendering and fallback behavior when
theme settings are missing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 12:29:52 +03:00
Aleksei Sviridkin 31c15b282f feat(network): mechanism-agnostic LoadBalancer exposure (ExposureClass + ServiceExposure) (#3081)
## What this PR does

Lays the foundation for migrating tenant LoadBalancer exposure off the
deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off
MetalLB onto Cilium-native announcement — the mechanism-agnostic shape
prescribed when the earlier chart-side approach (#2468) was reverted in
#2603.

It adds a new `network.cozystack.io/v1alpha1` API group and a controller
that translates intent into whichever LoadBalancer backend the cluster
admin configured:

- `ExposureClass` (cluster-scoped, admin): binds a logical class name to
a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP
address scope — the StorageClass/IngressClass pattern.
- `ServiceExposure` (namespaced, tenant intent): "expose this Service
via class X"; status reports the resolved backend and assigned IPs.
- A `serviceexposure` controller renders the backend's pool + announcer
once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium
`CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to
the namespaces that opted in. Backends are pluggable behind a small Go
interface, so adding BGP or another cloud LB is one implementation, not
an API change. The controller never mutates the target Service — the
chart still owns its shape.

Platform wiring: `publishing.exposureClass.*` renders the admin
`ExposureClass` and the host ingress publishes through it. The default
(no class set, `externalIPs`) leaves every existing cluster's behaviour
byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes
a single `ExposureClass.spec.backend` change.

This is the foundation only — Part of #2586. Converting per-app
exposure, flipping Cilium `l2announcements` on, changing the default,
and removing MetalLB are deferred to later phases.

### Release note

```release-note
feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added new `ExposureClass` and `ServiceExposure` custom resources with
support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB),
including optional L2 and address configuration.
* Added Helm wiring and templates to enable service/ingress publishing
via `ServiceExposure`, selecting a configured (or default)
`ExposureClass`.
* **Bug Fixes**
* Improved reconciliation, finalizer-based cleanup, and status updates
for readiness and assigned IP reporting, including safer handling when
backends or managed resources change.
* **Tests**
* Added end-to-end and unit test coverage for backend rendering,
readiness, and cleanup flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 12:29:48 +03:00
Aleksei Sviridkin 50748fed98 fix(kubernetes): make the default md0 node group removable (#2936)
Same-repo mirror of #2753 by @mattia-eleuteri, opened so the `Build` CI
job can run. Fork PRs skip the OCIR registry login (`if:
!github.event.pull_request.head.repo.fork` in `pull-requests.yaml`) and
so can't push the per-PR images the build needs — meaning a fork PR can
never pass `Build`, regardless of the change. The commits here are
unchanged and authored by @mattia-eleuteri; full description and
discussion in #2753.

This includes migration **44** (renumbered from 43 after the
seaweedfs-db migration landed on `main`) and bumps `targetVersion` to
**45**.

Supersedes #2753.

```release-note
fix(kubernetes): the default `md0` node group is now applied only when no node groups are configured, so it can be removed or replaced. Migration 44 pins `md0` explicitly on existing clusters to preserve their topology.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Empty `nodeGroups` settings now automatically provision a default
`md0` worker group.
* Existing environments are updated to keep this default group available
during upgrades.

* **Bug Fixes**
* User-defined node groups now fully replace the default instead of
being merged with it.
* Improved handling of node-group resources and permissions in
Kubernetes app rendering.

* **Documentation**
* Updated Kubernetes app docs and schema details to reflect the new
default behavior.

* **Tests**
* Added coverage for default and custom node-group configurations, plus
migration behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 12:08:09 +03:00
Aleksei Sviridkin 81198b294f fix(bucket): clean up orphaned cert-manager ACME solver resources on delete (#3060) (#3076)
## What this PR does

After deleting a `Bucket` app CR, cert-manager's temporary ACME HTTP-01
solver
resources (`cm-acme-http-solver-*` Ingress/Pod/Service) were left
orphaned in the
tenant namespace. They are created by cert-manager to answer an ACME
challenge
and are owned by the `Challenge`/`Order` → `Certificate`, not by the app
Ingress.
If a challenge gets stuck and the app is deleted, the parent Ingress is
removed
but the orphaned challenge leaves the solver objects with no live owner
to
cascade-delete them, so Helm/Flux uninstall cannot reap them.

This adds a `post-delete` cleanup hook that deletes the release's
`Certificate`/`Order`/`Challenge` (cert-manager then cascade-removes the
solver
objects) and then sweeps any leftover solver objects by the reliable
`acme.cert-manager.io/http01-solver=true` label.

RBAC is verified against the vendored cert-manager CRDs
(`cert-manager.io` → certificates; `acme.cert-manager.io` → orders,
challenges)
plus core `services`/`pods` and `networking.k8s.io` ingresses. The
cleanup Job is
hardened: non-root (uid 65534), read-only root fs, all caps dropped,
`seccompProfile: RuntimeDefault`. Failures are logged but never block
teardown.

The preventive wildcard-cert guard the issue mentions already exists in
the
vendored `bucket-system` Ingress template (it drops the ACME annotations
in
wildcard mode), so no preventive change is needed — only cleanup.

No user data is affected: only cert-manager's transient challenge
artifacts are
removed.

Fixes #3060.

### Release note

```release-note
fix(bucket): clean up orphaned cert-manager ACME http01-solver resources (Ingress/Pod/Service) left behind after a Bucket app is deleted.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added a bucket app `test` command that runs Helm unit tests.
* Introduced a post-delete ACME cleanup hook to remove release-scoped
cert-manager TLS and HTTP-01 solver resources, using dedicated
least-privilege access controls.
* **Bug Fixes**
* Improved cleanup safety with bounded execution and hardened container
security.
* **Tests**
* Added a Helm test suite covering rendered hook documents, including
host-scoped targeting and negative assertions to prevent overly broad
deletions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 11:27:53 +03:00
Aleksei Sviridkin 544ba5bf16 fix(harbor): preserve jobservice and trivy storageClass on upgrade (#2930)
## What this PR does

The `apps/harbor` chart never passed `storageClass` to the jobservice
`jobLog` PVC, so the upstream chart rendered it without a
`storageClassName`. On first install the default-StorageClass admission
then stamps the cluster default onto the live PVC. Because a PVC's
`storageClassName` is immutable, the rendered manifest (no class) and
the live object (admission-defaulted class) drift apart, which surfaced
as upgrade failures on the `harbor-*-system` HelmRelease.

This propagates `storageClass` to the jobservice PVC, with one
safeguard: a `lookup` reads the live jobservice PVC and reuses its
current `storageClassName` when the PVC already exists, only falling
back to the configured `storageClass` for a fresh install. Keeping the
rendered value equal to the live one means an existing PVC whose
admission-defaulted class differs from the configured `storageClass` is
preserved rather than triggering an immutable-field rejection on
upgrade. A live PVC pinned to an explicit `""` (no dynamic provisioning)
round-trips through the upstream `"-"` sentinel instead of collapsing to
an omitted field, and the spec access is guarded so a PVC with no
`storageClassName` key stays omitted.

The trivy `StatefulSet`'s `volumeClaimTemplate` carried the same latent
drift — `.Values.storageClass` was forwarded blindly into a VCT, which
cannot be patched at all — so it now gets the identical lookup-preserve
treatment, keyed off `data-<release>-trivy-0`. This closes both halves
of #2368.

Also adds a helm-unittest regression suite and wires a `test:` target so
`make unit-tests` stops skipping `packages/apps/harbor`.

Validated with `helm template` across storageClass/trivy permutations
and the new unit tests. The jobservice preserve path was additionally
confirmed with a server-side dry-run against a running harbor release:
`lookup` resolved the live PVC's class, and with `storageClass`
overridden to a different class the jobservice render stayed pinned to
the live PVC's class — confirming the immutable-field rejection is
avoided. The trivy change applies the same proven pattern. Note: under
`helm template`/helm-unittest `lookup` returns empty, so the unit suite
exercises only the fresh-install fallback; the preserve branches are
covered by the live `lookup`.

Fixes #2368

### Release note

```release-note
fix(harbor): preserve the jobservice PVC and trivy volumeClaimTemplate storageClass on upgrade, fixing upgrade failures caused by storageClassName drift
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Harbor now preserves the existing jobservice storage class during
upgrades and ensures it’s applied on initial installation when
available, with consistent handling for Trivy persistence as well.

* **Tests**
* Added a Helm unittest regression suite covering storage class
propagation and the correct conditional rendering of persistence blocks
when storage class and Trivy settings vary.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 11:27:50 +03:00
Aleksei Sviridkin cad4bd780a fix(dashboard): join pod phase on (namespace,pod) instead of uid (#2782)
## Problem

The **Capacity Planning** and **Namespaces** dashboards join
request/limit
metrics to `kube_pod_status_phase` via `* on (uid) group_left(phase)`.

Some pods (e.g. static pods like `ssh-proxy`) carry no `uid` label, so
every
such series collapses to the same empty match key and collides. The
query
backend rejects this with a 422:

> duplicate time series on the right side of `* on(uid)
group_left(phase)`

The affected panels render an error instead of data.

## Fix

Switch the join key to the always-unique `(namespace, pod)` and filter
`kube_pod_status_phase{phase="Running"} == 1` — the same pattern already
used
in `dashboards/gpu/gpu-quotas.json`. Sum results are unchanged (the old
multiply-by-0/1 is equivalent to filtering on `== 1`).

8 queries updated across two dashboards (CPU + memory, requests +
limits):
- `dashboards/main/capacity-planning.json` (6)
- `dashboards/main/namespaces.json` (2)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Corrected Capacity Planning dashboard CPU/Memory percentages and
unrequested-resource calculations so they consider only running pods.
* Fixed the “% Requested” and “Unrequested” panels for both CPU and
Memory to use consistent running-pod filtering and metric matching.
* Updated the Namespaces dashboard CPU and Memory repeated panels so
Requests and Limits reflect only running-pod metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

### Release note

```release-note
fix(dashboard): fix Capacity Planning and Namespaces dashboards failing with "duplicate time series" errors when static pods are present
```
2026-06-30 11:27:46 +03:00
Aleksei Sviridkin ab67dd8d7b ci(renovate): force chore(deps) commits and add area/dependencies label (#3015)
## What

Make Renovate PRs land with consistent, conventional labels and clear a
recurring Dependency Dashboard warning.

## Why

Renovate's `:semanticCommits` preset picks `fix(deps)` for stable
(`>=1.0`) production modules and `chore(deps)` for `0.x` modules. The
result:

- `fix(deps)` bumps were auto-labeled **`kind/bug`** by the PR
auto-labeler → routine dependency updates polluted the bug list.
- The auto-labeler has no `deps → area/*` mapping, so **every** Renovate
PR fell through to **`area/uncategorized`**.
- `kubevirt.io/csi-driver` is pinned to an untagged pseudo-version
(`v0.0.0-2026…`) with no release tags, producing a permanent `no-result`
lookup failure on the Dependency Dashboard.

## Changes

- **`.github/renovate.json`**
- Pin `semanticCommitType: chore` + `semanticCommitScope: deps` → every
dependency PR is now `chore(deps): …` (maps to `kind/cleanup`, no more
bogus `kind/bug`).
- Disable Renovate for `kubevirt.io/csi-driver` (commit-pinned,
unbumpable) to clear the dashboard lookup warning.
- **`.github/labels.yml`** — add `area/dependencies` (justified: 25+
dependency PRs queued, well over the 3-issue threshold for a new area).
- **`.github/workflows/pr-labeler.yaml`** — map scope `deps →
area/dependencies`.

## Result

Future Renovate PRs: `chore(deps): …` → `automated` + `kind/cleanup` +
`area/dependencies` + `do-not-merge/hold`.

## Notes

- `renovate.json` / `labels.yml` take effect once merged; the labeler
change applies to PRs opened or edited after merge.
- The `sigs.k8s.io/yaml` lookup warning on the dashboard is left
untouched — it's a normal module and reads as a transient datasource
hiccup, not a config issue.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Added a new `area/dependencies` label to consistently tag
dependency-related pull requests.
* Updated the PR auto-labeler to map commit/title `deps` scopes to
`area/dependencies`.
* Standardized Renovate dependency commit messages to use `chore(deps)`,
including special handling for Go module updates, and disabled Renovate
updates for the `kubevirt.io/csi-driver` Go module.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 11:27:43 +03:00
Myasnikov DaniilandClaude Opus 4.8 4c5a8c830c test(platform): cover migration 47 fail-closed contract
Migration 47's safety property is that any per-object read or patch failure
aborts the migration (exit 1) before the cozystack-version stamp advances, so
the pre-upgrade hook fails and Helm never rolls the new kubernetes chart against
an un-pinned cluster (whose live md0 MachineDeployment would otherwise be
pruned). That branch was verified by reading only.

Add hack/kubernetes-md0-migration.bats, mirroring the kubectl-mock style of the
plain-shell cozytest.sh runner: a fake kubectl on PATH simulates a non-NotFound
read failure and a patch failure, and the tests assert the migration exits
non-zero and never reaches the version stamp (no `kubectl apply` of
cozystack-version is logged). A positive-control case asserts a clean run does
reach the stamp, so the "no stamp" assertions are meaningful rather than vacuous.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-30 12:08:18 +05:00
Myasnikov DaniilandClaude Opus 4.8 c818ece25b test(kubernetes): cover md0 removability with user-defined nodeGroups
The existing case pins the empty-nodeGroups -> default md0 path (the helper's
else-branch). This adds the complementary if-branch: when a user supplies their
own nodeGroups, they are authoritative and the built-in md0 is NOT merged in, so
md0 is removable. The assertion selects every rendered MachineDeployment and
requires it to be the user's own group, so a regressed md0 leak surfaces as a
second MachineDeployment named <release>-md0 that fails the assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-30 12:08:18 +05:00
Myasnikov Daniil ab55ff1924 test(kubernetes): cover the default md0 node group render
Assert that an empty nodeGroups renders the built-in md0 group via the
kubernetes.nodeGroups helper else-branch (the behaviour that makes md0
removable): the default md0 MachineDeployment is emitted and the default 20Gi
system disk flows through. A minimal VirtualMachineClusterInstancetype plus
dummy CAPI objects are registered so the per-group instanceType lookup and the
old-template preservation LISTs resolve under helm-unittest.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-30 12:08:18 +05:00
Myasnikov Daniil b9749cec80 fix(platform): md0 migration -- fail closed and preserve the no-delete label
Migration 47 pins the implicit default md0 node group before the kubernetes
chart upgrade so the live md0 MachineDeployment is not pruned. Two correctness
fixes:

- Stamp the cozystack-version ConfigMap via the shared stamp_cozystack_version
  helper instead of `kubectl create configmap | kubectl apply`. The label-less
  apply by the same field manager stripped the platform.cozystack.io/no-delete
  label migration 42 added, dropping cozystack-version out of the
  cozystack-no-delete-guardrail policy; the helper re-emits the label and is
  enforced for every migration >= 42 by hack/cozystack-version-stamp.bats.
- Fail closed. A non-NotFound read error is now counted as a failure rather
  than silently skipped, and any failure exits 1 so the pre-upgrade hook aborts
  before Helm rolls out the new chart against clusters whose md0 is not yet
  pinned. The version stamp stays at 47 and the migration retries next upgrade.

Also narrow the merge patch to only md0 so a concurrent edit to a sibling node
group between the read and the patch cannot be reverted.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-30 12:08:17 +05:00
mattia-eleuteri 602b8d06fd fix(platform): add migration 47 to pin implicit md0 node group
After the kubernetes chart stops force-merging a default md0 node group,
clusters that relied on the implicit md0 (defined other nodeGroups without
md0) would have their live md0 MachineDeployment pruned by Helm on the next
reconcile. Migration 47 makes md0 explicit on those clusters to preserve the
current node topology; it is idempotent, treats a missing CRD as a no-op, and
skips clusters that already define md0.

Signed-off-by: mattia-eleuteri <mattia@hidora.io>
2026-06-30 12:08:17 +05:00
mattia-eleuteri 7e3c5f99db fix(kubernetes): apply default md0 node group only when nodeGroups is empty
The chart shipped a default nodeGroups.md0 in values.yaml. Helm deep-merges
values, so the default md0 was re-added on top of any user-supplied
nodeGroups; combined with Kubernetes stripping null values, md0 could never
be removed by configuration. Move the default into the template
("kubernetes.nodeGroups" helper) so it is applied only when nodeGroups is
empty. User-supplied node groups now fully replace the default, letting users
name and omit groups (including md0) freely.

Test fixtures that relied on the chart-default md0 to supply diskSize via
values merge now set it explicitly.

Signed-off-by: mattia-eleuteri <mattia@hidora.io>
2026-06-30 12:08:17 +05:00
Aleksei Sviridkin 32c4d60b43 fix(kubernetes): run the Talos reconcile Job in the main install phase
The talos-reconcile Job is the only thing that creates the
TalosConfigTemplate the worker MachineDeployment's bootstrap.configRef
points to; CAPI blocks MachineSet creation until it exists, so tenant
workers cannot scale until the Job has run.

The Job was a post-install/post-upgrade Helm hook, which runs only after
Helm has applied the release's main resources and, when Helm waits, after
they become Ready. But the same release emits in-tenant addon HelmReleases
— notably cilium — whose readiness needs worker nodes, and those nodes
need the TalosConfigTemplate this Job produces. So when the main-resource
wait is in effect the install deadlocks: it waits for cilium to be Ready,
cilium waits for workers, the workers wait for the template, and the
template waits for this hook — which never runs because the install never
finishes the wait that precedes the hook phase. The install times out and
the MachineDeployment never scales. The release sets
helm-install-disable-wait to skip that wait, but its effect is unreliable,
which makes the failure bimodal: the template lands in ~46s when the wait
is skipped, and the install is stranded when it is not.

Render the Job and its RBAC/ServiceAccount/egress policy as ordinary
main-phase resources. The Job is then created during the apply itself, not
after the wait, and runs on the management cluster where nodes exist
(unlike the tenant cilium-operator), so it produces the template, the
MachineDeployment scales, workers join, cilium goes Ready, and the wait
(if any) closes within the install budget — independent of whether
helm-install-disable-wait takes effect.

The Job is immutable, so it carries a content-hash name suffix over its
rendered spec (mirrors the KubevirtMachineTemplate idiom in cluster.yaml):
an identical render keeps the name and Helm no-ops it, while any spec
change yields a new name and a fresh Job, so no spec change ever lands on
the same name and triggers an immutable-field patch error. The applied
TalosConfigTemplate keeps its stable <release>-<group> name, so an
unchanged render re-applies a byte-identical template — a CAPI no-op that
never rolls existing workers.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 07:01:51 +03:00
Aleksei Sviridkin 4f8c200c5b test(securitygroup): cover egress toApp/toSG peer projection round-trip
The ingress fromApp/fromSG projection is exercised end to end, but the
symmetric egress toApp/toSG path — toApp to a lineage-label
endpointSelector and toSG to the referenced group's membership-label
endpointSelector — had no test. A wrong label key there would silently
point an egress allow at the wrong endpoints. Pin the projection and the
reverse reconstruction with a Create+Get round-trip, mirroring the
existing ingress coverage.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 01:20:25 +03:00
Aleksei Sviridkin a86817915a fix(securitygroup): report async deletion for finalizer-guarded policies
The backing CiliumNetworkPolicy carries the membership finalizer, so
deleting a SecurityGroup is virtually always asynchronous: the policy
gets a deletionTimestamp and survives until the securitygroup-controller
strips the membership labels off member pods and removes the finalizer.
REST.Delete nonetheless returned (nil, true), reporting the object as
instantly deleted, so the endpoint emitted a Success status claiming
completion instead of surfacing the terminating object — a
rest.GracefulDeleter contract violation for any client that distinguishes
the two to detect pending teardown.

After the delete, read the object's state back through the direct
(uncached) client and report deleted=true only when it is actually gone;
while a finalizer holds it, return deleted=false with the terminating
object. Using the direct client avoids a stale cache-backed read
misreporting an async delete as instant, and a dry-run (which leaves the
object in place) is resolved from the object's authoritative finalizers.
Implement rest.MayReturnFullObjectDeleter so the DELETE endpoint is
advertised and serialized as returning the SecurityGroup object, matching
the generic registry.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 01:20:14 +03:00
Aleksei Sviridkin cc4fb40da7 fix(securitygroup): project rules-less SecurityGroup to a valid policy
A SecurityGroup with attachments but no ingress or egress rules exists
only to stamp its membership label on the attached applications' pods, so
other SecurityGroups can reference it as a peer. Its backing
CiliumNetworkPolicy carried only an endpointSelector, which the
cilium.io/v2 CRD rejects synchronously: the spec anyOf requires at least
one of ingress/ingressDeny/egress/egressDeny, so a selector-only policy
fails with "spec.ingress: Required value". A membership-only group
could therefore never be created.

Always serialize the backing policy's ingress section, emitting an empty
list when the group has no ingress rules. An empty list is the CRD's
documented no-op ("if omitted or empty, this rule does not apply at
ingress") and enableDefaultDeny defaults to false for a direction with
no rules, so the policy is schema-valid yet datapath-inert: it neither
allows nor denies, matching the additive, non-isolating semantics of an
empty rule list. The projection guarantees the slice is non-nil so it
never marshals to null, which the CRD (no nullable fields) would also
reject. The empty section collapses back to nil on read, so a
membership-only group round-trips with no ingress or egress.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-30 00:36:09 +03:00
Алексей Артамонов 2587458bd4 fix(qdrant)!: reclaim data PVC by name prefix, not volumeClaimTemplate label
The previous approach routed a release-unique label through
qdrant.persistence.additionalLabels, which the upstream chart renders into
the StatefulSet's volumeClaimTemplates[].metadata.labels. That field is
immutable on an existing StatefulSet, so for any already-deployed Qdrant the
API server would reject the upgrade and the <release>-system HelmRelease
(retries: -1) would retry-fail forever — permanently wedging reconciliation.
Label changes on a volumeClaimTemplate also never apply to PVCs that already
exist, so the cleanup hook could not reclaim those instances anyway.

Switch the post-delete hook to select PVCs by the deterministic StatefulSet
naming prefix (qdrant-storage-<release>-) instead. PVC names derive purely
from the release name, so this needs no StatefulSet change (upgrade-safe) and
also reclaims PVCs from instances created before the hook existed.

- qdrant.yaml: drop the additionalLabels passthrough entirely.
- hooks/cleanup-pvc.yaml: delete by name prefix via kubectl get -o name | grep.
- tests: assert the name-prefix command instead of the label passthrough.
- e2e: assert qdrant-storage-<name>-0 is gone after delete (async-waited).

BREAKING CHANGE: deleting a Qdrant app now permanently removes its data PVC
and the vector data it holds.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
2026-06-29 23:10:57 +03:00
Алексей Артамонов 6a43adbd09 fix(bucket): drop no-op label-based order/challenge delete
Per maintainer review: the label selector
cert-manager.io/certificate-name=<release>-ui-tls matches nothing —
Challenge objects carry no labels, and on Orders certificate-name is an
annotation (not a label). Orders are already reaped by the Certificate
owner cascade and Challenges by the host-scoped .spec.dnsName sweep, so
the line deleted nothing and its comment misstated cert-manager labeling.

Remove the dead delete, replace the inaccurate comment, and drop the
test assertion that pinned it; add negative assertions so a label-based
order/challenge delete cannot be reintroduced.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
2026-06-29 23:10:24 +03:00
Aleksei Sviridkin fdbb9cc55a test(e2e): make etcd.bats teardown run and settle under cozytest
The e2e runner (hack/cozytest.sh) is not real bats: it never invokes
setup()/teardown() and never opens fd 3. So etcd.bats's setup() never ran
and dump_diagnostics wrote to an unopened fd 3, silently aborting on
failure — and etcd leaked into the shared tenant-test namespace.

Move teardown into etcd_drain() and run it inline at the end of the last
scenario (under set -e, so a teardown that does not settle fails the
suite) and as cozy_cleanup (the best-effort hook cozytest runs at suite
exit and on the first failing test). The first scenario also drains up
front so a dirty namespace cannot taint it.

etcd_drain reclaims what a plain CR/HR delete leaves behind:
- the StatefulSet's retained data-etcd-* PVCs (not owned by the Etcd CR);
  a later install otherwise reuses them, boots the old member identity,
  never forms a fresh quorum, and the chart's helm --wait fails; and
- the cluster-scoped Kamaji DataStore, whose leaked finalizer wedges the
  HR uninstall — cleared each pass until the Etcd CR, HR, DataStore, and
  PVCs are all gone, bounded by a deadline that fails the drain on
  timeout. Remnants are probed by name so a transient API error is never
  mistaken for a successful delete.

Drop --ignore-not-found from the teardown waits (not a valid wait flag);
the drain polls for existence instead. Diagnostics now go to stdout,
which cozytest captures and prints on failure.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-29 16:01:48 +03:00
Aleksei Sviridkin 4404c5c5eb fix(etcd): pin tenant etcd to v3.5.31 to restore consistent watches (#3122)
## What this PR does

Tenant Kubernetes control planes get their etcd from the `extra/etcd`
chart, which creates an `EtcdCluster` (`etcd.aenix.io/v1alpha1`)
reconciled by etcd-operator v0.4.5. That operator hardcodes the etcd
image to `quay.io/coreos/etcd:v3.5.12`, and the v1alpha1 CRD exposes no
version field, so every tenant etcd runs 3.5.12.

kube-apiserver only enables the `RequestWatchProgress` storage feature
on etcd >= 3.5.13 (or >= 3.4.31). Kubernetes >= 1.31 enables
`ConsistentListFromCache` — locked-to-true in 1.35 — which depends on
it. On etcd 3.5.12 every consistent watch therefore fails inside the
stream with HTTP 500 (`the required storage feature RequestWatchProgress
is disabled`), which breaks watch-based clients of aggregated APIs in
modern (k8s 1.35) tenants.

This pins the etcd container image to a >= 3.5.13 patch release
(v3.5.31) by setting `image` on the `etcd` container in the
EtcdCluster's `spec.podTemplate.spec`. etcd-operator v0.4.5
strategically merges `podTemplate.spec` over its generated pod spec
(matched by container name), so the override takes effect with no
operator change — the same mechanism the chart already uses for the
metrics port, probes, and resources. It restores consistent watches
without waiting on the etcd-operator v1alpha2 migration (#2859), which
is the longer-term fix.

A helm-unittest suite asserts the etcd container carries the pinned
image and is no longer left on the operator default v3.5.12.

Addresses #3080.

### Upgrade impact

On merge, etcd-operator reconciles every existing tenant `EtcdCluster`
and rolls its StatefulSet from 3.5.12 to 3.5.31. This is a safe in-minor
rolling patch upgrade (no etcd storage-format change, quorum preserved
by the operator's rolling strategy), so expect a brief, sequential etcd
pod restart per tenant and no data migration.

### Release note

```release-note
fix(etcd): pin tenant etcd to v3.5.31 so kube-apiserver can enable RequestWatchProgress (consistent watches) on Kubernetes 1.35 tenants
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Pinned the bundled etcd image to `quay.io/coreos/etcd:v3.5.31` to
improve compatibility with Kubernetes watch-related storage behavior.
* Prevented deployments from reverting to the older default etcd image.
* **Tests**
* Added a new test suite to verify the template uses the expected pinned
etcd image and blocks regressions.
* **Chores**
* Updated dependency automation to recognize and manage the pinned etcd
image version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 15:46:12 +03:00
Aleksei Sviridkin f03d2a510f docs(securitygroup): correct empty-list semantics — an empty list does not deny
The Ingress/Egress field docs claimed "an empty list denies all ingress/egress",
which is a false security guarantee. A SecurityGroup projects onto a
CiliumNetworkPolicy that only ADDS allow rules, and Cilium policies are additive
over the tenant's blanket-allow baseline, so an empty list does not isolate a
pod — effective connectivity stays open. Deny / default-deny enforcement depends
on the default-deny baseline, which is separate future work.

Rewrite the API field docs to the accurate semantics, regenerate the OpenAPI so
the served schema description matches, and fix the one DESIGN.md sentence that
still asserted "denies all traffic" (it already contradicted its own safety
section, which correctly states the API cannot deny anything today).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-29 15:40:23 +03:00
Aleksei Sviridkin 881591d34d fix(securitygroup): log swallowed errors in the membership reconciler
The reconciler intentionally treats a malformed attachments annotation and a
failed pod-mapping List as empty / no-op to stay resilient, but it swallowed
both errors silently. That hides real defects: a bad annotation writer, or a
transient API error that drops a pod update until the next periodic resync with
no trace. Log both while preserving the resilient return behavior.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-29 15:40:23 +03:00
Aleksei Sviridkin 11ed2a960f fix(securitygroup): forward watch errors and field-filter deletions in the API
Two watch-proxy defects in the SecurityGroup REST storage:

- watch.Error events from the backing CiliumNetworkPolicy watch (e.g. a 410 Gone
  for an expired resourceVersion) carry a *metav1.Status, not a
  CiliumNetworkPolicy. The CNP type assertion dropped them, so the client saw a
  cleanly closed stream and never performed the required relist. Forward error
  events verbatim before the type assertion.

- The DELETED-event bypass the label selector needs (a policy whose labels mutate
  out of the selector is deleted carrying non-matching labels, so dropping it
  would strand cached clients) was wrongly applied to the field selector too.
  Name and namespace are immutable, so a field-selected watch (metadata.name=sg-a)
  received other objects' deletions and could corrupt filtered client caches.
  Apply the field filter to every event type.

Both paths are covered by new tests driving a fake backing watch.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-29 15:24:24 +03:00
Aleksei Sviridkin 09f4d8bb82 build: build the securitygroup-controller image in the top-level build target
The securitygroup-controller package has a working `make image` target, but it
was never wired into the top-level Makefile `build:` list, so CI never builds or
pushes the controller image. The shipped values.yaml therefore keeps its
`securitygroup-controller:v0.0.0` placeholder and the controller Deployment
lands in ImagePullBackOff, since that image tag does not exist in the registry.
Add the missing build line next to the other first-party controllers so CI
builds, pushes, and digest-pins the image like every other controller.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-29 15:04:46 +03:00
scooby87 4db565282b fix(postgres): garbage-collect init-job after completion (#3055) (#3069)
## Problem

After deleting a `Postgres` app CR, the chart-managed init Job
`<release>-init-job` and its completed Pod remain in the tenant
namespace indefinitely (reported in #3055).

The init Job is a Helm hook (`post-install,post-upgrade`) whose only
delete policy is `before-hook-creation`. That policy deletes the
previous hook only right before a new one is created (next upgrade) — it
never fires on `helm uninstall`. With no `hook-succeeded` and no TTL,
the completed Job and Pod leak.

## Root cause

`packages/apps/postgres/templates/init-job.yaml` — the Job has no
garbage-collection mechanism for its completed state.

## Fix

Add `ttlSecondsAfterFinished: 120` to the Job spec. The native
Kubernetes TTL controller then GCs the Job and its Pod ~2 minutes after
completion, regardless of the Helm hook lifecycle.

The init Job is a one-shot, idempotent reconciler of
users/databases/roles (`init-script.yaml`) — once it has run it is not
needed until the next upgrade (when Helm recreates it), so GC-ing the
finished Job is safe.

This approach deliberately **does not** touch the hook-delete-policy /
hook-timeout semantics, which the `bootstrap.recovery` path depends on
(see the header comment in `init-job.yaml`). Under `bootstrap.enabled`
the Job is skipped entirely, so the change is inert there.

## Tests

Added a `helm-unittest` suite
`packages/apps/postgres/tests/init_job_cleanup_test.yaml`:
- asserts the init Job renders `spec.ttlSecondsAfterFinished: 120`
- asserts the init Job is skipped under `bootstrap.enabled: true`

Verified locally with `helm template` (TTL present on default render;
Job absent under bootstrap recovery).

## Test plan
- [x] `helm template` renders `ttlSecondsAfterFinished: 120` on the init
Job
- [ ] `helm unittest packages/apps/postgres` green in CI
- [ ] e2e `test-apps-postgres` green

Fixes #3055

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Postgres bootstrap init Jobs now automatically clean up after
completion using a post-finish TTL.

* **Tests**
* Added a new integration test suite to verify the init Job renders with
the expected cleanup TTL, and that the init Job is skipped when
bootstrap is enabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 14:34:51 +03:00
scooby87 49555e562b fix(openbao): garbage-collect the data PVC on delete (#3058) (#3073)
## What this PR does

After deleting an `OpenBAO` app CR, the StatefulSet data PVC
(`data-<release>-0`)
was left orphaned in the tenant namespace, consuming storage
permanently. The
OpenBAO app renders a Flux `HelmRelease` over the vendored upstream
OpenBAO chart,
and StatefulSet-templated PVCs are never garbage-collected on
StatefulSet deletion
by default — the chart set no `persistentVolumeClaimRetentionPolicy`.

This passes a retention policy through the HelmRelease values. The
upstream chart
renders it onto the StatefulSet from
`server.persistentVolumeClaimRetentionPolicy`
(verified in the vendored `server-statefulset.yaml`) — **not** from
`server.dataStorage.*`:

```yaml
server:
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Delete
    whenScaled: Retain
```

A `test` target is added to the package Makefile so the new
helm-unittest suite
(asserting the passthrough) actually runs in CI —
`hack/helm-unit-tests.sh` only
runs packages that define a `test` target.

> [!WARNING]
> **Deleting an OpenBAO app now permanently removes its data PVC and the
data it holds.**
> This is intentional and fixes the storage leak. `whenScaled: Retain`
is kept on
> purpose so scaling an HA/raft cluster down does **not** destroy a
member's data —
> only full deletion of the release reclaims storage. Back up before
deleting.

Fixes #3058.

### Release note

```release-note
fix(openbao): set persistentVolumeClaimRetentionPolicy so the data PVC is removed when an OpenBAO app is deleted, instead of being orphaned. WARNING: deleting an OpenBAO app now permanently removes its data PVC and the data it holds (scale-down still retains data).
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added configurable PVC retention behavior for OpenBao: storage is
deleted when the release is removed and retained when scaling down.
* **Tests**
  * Added a Helm unit testing workflow to run chart/unit tests.
* Introduced assertions to confirm PVC retention settings are correctly
passed through and rendered as expected.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 14:32:30 +03:00
scooby87 959a9b1fdb fix(kubernetes): clear kamaji datastore-secret finalizer to unblock namespace deletion (#3062) (#3078)
## What this PR does

After deleting a `Kubernetes` app CR / tenant, the secret
`<release>-datastore-config` kept the finalizer
`finalizer.kamaji.clastix.io/datastore-secret`, which was never removed
—
**blocking deletion of the entire namespace** (observed stuck in
`Terminating`
for 115 minutes). Kamaji adds this finalizer to the datastore connection
Secret
and normally strips it as part of TenantControlPlane (TCP) deletion; if
the TCP
is force-deleted or Kamaji does not reconcile in time, the finalizer
hangs and
the Secret can never be garbage-collected, wedging the namespace.

This extends the existing `pre-delete` cleanup hook
(`packages/apps/kubernetes/templates/delete.yaml`):

- **Step 4** now deletes the TenantControlPlane with `--wait=true
--timeout=120s`
so Kamaji's own finalizer-cleanup chain has a chance to run before we
fall
  through.
- **Step 4b (backstop)** patches `metadata.finalizers: null` on
`<release>-datastore-config` only if it still exists. By this point the
TCP is
gone, the Secret is owner-ref'd to the deleted TCP and only awaiting GC,
so
clearing the finalizer merely lets GC reap it — it cannot strip a
finalizer
while a live TCP still needs the datastore. The Secret holds only
connection
config, not tenant data (which lives in the external etcd datastore), so
no
  data is lost.
- **RBAC** adds a `secrets` rule scoped via `resourceNames` to exactly
  `<release>-datastore-config`, verbs `get`/`patch` only.

The Job is hardened: non-root (uid 65534), read-only root fs, all caps
dropped,
`seccompProfile: RuntimeDefault`, writable `/tmp` for kubectl cache.
Failures at
both steps are logged (the Step 4b warning explicitly mentions the
namespace may
stay Terminating) but the hook still completes so it cannot itself block
teardown.

This is the most impactful of the create→delete leak sweep findings:
without it,
tenant teardown hangs and the create→delete cycle requires manual
intervention.

Fixes #3062.

### Release note

```release-note
fix(kubernetes): clear the leftover Kamaji `finalizer.kamaji.clastix.io/datastore-secret` finalizer on the datastore-config Secret during tenant teardown, so deleting a Kubernetes app / tenant no longer leaves the namespace stuck in Terminating.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Extended the pre-delete cleanup workflow to more reliably remove
`TenantControlPlane` and handle slow deletions with a bounded wait.
* **Security**
* Hardened the cleanup job by running as non-root with restricted
security settings, dropped capabilities, and a safe writable temp
location.
* **Bug Fixes**
* Added a conditional backstop to remove leftover `finalizers` from the
relevant datastore Secret so namespace termination isn’t blocked.
* **Tests**
* Added a YAML test suite to validate the wait-and-patch behavior and
ensure the patch runs only after the `TenantControlPlane` is gone.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 14:25:49 +03:00
scooby87 080ceac9c0 fix(gateway): clean up orphaned ACME account secret on tenant module delete (#3089) (#3093)
## What this PR does

When the `gateway` tenant module is disabled (its HelmRelease
uninstalled), cert-manager's ACME account private-key secret
`cozystack-acme-account` was left orphaned in the tenant namespace
(label `app.kubernetes.io/managed-by: cert-manager`, no
`ownerReferences`).

This adds a `post-delete` cleanup hook
(`packages/extra/gateway/templates/hooks/cleanup-acme.yaml`) that
deletes the secret by name. Verified on a live cluster that
`cozystack-acme-account` is referenced by exactly one Issuer — the
gateway module's own `cozystack-gateway` Issuer — so it is
gateway-specific and safe to remove on gateway teardown (it is not
shared with other modules, which use the cluster-wide `letsencrypt-prod`
issuer).

The cleanup Job is hardened: non-root (uid 65534), read-only root fs,
all capabilities dropped, `seccompProfile: RuntimeDefault`, resource
requests/limits; RBAC scoped via `resourceNames` to exactly
`cozystack-acme-account` (get/delete). A failure is logged but never
blocks teardown.

No user data is affected — only cert-manager's ACME account key
(re-registered on next enable).

Fixes #3089.

### Release note

```release-note
fix(gateway): clean up the orphaned cert-manager ACME account secret (cozystack-acme-account) left behind when a tenant's gateway module is disabled.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added automatic cleanup after gateway removal to delete the stored
ACME account secret in the release namespace.
* Included a dedicated, restricted cleanup job with the required
permissions to run safely during uninstall.

* **Bug Fixes**
  * Made the gateway test target always runnable.
* Cleanup now ignores missing secrets and warns on failure instead of
stopping uninstall flows.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 14:21:42 +03:00
Aleksei Sviridkin 511f2a3c1c test(e2e): smoke-test the ServiceExposure controller
Add a deterministic e2e for the externalIPs backend: it creates an
ExposureClass, a Service with pinned externalIPs, and a ServiceExposure,
then asserts the controller resolves the class, renders no MetalLB/Cilium
pool, reports the assigned IPs, and cleans up via its finalizer on delete.

The externalIPs path is chosen so the test creates no backend pool — it
cannot collide with the cluster's existing LoadBalancer address pools or
depend on real L2 reachability. The metallb and cilium rendering paths are
covered by the Go unit tests.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-29 13:41:25 +03:00
Myasnikov DaniilandClaude Opus 4.8 9f1819cdde ci(renovate): force chore(deps) on direct Go modules via user packageRule
The top-level semanticCommitType=chore did not actually take effect for
direct dependencies. config:recommended extends
:semanticPrefixFixDepsChoreOthers, whose packageRule re-sets
semanticCommitType=fix for depTypes [dependencies, require]. Renovate
packageRules override top-level config, and the gomod manager assigns
depType "require" to direct deps — so direct-dep PRs (the dominant type)
still resolved to fix(deps) and were auto-labeled kind/bug, the exact
pollution this PR set out to remove.

Add a user packageRule (evaluated last, so it wins over the preset rule)
that forces semanticCommitType=chore / scope=deps for the gomod manager.
The top-level keys are now redundant but kept for readability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-29 15:18:26 +05:00
Myasnikov DaniilandClaude Opus 4.8 d098423f8a ci(renovate): force chore(deps) commits and add area/dependencies label
Renovate's :semanticCommits preset picks fix(deps) for stable (>=1.0)
production modules and chore(deps) for 0.x modules, so routine bumps
landed inconsistently — fix(deps) bumps were auto-labeled kind/bug,
polluting the bug list. Pin semanticCommitType to chore (and the scope
to deps) so every dependency PR is chore(deps): … and maps to
kind/cleanup.

The PR auto-labeler had no deps -> area/* mapping, so every Renovate PR
fell through to area/uncategorized. Add an area/dependencies label and
map the deps scope to it.

Also disable Renovate for kubevirt.io/csi-driver: it is pinned to an
untagged pseudo-version with no release tags, producing a permanent
no-result lookup failure on the Dependency Dashboard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-29 15:18:26 +05:00
myasnikovdaniil 90aa850c95 ci(labeler): map cozystack-ui and console scopes to area/dashboard (#3117)
## What this PR does

The cozystack-ui SPA was vendored into the monorepo in #2963 and now
lives under `packages/system/dashboard`. PRs touching the UI are likely
to use `cozystack-ui` or `console` as their Conventional Commits scope,
neither of which the PR auto-labeler recognizes today — so they fall
through to `area/uncategorized`.

This maps both scopes to the existing `area/dashboard` (already
described as "dashboard / UI"). No new label is introduced;
`.github/labels.yml` is unchanged.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Expanded automatic PR labeling to add the `area/dashboard` label when
a PR title scope is `cozystack-ui` or `console`.
* Existing label rules and parsing behavior remain unchanged for all
other scopes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 15:16:38 +05:00
Aleksei Sviridkin 6896a2a4db feat(kubernetes): expose tenant apiserver authentication in app values (#3123)
## What this PR does

The managed `Kubernetes` app (Kamaji-backed tenant control planes)
exposed no way to configure the tenant `kube-apiserver` authentication
through the application values — there was no knob for OIDC, the
structured `AuthenticationConfiguration`, extra apiserver flags, or
extra volumes. The only workaround was patching the underlying
`KamajiControlPlane` out-of-band, which is invisible to GitOps and lost
when the cluster is recreated from values.

This adds three opt-in fields under `controlPlane.apiServer`, all empty
by default so existing clusters render unchanged:

- `extraArgs` → `KamajiControlPlane.spec.apiServer.extraArgs`
- `extraVolumeMounts` → `spec.apiServer.extraVolumeMounts`
- `extraVolumes` → appended to `spec.deployment.extraVolumes`

This unblocks OIDC SSO via the `--oidc-*` flags and, more importantly,
structured authentication: `--authentication-config` mounted from a
ConfigMap/Secret plus `--requestheader-uid-headers`, which is the only
way to map a user UID for aggregated/extension apiservers.

Because the control-plane Deployment runs on the management cluster, the
passthrough is validated to keep tenant and management isolated:
`extraVolumes` accept only `configMap`/`secret` sources (host-reaching
sources such as `hostPath`/`csi` and token projection via `projected`
are rejected); each volume needs a unique, non-empty name and exactly
one source; the reserved `talos-ca`/`talos-tls-cert` names are rejected;
and each `extraVolumeMount` must reference a user-declared
`extraVolume`, so the chart-managed talos secret volumes cannot be
mounted. Secret/ConfigMap references resolve in the tenant's own
management-side namespace and are intentionally allowed — this stays
within the tenant's existing security domain.

Generated artifacts (`values.schema.json`, README, Go types + deepcopy,
ApplicationDefinition) are regenerated from the values annotations. Helm
unit tests cover the default no-op render, each field's passthrough, and
every rejection path.

Addresses #3074.

### Screenshots

N/A — no UI changes.

### Release note

```release-note
feat(kubernetes): expose tenant apiserver authentication via controlPlane.apiServer.extraArgs / extraVolumes / extraVolumeMounts, enabling OIDC and structured AuthenticationConfiguration on tenant clusters
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for customizing the Kubernetes API server with extra
command-line arguments, additional volume mounts, and extra volumes.
* Updated chart values and schema so these options can be configured
directly.

* **Bug Fixes**
* Added validation to prevent invalid or conflicting volume names and
disallowed volume sources.
* Ensured mounts only reference declared extra volumes and reserved
control-plane names can’t be overridden.

* **Tests**
* Added coverage for default behavior, valid customization paths, and
invalid configuration scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 13:13:54 +03:00
myasnikovdaniil 80e3fa9cbe fix(dashboard): render error and unknown-type states instead of an infinite spinner (#3098)
## What this PR does

Two console pages ordered their loading guard ahead of the terminal
state, making the latter unreachable. Both are pre-existing upstream
behaviors carried in by the cozystack-ui vendoring (#2963) and flagged
as non-blocking findings in that PR's review.

- **`ApplicationDetailPage`** checked `isLoading || !instance || !ad`
before `if (error)`. On a failed GET (403/404/5xx) `isLoading` is false
and `instance` stays undefined, so the page rendered the loading spinner
forever and never reached the "not found" branch. The error branch now
runs first.
- **`ApplicationListPage`**'s `defsLoading || (!ad && plural)` guard
stayed true for any unrecognized `plural` once definitions had loaded,
so the "Unknown application type." branch was unreachable. The guard now
only spins while definitions are loading.

## Verification

- Added co-located regression tests for the failed-GET (detail) and
unknown-plural (list) paths.
- `pnpm typecheck` clean; full console suite green (`pnpm test`: 317
passed, +4 new).

```release-note
fix(dashboard): the console now shows a not-found message on a failed application GET and an unknown-type message for an unrecognized resource, instead of spinning forever
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved loading and error handling on application pages so users see
the correct message at the right time.
* “Unknown application type” now appears only after application
definitions finish loading, instead of being masked by a loading state.
* Application detail pages now show a “not found” message when the
requested app can’t be loaded, rather than continuing to display a
spinner.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 14:02:49 +05:00
Myasnikov DaniilandClaude Opus 4.8 df221f98d4 ci(labeler): map cozystack-ui and console scopes to area/dashboard
The cozystack-ui SPA was vendored into the monorepo (#2963) and now lives under packages/system/dashboard. Map its likely PR title scopes to the existing area/dashboard so UI PRs no longer fall through to area/uncategorized.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-29 14:02:29 +05:00
myasnikovdaniil 61a574393a build(dashboard): pin console image builder to pnpm 11.1.2 and repoint vendor doc link (#3099)
## What this PR does

Two post-vendor cleanups from the #2963 review:

- **Pin the console image builder to pnpm 11.1.2.** The `Containerfile`
ran `corepack prepare pnpm@latest`, so the container build could drift
to a newer pnpm major than the `pnpm@11.1.2` pinned in `package.json`
and used by CI. `--frozen-lockfile` guards the lockfile, but pinning the
builder makes the image build reproducible and consistent with CI.
- **Repoint a stale doc link.** `docs/storage-immutability.md`
referenced the now-vendored `cozystack/cozystack-ui#6`; it now points at
the in-tree console path.

## Verification

- No source change; `pnpm typecheck` + `pnpm test` unaffected.
- Builder pnpm version now matches `package.json`'s `packageManager`
field.

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Clarified the storage immutability documentation by updating the UI
reference to an in-repo path.

* **Chores**
* Standardized the build environment to use a specific pnpm version,
helping keep installs and builds consistent.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-29 14:02:28 +05:00