source-watcher reconciles one ArtifactGenerator per PackageSource and
builds the resulting ExternalArtifacts. On the management cluster that is
~100 artifact builds during install, and the platform HelmRelease blocks
until all of them are Ready.
It was the only Flux controller left untuned. In the bootstrap all-in-one
pod it ran without an explicit --concurrent (binary default 10) and with
the shared 100m CPU request; in the production FluxInstance it was missing
from the --concurrent / CPU patch that source-, kustomize- and
helm-controller already receive. Under load its per-artifact builds finish
at staggered times, leaving a random tail of not-yet-Ready artifacts that
times out the platform install.
Raise source-watcher to --concurrent=20 in both deployments. In the
bootstrap all-in-one pod also give it a 500m CPU request; the pod has no
CPU limits, so the request is what drives its CPU share under contention.
In the production FluxInstance add it to the shared reconciler patch, so it
now receives the same --requeue-dependency=5s and 2000m/2048Mi limits the
other tuned controllers already have.
The flux-aio module exposes a resources override for the watcher (set via
CUE) but no args override, so the bootstrap --concurrent bump is a
post-render patch in the Makefile.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
ComputePools derived a pool's Available with SubtractWithNonNegativeResult over
the union of the budget and carve-out keys, so a bounded child carving a
resource its parent does not bound surfaced that resource as 0 in the parent
pool's Available. The controller then wrote a tenant-quota-allocated clamping
the parent namespace to zero of a resource it never limited, blocking its
workloads (observed on a cluster: a memory-bounded child froze its cpu-only
parent's memory). Mask Available to the budget's resource names so only
resources the pool root actually bounds are enforced; mirrors the fix already
applied to Pool.Overcommitted.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
Pool.Overcommitted reported a resource as overcommitted whenever a sub-tenant
carved out a resource the pool root does not bound. Admission explicitly allows
a child to bound a resource its parent leaves unbounded, and an unbounded budget
can never be overcommitted, so this produced a spurious QuotaOvercommitted
warning event on every reconcile. Restrict the check to resources present in
the pool budget.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
Take a tenant's declared budget from its chart-rendered tenant-quota
ResourceQuota (.spec.hard) instead of the raw tenant.spec.resourceQuotas
values. The chart expands the shorthand values into real quota resource
names and applies the cluster's request/limit allocation ratios, so
reading the rendered object keeps the controller-owned allocated quota in
exactly the same key space — the two quotas compose correctly and the
controller never replicates that cluster-configurable transformation.
Also skip pools that need no tightening beyond the chart quota (a lone
tenant with no sub-tenants and no carve-outs): the controller writes an
allocated quota only when a budget is actually shared or carved up, so a
single-tenant namespace is left exactly as the chart renders it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Add a tenant quota reconciler to cozystack-controller that realises the
hierarchical (sub-tree) quota model at runtime. Each tenant's declared
quota is the budget for its whole sub-tree: a child that declares its own
quota carves a fixed slice out of the parent budget, while a child without
a quota shares the parent's remaining pool.
The controller maintains a per-namespace 'tenant-quota-allocated'
ResourceQuota (Kubernetes enforces the most restrictive quota, so it binds
without fighting Flux over the chart-rendered 'tenant-quota'), aggregating
sub-tree usage so a pool's members collectively stay within the budget. A
--tenant-quota-buffer-percent flag temporarily inflates pool budgets so
workloads already over a freshly-introduced quota keep running during
rollout; overcommitted pools emit a QuotaOvercommitted event.
The model and the selector-spans-namespaces/aggregated-usage approach are
adapted from OpenShift's ClusterResourceQuota controllers (openshift/api,
library-go, cluster-policy-controller, apiserver-library-go), Apache-2.0,
Copyright Red Hat, Inc. and the OpenShift Authors.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
## What this PR does
Introduces a tenant-facing `SecurityGroup` resource in a new
`sdn.cozystack.io/v1alpha1` API group, served by the Cozystack
aggregated API server as a 1:1 projection of a `CiliumNetworkPolicy`. A
SecurityGroup attaches to one of the tenant's managed applications by
reference (`spec.targetRef: {apiGroup, kind, name}`) and declares
allow-list ingress/egress for that application. Tenants manage network
policy for their own apps entirely through
`securitygroups.sdn.cozystack.io`, without being granted any access to
the `cilium.io` API group.
The translation is synchronous and stateless — the API server converts
each SecurityGroup into a CiliumNetworkPolicy in the same namespace on
write and back on read; there is no controller. The backing policy's
`endpointSelector` is **derived** from `targetRef` (matched on the
application's lineage labels
`apps.cozystack.io/application.{group,kind,name}`), never copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
Policies owned by this API carry an `sdn.cozystack.io/securitygroup`
marker label that the storage always re-asserts and hides from the view,
so it only ever surfaces or mutates its own policies and never touches
platform-managed network policies.
The value is self-service network policy on a Cozystack-native,
RBAC-scoped surface, without exposing Cilium's version-coupled CRDs to
tenants. Because the selector is derived from an authorized application
reference rather than free-form tenant input, the RBAC boundary is
structural, not selector-validated: a tenant cannot express network
policy against arbitrary or platform-owned pods. The full design —
selector derivation from lineage labels, the structural boundary, the
additive-policy reality (a SecurityGroup widens a managed application's
allowed traffic but cannot sever its platform-managed management plane,
and equally cannot tighten below the platform baseline), the in-tree
CiliumNetworkPolicy mirror that avoids a Cilium Go-module dependency,
RBAC, and deferred extensions (raw-pod targeting, cluster scope,
reusable groups) — is in `pkg/apis/sdn/DESIGN.md`.
Implements the attachment model from #1614.
### Release note
```release-note
feat(api): add the SecurityGroup resource (sdn.cozystack.io/v1alpha1) — a tenant-facing, namespace-scoped firewall that attaches to a managed application by reference and projects 1:1 onto a CiliumNetworkPolicy with a machine-derived endpoint selector, letting tenants manage their application's network policy without cilium.io access.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the `SecurityGroup` API resource (`sdn.cozystack.io` v1alpha1)
with a namespaced projection backed by Cilium network policies.
* Enabled the SDN `APIService` and wired SDN routing, scheme, and REST
support.
* Expanded RBAC to include SecurityGroup access and required Cilium
policy permissions.
* **Documentation**
* Added a design document describing projection behavior and rule
semantics.
* **Bug Fixes**
* Improved API rule validation coverage for missing list-type fields.
* **Tests**
* Added SDN round-trip, fuzzing, RBAC, and comprehensive REST
create/get/list/update/delete/watch/resource-version tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Lays the foundation for migrating tenant LoadBalancer exposure off the
deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off
MetalLB onto Cilium-native announcement — the mechanism-agnostic shape
prescribed when the earlier chart-side approach (#2468) was reverted in
#2603.
It adds a new `network.cozystack.io/v1alpha1` API group and a controller
that translates intent into whichever LoadBalancer backend the cluster
admin configured:
- `ExposureClass` (cluster-scoped, admin): binds a logical class name to
a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP
address scope — the StorageClass/IngressClass pattern.
- `ServiceExposure` (namespaced, tenant intent): "expose this Service
via class X"; status reports the resolved backend and assigned IPs.
- A `serviceexposure` controller renders the backend's pool + announcer
once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium
`CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to
the namespaces that opted in. Backends are pluggable behind a small Go
interface, so adding BGP or another cloud LB is one implementation, not
an API change. The controller never mutates the target Service — the
chart still owns its shape.
Platform wiring: `publishing.exposureClass.*` renders the admin
`ExposureClass` and the host ingress publishes through it. The default
(no class set, `externalIPs`) leaves every existing cluster's behaviour
byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes
a single `ExposureClass.spec.backend` change.
This is the foundation only — Part of #2586. Converting per-app
exposure, flipping Cilium `l2announcements` on, changing the default,
and removing MetalLB are deferred to later phases.
### Release note
```release-note
feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added new `ExposureClass` and `ServiceExposure` custom resources with
support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB),
including optional L2 and address configuration.
* Added Helm wiring and templates to enable service/ingress publishing
via `ServiceExposure`, selecting a configured (or default)
`ExposureClass`.
* **Bug Fixes**
* Improved reconciliation, finalizer-based cleanup, and status updates
for readiness and assigned IP reporting, including safer handling when
backends or managed resources change.
* **Tests**
* Added end-to-end and unit test coverage for backend rendering,
readiness, and cleanup flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The reconciler intentionally treats a malformed attachments annotation and a
failed pod-mapping List as empty / no-op to stay resilient, but it swallowed
both errors silently. That hides real defects: a bad annotation writer, or a
transient API error that drops a pod update until the next periodic resync with
no trace. Log both while preserving the resilient return behavior.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add the serviceexposure controller to the cozystack-controller binary. It
resolves a ServiceExposure's ExposureClass to a backend and renders the
backend-specific pool and announcer CRs, then reports the assigned IPs in
status.
Backends are pluggable behind a small interface, one per file:
- externalIPs: renders nothing (the node owns the address) — preserves the
historical default behaviour.
- metallb: IPAddressPool + L2Advertisement in cozy-metallb, scoped to the
target Service's namespace.
- cilium: CiliumLoadBalancerIPPool (cilium.io/v2) + CiliumL2AnnouncementPolicy
(cilium.io/v2alpha1), scoped to the Service via serviceSelector.
- robotlb: renders nothing (the cloud LB owns allocation and announcement).
The rendered CRs belong to foreign API groups not in the Go scheme, so they
are emitted as unstructured objects. They carry no OwnerReference (the
Cilium CRs are cluster-scoped and the MetalLB CRs live in a different
namespace than the owning ServiceExposure), so ownership is tracked via a
management label plus owner annotations, cleanup is driven by a finalizer,
and the controller refuses to take over an identically-named resource it
does not manage. The controller never mutates the target Service, leaving
its shape to the chart that renders it.
Grant the controller write access to metallb.io and cilium.io pool /
announcer resources.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add a controller that keeps each SecurityGroup's membership label
(securitygroup.sdn.cozystack.io/<name>) in sync with its attachments: it
stamps the label onto the pods of every attached application and removes it
when an application is detached or the SecurityGroup is deleted. The backing
CiliumNetworkPolicy's endpointSelector matches that label, so this is what
makes a SecurityGroup apply to its members.
The controller watches the marked backing policies and managed-app pods. It
resolves each attachment through the lineage labels the lineage webhook
already stamps — never the attachment list directly — and only ever labels
pods in the SecurityGroup's own namespace, using single-key merge patches so
it cannot clobber another group's label or a pod's lineage labels. A finalizer
on the backing policy guarantees the labels are stripped before the policy is
removed. Informer caches are bounded to managed pods and SecurityGroup-owned
policies.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Implements the tenant Flux sharding design from #2817: a new
`flux-shard-operator` system package that spreads tenant HelmReleases
across multiple helm-controller shards, so a noisy tenant (e.g. a
HelmRelease stuck in infinite remediation) cannot degrade the others.
- **Placement controller** owns the tenant→shard assignment: the unit of
placement is the tenant (all of its HelmReleases share one shard label),
distribution is greedy least-loaded weighted by HelmRelease count, with
pinned tenants, threshold-gated rebalance, paced moves, and never moving
deleting tenants. Watches are metadata-only, so the controller does not
decode the helm-controller status-patch firehose. Assignments are
recorded on tenant namespaces; HelmRelease labels remain the source of
truth on restarts.
- **Shard runtime** is provisioned by the operator: `shardCount`
helm-controller Deployments cloned from the flux-aio `flux` Deployment
and sanitised, so the image and feature-gates stay version-synced
automatically. flux-aio itself is unchanged and keeps owning everything
without a shard key.
- **CREATE-only mutating webhook** (`failurePolicy: Ignore`) stamps the
tenant's shard onto every HelmRelease at admission, so each one is born
on the correct shard regardless of creation path; any miss degrades
gracefully to the placement controller.
- The hand-rolled `flux-tenants` Deployment is retired: the operator
drains it (backfill `tenants` → `shard<i>`) and deletes it once empty;
migration 44 removes a leftover only under the same guard.
Ships with `shardCount: auto` (the default): the operator sizes the
shard count from the tenant HelmRelease count — `K = clamp(ceil(H/100),
1, min(16, T))` with hysteresis (scale up past 120 HR/shard, down under
60) — so small clusters stay at one shard (≈ today's behaviour) and
large fleets shard out automatically. An integer value pins the count
explicitly.
One deviation from the design doc: the webhook/controller scope is the
`sharding.fluxcd.io/key` label rather than
`internal.cozystack.io/tenantmodule` — every tenant HelmRelease is born
with `key=tenants` (stamped by ApplicationDefinition `release.labels`
and chart templates), so the webhook rewrites it and the placement
controller continuously relabels stragglers; `tenantmodule` covers only
the 6 tenant chart modules.
### Screenshots
Not a UI change.
### Release note
```release-note
feat(fluxcd): tenant HelmReleases are now reconciled by operator-managed helm-controller shards; the flux-shard-operator balances tenants across `shardCount` instances and retires the legacy flux-tenants deployment
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced **flux-shard-operator** to shard helm-controller workloads
with deterministic tenant placement, optional rebalance controls, and
autosized shard count.
* Added a **CREATE-only mutating webhook** to stamp the correct shard
label on tenant HelmReleases.
* **Bug Fixes**
* Application updates now preserve existing **flux-shard** label
assignments to prevent unwanted shard re-stamping.
* **Chores**
* Added platform packaging for the new component, including an image
build step.
* Removed the legacy tenants deployment and added a migration to retire
it safely.
* **Tests**
* Expanded unit and chart tests for placement, provisioning, webhook
behavior, and configuration parsing.
* **Documentation**
* Added operator documentation describing configuration and metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolve go.mod conflict: keep main's go 1.26.4 and golang.org/x/* bumps, and keep gomodules.xyz/jsonpatch/v2 v2.4.0 as a direct dependency (imported by the flux-shard-operator admission webhook) instead of main's indirect entry. go mod tidy leaves go.sum consistent.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
BuildShardDeployment clones the flux-aio pod spec, which carries a required podAntiAffinity on app.kubernetes.io/name=flux. Inherited onto a shard pod it forbids scheduling on the flux-aio node, so on a single schedulable node every shard stays Pending and the tenant plane stalls (no ready shard -> assignments deferred -> HelmReleases stuck on key=tenants).
Drop the podAntiAffinity during sanitisation while keeping any benign nodeAffinity (e.g. os=linux). The fixture now carries flux-aio's affinity and TestBuildShardDeployment asserts the strip.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The XValidation justification was factually wrong: the deployed
helm-controller CRD has no rule rejecting a Strategy alongside a
Remediation entry (only .retryInterval is gated on Strategy.Name), so
the claimed "silently break every HR" safety net does not exist.
Reword the comment in all three test files to the real reason
Remediation stays nil: retries are driven by a single
Strategy.RetryInterval path, matching cozystack-operator's
PackageReconciler.buildHelmReleaseSpec.
Address review feedback from lexfrei on rest_helmrelease_spec_test.go,
rest_timeout_test.go and internal/operator/package_reconciler_test.go.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replacing the whole resources block dropped cloned values the override
did not name (e.g. the inherited cpu limit when only a memory limit is
configured), contradicting the documented inherit-when-empty semantics.
Also document the intentional provisioner-vs-placement sizing
asymmetry.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
ComputePlacement ignores pins to non-existent shards, but rebalance
treated any pinned tenant as sticky, so an invalid pin blocked expected
moves. Honor only pins placement itself honors.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
A failed patch no longer refreshes the rebalance cooldown or inflates
the moves counter, so retries are not delayed and not double-counted.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The Update handler rebuilds the HelmRelease from the Application, which
re-stamped sharding.fluxcd.io/key with the ApplicationDefinition default
and bounced the HelmRelease off its operator-assigned shard on every
update. Carry the live label over from the current HelmRelease instead.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Operators who already hold a wildcard TLS certificate (purchased, or
issued by a corporate CA) had no way to make a tenant Gateway serve
under it: the only cert modes were http01 (per-listener ACME) and
dns01 (controller-minted wildcard via ACME).
Add a third mode, existingSecret, that references a pre-existing TLS
Secret in the TenantGateway's own namespace via spec.wildcardSecretRef.
In this mode the controller mints no ACME Issuer and no Certificate;
the wildcard, apex, and per-child-apex HTTPS listeners reference the
supplied Secret directly. A missing wildcardSecretRef fails fast with
Ready=False rather than producing a Gateway pointing at a nonexistent
Secret. Switching into existingSecret from http01/dns01 reclaims the
now-unused Issuer and Certificates so no ACME machinery is left behind.
The non-cert listeners (http->https redirect, TLS passthrough) are
unaffected and continue to render.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
--shard-count now accepts "auto" (the new default) or an explicit
positive integer. Auto mode drives the shard count from the tenant
HelmRelease count — K = clamp(ceil(H/100), 1, min(16, T)) — applied
with hysteresis anchored on the currently provisioned shards (scale up
eagerly past 120 HR/shard, scale down lazily under 60 HR/shard) so K
does not flap around sizing boundaries.
The per-shard target is 100 HelmReleases (down from the 150 in the
design doc) to leave headroom for existing tenants growing their
HelmRelease count without an immediate reshard.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
E2E caught the cloned shard crashlooping on Talos: the installer injects
the node-local KubePrism endpoint (KUBERNETES_SERVICE_HOST=localhost,
KUBERNETES_SERVICE_PORT=7445) into hostNetwork workloads, and the clone
carried it into a non-hostNetwork pod where localhost is dead. Sanitise
those env vars away so shards use the in-cluster defaults.
Defense in depth for the same failure mode: the placement controller now
defers reassignments whose target shard Deployment has no ready replica,
so a backfill can never hand tenants to a helm-controller that is not
actually running, and reacts to shard readiness changes directly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The installer no longer ships the hand-rolled tenant helm-controller
shard: flux-shard-operator provisions operator-managed shard
Deployments cloned from flux-aio at runtime instead, and deletes a
running flux-tenants Deployment only after the legacy tenants bucket
drains. flux-aio itself stays unchanged and keeps owning everything
without a shard key; the Makefile lines that synced the image/version
into the removed manifest go away with it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Three leader-elected/served parts on metadata-only watches, so the
operator never decodes the helm-controller status-patch firehose:
- placement controller: singleton full-sync reconciler that rebuilds
the tenant index from HelmRelease labels (restart-safe), records the
assignment on the tenant namespace, self-heals HR labels and paces
reassignments so a backfill never floods a target shard;
- shard provisioner: reconciles helm-controller-shard<i> Deployments
cloned from the flux-aio helm-controller container and sanitised
(no host networking, no localhost cross-container wiring, per-shard
watch-label-selector), prunes drained extra shards and retires the
legacy flux-tenants Deployment once the tenants bucket is empty;
- CREATE-only mutating webhook: stamps the tenant's recorded shard on
newborn HelmReleases so each one is born on the correct shard; every
miss is permissive and falls back to the placement controller.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Tenant-granular placement for sharding the management-cluster tenant
helm-controller: tenant key extraction from HelmRelease metadata,
greedy least-loaded assignment weighted by HelmRelease count (with the
N-tenants-over-N-shards 1-per-shard guarantee), minimal-movement
rescale, threshold-gated rebalance with per-tenant cooldown hook,
pinned tenants, and the autosizing recommendation formula.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
## What this PR does
Upgrades Flux v2.7.3 → v2.8.0 across both the vendored tenant chart and
the embedded management-cluster manifests, and folds in the chart fixes
that become hard errors under the new strict helm-controller v1.5.
Flux v2.8's helm-controller v1.5.0 ships:
- **Server-Side Apply with `--force-conflicts`** — strict CRD schema
enforcement; misplaced fields (silently dropped on v2.7) now fail the
apply.
- **kstatus-based health checking** by default — parent HR waits for
every applied resource (including child HRs) to be Ready before flipping
its own Ready, surfacing latent ordering deadlocks.
- **Helm v4 embedded** — `upgrade.force: true` is deprecated and now
conflicts with SSA.
- **`HelmRelease.spec.healthCheckExprs`** — prerequisite for proper
readiness gating (used in PR #2601 split).
Folds in PR #2612 (kubevirt-instancetypes null TPM fix) since the same
Flux upgrade triggers it.
## Commits
**Flux upgrade itself:**
- `feat(fluxcd)`: bump `flux-operator` / `flux-instance` vendored charts
to v0.48.0; web UI opt-in.
- `feat(flux)`: regenerate embedded management-cluster manifests via
`make update` in `packages/core/flux-aio` (timoni bundle build).
**Chart fixes for strict SSA — fields the chart sent that v2.7 silently
dropped, v2.8 rejects:**
- `fix(kubevirt-instancetypes)`: drop persistent strip that produced
null `preferredTPM` (folds #2612).
- `fix(foundationdb)`: relocate `faultDomain`, `imageType`, `labels`,
`minimumUptimeSecondsForBounce` from inside `automationOptions` to
direct children of `spec`.
- `fix(kafka)`: place `enableServiceLinks` under `template.pod`, not a
phantom `template.spec`.
- `fix(vm-instance)`: emit `disk: {}` (not `disk:`/null) when no bus is
set.
- `fix(platform)`: drop deprecated `upgrade.force: true` from
HelmReleases; fix `kafka` WorkloadMonitor `replicas` paths.
**Ordering / deadlock fixes under v2.8 kstatus:**
- `fix(vpa)`: break circular wait between parent install and nested
`vpa-for-vpa` HR.
- `fix(kubernetes)`: drop lookup-guarded parent-HR `dependsOn` on tenant
addon child HRs (parent waits on child via kstatus, child waited on
parent — deadlock).
**E2E waits for v2.8 kstatus timing:**
- `test(e2e)`: bump app HR-Ready waits to 5m (was 20s–100s under v2.7's
faster dispatch).
- `test(e2e)`: wait for parent HR Ready before downstream asserts in
`run-kubernetes.sh` and `vminstance.bats`.
## Scope discipline
This PR is part of the split of #2619 (the consolidated CI fixes branch)
into review-friendly pieces. Companion PRs:
- **PR #2601** (seaweedfs split) — folded into this PR (commits
`29c6afc8`, `0e8b46d7`, `7157158c`, `dccdeb52`, `f880b324`): the
seaweedfs-system → seaweedfs-db + seaweedfs-system split, its adoption
migration 43 (targetVersion 44), and the configurable db resources all
land here, because the strict-SSA `upgrade.force` removal and the
kstatus parent-HR timeout bump only make sense together with the split.
#2601 is superseded.
- **PR #2558** (drop 3× retry on `Run E2E` + `Install Cozystack`) —
independent, lands separately.
- Several smaller standalone fixes lifted out of #2619 (startup probes,
cert-manager `dependsOn`, prepull machinery, CSI HR timeout, NFS/OIDC
test improvements) — opened as separate PRs.
## Verification
- `helm template` renders cleanly for both `fluxcd` and
`fluxcd-operator` packages with `web.enabled=false` (default) and
`web.enabled=true`.
- Embedded `cmd/cozystack-operator` binary contains the v1.5.0 / v1.8.0
/ v2.1.0 controller image strings.
- No references to the v0.39-removed `--disable-wait-interruption` flag
anywhere in `packages/` or `internal/`.
### Release note
```release-note
Flux upgraded to v2.8.0 (helm-controller v1.5 — Helm v4 Server-Side Apply with --force-conflicts, kstatus health checking).
When upgrading existing clusters:
- Kubernetes 1.33+ is now required for the platform (management) cluster, and for any tenant cluster that enables the optional (default-off) Flux addon — that addon ships the bumped Flux too.
- HelmReleases no longer set `upgrade.force: true`. Helm v4 SSA resolves field-ownership conflicts automatically (`--force-conflicts`), but that is not the old client-side replace: immutable-field changes (e.g. StatefulSet volumeClaimTemplates/serviceName) no longer self-heal and require manual recreation — delete the object (e.g. `kubectl delete sts <name> --cascade=orphan`) and let Flux recreate it.
- KubeVirt: persistent TPM/EFI is re-enabled for the Windows 11/2k22/2k25 preferences (KubeVirt 1.8 VMPersistentState); each affected VM provisions an extra RWO backend-storage PVC from the default StorageClass.
- KubeVirt: the EOL centos.7*/centos.stream8* preferences are retained as deprecated, hidden aliases (`tags: hidden`, `instancetype.kubevirt.io/deprecated: "true"`) — existing VMInstances on these profiles keep rendering and need no action on upgrade, but the profiles are no longer offered for new VMs; repoint to centos.stream9/10 when convenient. The gn1.* GPU instancetypes are likewise retained.
- FoundationDB: imageType now reaches the operator (silently dropped pre-SSA); it is pinned to `split` to match the value existing clusters effectively ran, so upgrades stay non-disruptive. Set `imageType: unified` to migrate deliberately.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Flux Status web UI: optional web server, config secret, service port,
HTTPRoute/Ingress, network policy, and RBAC roles.
* New SeaweedFS DB Helm chart and optional managed DB release.
* **Improvements**
* CRD/schema enhancements: new provider kinds, validations, variant
option, and external checksum refs.
* Raised Kubernetes prerequisite to 1.30+; extended e2e timeouts for
reliability.
* **Chores**
* Bumped Flux Operator and Flux versions; documentation links updated to
fluxoperator.dev.
* **Bug Fixes**
* Removed aggressive HelmRelease force-upgrade/install flags.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2602?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address non-blocking review feedback from lexfrei: the flux-tenants
sharded-controller Deployment carried a stale app.kubernetes.io/version: v2.7.3
while running helm-controller v1.5.0 (Flux 2.8.0). The flux-aio Makefile update
target only synced the container image, so the version label drifted on every
bump and make update never self-corrected. Bump the label to v2.8.0 and extend
the yq patch to carry app.kubernetes.io/version from the source flux Deployment.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Re-render internal/fluxinstall/manifests/fluxcd.yaml via
`make update` in packages/core/flux-aio (timoni bundle build against
oci://ghcr.io/stefanprodan/modules/flux-aio at the latest tag) so
the cozystack-operator binary embeds Flux 2.8.0 instead of 2.7.3.
Auto-syncs the helm-controller image into fluxcd-tenants.yaml.
Embedded versions now match the chart bump in the prior commit:
flux: v2.7.3 -> v2.8.0
helm-controller: v1.4.3 -> v1.5.0 (enables healthCheckExprs)
source-controller: v1.7.3 -> v1.8.0
kustomize-controller: v1.7.2 -> v1.8.0
notification-controller:v1.7.4 -> v1.8.0
source-watcher: v2.0.2 -> v2.1.0
Verified by rebuilding cmd/cozystack-operator and running `strings`
on the binary -- the expected v1.5.0 / v1.8.0 / v2.1.0 image strings
are now embedded in place of the v1.4.x / v1.7.x ones.
Structure preserved:
- single AIO Deployment named "flux" in namespace cozy-fluxcd
- all 5 controllers + source-watcher as containers in one pod
- custom tolerations from flux-aio.cue intact
- Namespace pod-security label still "privileged" via the
Makefile yq post-processing step
Net diff is -3664 lines (mostly CRD schema simplification in 2.8).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Adds a `ValidatingAdmissionPolicy` that blocks `DELETE` on critical
platform
objects labeled `platform.cozystack.io/no-delete=true`. Native,
in-process CEL
evaluation — no webhook DaemonSet, no Service, no TLS, no image to
maintain.
Requires Kubernetes 1.30+.
This is an alternative to the closed#2402 (admission webhook approach).
### Protected objects (labeled in this PR)
| Resource | Kind | Source |
|---|---|---|
| `cozy-system` | Namespace |
`packages/core/installer/templates/cozy-system-{namespace,labels}.yaml`
|
| `tenant-root` | Namespace |
`packages/system/cozystack-basics/templates/tenant-root.yaml` |
| `tenant-root` | HelmRelease (ns: tenant-root) | same |
| `cozystack-version` | ConfigMap (ns: cozy-system) |
`packages/core/platform/templates/cozystack-version.yaml` |
| `cozystack-packages` | OCIRepository (ns: cozy-system) |
`packages/core/platform/templates/repository.yaml` |
| `letsencrypt-prod`, `letsencrypt-stage`, `selfsigned-cluster-issuer` |
ClusterIssuer |
`packages/system/cert-manager-issuers/templates/cluster-issuers.yaml` |
| `linstorcluster` | LinstorCluster |
`packages/system/linstor/templates/cluster.yaml` |
| `packages.cozystack.io`, `packagesources.cozystack.io` | CRD | label
stamped in `internal/crdinstall/install.go` |
### Bypass
```bash
kubectl label <kind> <name> platform.cozystack.io/no-delete-
kubectl delete <kind> <name>
```
### Why VAP instead of a webhook
Same outcome as #2402 but with zero runtime infrastructure: no
DaemonSet, Service,
TLS certificate, or image. The kube-apiserver evaluates the CEL
`expression: "false"` in-process and denies because `validationActions:
[Deny]`.
`objectSelector` on the binding scopes evaluation to labeled objects
only — every
other DELETE in the cluster is unaffected.
### Release note
```release-note
feat(platform): add deletion-protection guardrail (ValidatingAdmissionPolicy)
blocking DELETE on cozystack platform objects labeled
`platform.cozystack.io/no-delete=true`. Bypass by removing the label first.
Requires Kubernetes 1.30+.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Deletion-protection guardrail: objects labeled
platform.cozystack.io/no-delete=true are denied DELETE requests; removal
of the label allows deletion.
* **Chores**
* Platform migration to v43 backfills the no-delete label onto system
resources and namespaces.
* **Tools**
* Added an "unprotect" teardown utility to remove the no-delete label
for uninstall/disaster recovery.
* **Documentation**
* Helm NOTES now warns when the guardrail is inactive due to missing
cluster API.
* **Tests**
* New tests and e2e checks validate deletion-protection behavior.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2650?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What
Implements the `strategy.backups.cozystack.io/Job` backup strategy — a
generic, application-agnostic driver that runs a user-supplied
Kubernetes Job to create (and restore) a backup.
## Background
This PR began as a draft written against an earlier shape of the backups
API. The API has since moved to a `BackupClass`-based dispatch model,
which broke the original draft on three counts:
- `BackupJob.Spec.StrategyRef` and `BackupJob.Spec.StorageRef` were
removed; the strategy and its parameters are now resolved from a
`BackupClass` into a `ResolvedBackupConfig`.
- Strategy handling moved from standalone per-strategy reconcilers to a
dispatch model: `BackupJobReconciler.Reconcile` /
`RestoreJobReconciler.Reconcile` resolve the strategy and switch into
per-strategy methods (`reconcileVelero`, `reconcileCNPG`,
`reconcileAltinity`, …).
- The draft's template context (`.Values`, `.Storage`) diverged from the
convention the other strategies adopted, and its `.Storage` half
targeted a storage API that does not exist.
## What changed
- **`reconcileJob` / `reconcileJobRestore`**
(`internal/backupcontroller/jobstrategy_controller.go`): replaces the
standalone `BackupJobStrategyReconciler` with the two dispatch methods
the `BackupJob` / `RestoreJob` reconcilers call. The Job strategy is the
generic counterpart of the Altinity driver — same lifecycle (render
`PodTemplateSpec` → run a one-shot `batch/v1.Job` → poll its terminal
condition → emit a `Backup`), minus the ClickHouse-specific
`applicationRef` gate, so it runs for any application. Shared package
helpers (`renderJobTemplate`, `jobConditionState`,
`getApplicationUnstructured`, status helpers, deterministic Job naming,
controller refs) are reused.
- **Template context** follows the in-tree convention used by the other
drivers: `.Application` (live application object), `.Release.Name` /
`.Release.Namespace`, `.Mode` (`backup` / `restore`), `.Parameters`
(resolved `BackupClass` parameters), and `.Backup` (restore runs only).
Strategy parameters round-trip through the `Backup`'s `driverMetadata`
so a later restore re-renders against the same values.
- **Restore** re-renders the same template with `.Mode == "restore"`.
Cross-namespace restore is unsupported (the RestoreJob target reference
is namespace-local), matching the Altinity driver.
- **`JobSpec.Template` doc + regenerated CRD**: the field doc previously
advertised `.Values`; it now documents the real context, and
`strategy.backups.cozystack.io_jobs.yaml` is regenerated to match (the
change is limited to that field's description).
- **Tests** (`jobstrategy_controller_test.go`): backup-Job creation /
args / labels / status, completion → `Backup` artifact with parameter
round-trip, Job-failure surfacing, missing-strategy terminal failure,
restore into the target namespace, restore completion, and the
parameter-prefix unit. A deliberately generic application kind pins that
the driver never gates on `applicationRef.kind`.
No RBAC changes are required: the controller already holds
`strategy.backups.cozystack.io/*`, `batch/jobs`, `apps.cozystack.io/*`,
and `backups` permissions.
## Notes for reviewers
- Backup and restore share one template, so a strategy author's
`PodTemplateSpec` must branch on `.Mode` to do anything
restore-specific. This matches the existing convention; worth confirming
it's the intended contract for the Job strategy's docs/examples.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Implemented full backup/restore job strategy with deterministic Job
creation, idempotent execution, and status/condition updates; added demo
example scripts and reusable shell helpers for a NATS JetStream
backup/restore workflow.
* **Documentation**
* Clarified Job template rendering context and listed supported template
variables; added README and step-by-step scenario docs for the NATS
example.
* **Tests**
* Added comprehensive unit tests covering job creation,
completion/failure handling, parameter extraction, and unmappable-target
failure cases.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/1721?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review feedback on the Job backup strategy:
- Assert .Application is templated: the rendered backup Job now reads
.Application.spec.replicas, the load-bearing context key that lets a
generic backup tool read live app spec (previously only .Release,
.Mode, .Parameters and .Backup had coverage).
- Pin failure-path messages so a reordered error path can't pass on the
wrong branch: the missing-strategy test asserts the strategy name, and
the unmappable-kind tests assert "kind not registered".
- Document why the full Status().Update in the success branches is safe
(the object is freshly Get() each pass; the StartedAt write requeues).
- Note the fixed batch/v1.Job backoffLimit=2 and link the follow-up to
make it configurable (#2742).
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Address review feedback from @lllamnyp on internal/crdinstall/install.go:
the deletion-protection label loop was untested — TestInstall_appliesAllCRDs
only checked that Install returned no error, so a refactor dropping the loop
would have shipped green and silently disabled VAP coverage on every Cozystack
CRD.
- Extend TestInstall_appliesAllCRDs to read each applied CRD back from the fake
client and assert platform.cozystack.io/no-delete=true is present.
- Add TestInstall_preservesExistingLabels: feeds in a CRD whose manifest already
carries app.kubernetes.io/managed-by=foo, asserts both that label and the
deletion-protection label end up on the applied object. Locks down the merge
contract in install.go:92-101 (the GetLabels()==nil branch is still covered
by the first test).
Verified the new assertions fail when the label-stamping loop is removed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Block accidental DELETE on critical platform objects (tenant-root,
cozy-system, LinstorCluster, CRDs, ClusterIssuers, cozystack-version
ConfigMap, cozystack-packages OCIRepository, tenant-root HelmRelease)
via a label-scoped ValidatingAdmissionPolicy. No webhook DaemonSet,
Service, TLS, or image — the API server evaluates the CEL expression
in-process. Requires Kubernetes 1.30+.
Bypass for legitimate deletes is two steps:
kubectl label <kind> <name> platform.cozystack.io/no-delete-
kubectl delete <kind> <name>
CRDs installed dynamically by the operator get the label stamped in
internal/crdinstall/install.go before the apply call.
Release note:
Added deletion-protection guardrail (ValidatingAdmissionPolicy) blocking
DELETE on cozystack platform objects labeled platform.cozystack.io/no-delete=true.
Bypass by removing the label first.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The HTTP-01 inheritance flow deadlocked at the route-Accepted step:
the apps/tenant chart correctly labelled the child namespace with
`namespace.cozystack.io/gateway=<owner>`, the parent Gateway's
allowedRoutes selector matched by label, but `collectHostnameClaims`
filtered routes by the static set `{tgw.Namespace} ∪ Spec.AttachedNamespaces`
— so a child route in a namespace reached purely via the inheritance
label was silently dropped. No per-listener Certificate, no per-app
HTTPS listener, no hostname match → `Accepted=False,
Reason=NoMatchingListenerHostname` indefinitely.
Caught by the new `child tenant's HTTPRoute attaches to parent's
Gateway via inheritance label` e2e bats — failed 3/3 retries on the
CI run for `afc7765e6`. The Go controller-level
TestReconcile_HTTP01CollectsHostnamesFromInheritingChildNamespaces
test pins the same contract at a faster feedback loop: an HTTPRoute
in a namespace labelled `namespace.cozystack.io/gateway=<owner>` but
NOT listed in Spec.AttachedNamespaces must still cause the
controller to render a per-listener HTTPS listener for its hostname.
The fix extends the `allowed` set in `collectHostnameClaims` to
also include namespaces matched by the gateway label selector, so the
hostname-claim collector and the Gateway's allowedRoutes selector
agree on which namespaces can attach. DNS-01 mode is unaffected — it
returns nil from `collectHostnameClaims` immediately and uses the
separate `collectInheritingChildApexes` path for SAN expansion.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
In DNS-01 mode, the parent's wildcard Certificate now covers every
inheriting child tenant's apex (`<child-apex>` + `*.<child-apex>`
as additional SANs) and the parent Gateway grows one wildcard HTTPS
listener per child apex referencing the same Certificate. Without
the expansion, a child route hostname like `harbor.alice.example.org`
fails to match the parent's listeners — `*.example.org` is a single-
label wildcard and `alice.example.org` is two labels deep — and the
TLS handshake fails the SNI check because the parent cert never
listed the child apex.
The controller discovers inheriting children by listing namespaces
labelled `namespace.cozystack.io/gateway = <tgw.Namespace>` (the
existing inheritance contract from Stage 1/3) and reading their
`namespace.cozystack.io/host` label, which apps/tenant chart writes
to the child apex. Children with no host label are skipped
(defensive); a child host equal to the parent apex is deduplicated
(cert-manager rejects duplicate dnsNames). HTTP-01 mode is
unchanged — its per-listener / per-cert flow already handles child
hostnames without any wildcard.
Listener naming: `https-child-<first-label>-<8-hex>`, where the hex
suffix is sha256(child-apex)[:8] so two children whose first DNS
label collides produce distinct listener names.
Cap: Gateway API hard-caps spec.listeners at 64 entries. The parent
uses 3 slots (http + https + https-apex) leaving ~61 wildcard
children per parent. Operators whose subtree fans out beyond that
must run the high-fanout branch on its own Gateway via
tenant.spec.gateway=true.
Tests (TDD red→green):
* TestReconcile_DNS01WildcardCertCoversInheritingChildApexes pins
the SAN list contains both `<child-apex>` and `*.<child-apex>`.
* TestReconcile_DNS01WildcardCertDeduplicatesChildApexEqualToParent
pins the dedupe path (child host == parent apex).
* TestReconcile_HTTP01WildcardCertNeverRendered pins the inverse:
HTTP-01 mode never renders a wildcard Certificate.
* TestReconcile_DNS01GatewayHasListenerPerChildApex pins the
listener-side expansion (one `*.<child-apex>` listener per
inheriting tenant, all referencing the parent's wildcard cert).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Gateway's HTTPS / TLS-passthrough listeners now match attached
namespaces by the namespace.cozystack.io/gateway label keyed on the
TenantGateway's own namespace, instead of a static
kubernetes.io/metadata.name In [list] whitelist. This restores
inheritance parity with the legacy ingress flow: child tenants of a
Gateway-owning ancestor publish through the ancestor's Gateway by
default — without their own LB IP / Certificate, and without
operator intervention to add each child by name to platform values.
Two writers populate the label:
* apps/tenant chart namespace.yaml — each tenant namespace gets a
label pointing at the nearest ancestor that owns a Gateway (self
if owning, inherited otherwise). The tenant Helm chart is the
source of truth for tenant-tree labels; the controller never
touches them (annotation-gated GC).
* cozystack-controller (ensureNamespaceLabels) — patches the label
onto every namespace in tgw.Spec.AttachedNamespaces, so cozy-*
system namespaces (cert-manager, monitoring, harbor, …) reach the
publishing Gateway alongside the tenant tree. Tracks its own
writes via the cozystack.io/gateway-attached-by annotation; on a
reconcile where an entry is removed from AttachedNamespaces, the
label is garbage-collected (annotation present + namespace no
longer expected → strip both label and annotation).
Also adds the gateway-attach label to tenant-root itself via
cozystack-basics/templates/tenant-root.yaml when
_cluster.gateway-enabled is true, so the Gateway in tenant-root's
own namespace passes its own selector for the controller-owned
http→https redirect HTTPRoute.
Tests (TDD, red→green):
* TestReconcile_HTTPSListenerUsesGatewayLabelSelector pins the new
MatchLabels shape on HTTPS listeners.
* TestReconcile_LabelsAttachedNamespaces pins the patch path: every
namespace in spec.AttachedNamespaces (plus tgw.Namespace) carries
namespace.cozystack.io/gateway=<owner> after reconcile.
* TestReconcile_LabelGCRemovesDroppedAttachedNamespaces pins the GC
path: removing an entry between reconciles strips the label.
* TestReconcile_LabelGCDoesNotStripHelmOwnedLabels pins the safety
contract: labels lacking the controller annotation (Helm-owned
tenant labels) are left alone.
* tenant-root_test.yaml adds the positive case for the conditional
label render under _cluster.gateway-enabled=true.
The HTTP listener (port 80) keeps the narrower static-name selector
— it allows only the tenant namespace plus cozy-cert-manager (for
HTTP-01 ACME challenges) and must NOT accept routes from app
namespaces or child tenants, since unattended routes on the HTTP
listener serve plaintext and leak credentials.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Complete the strategy.backups.cozystack.io/Job driver as the generic,
app-agnostic counterpart to the Altinity driver: render the strategy
PodTemplateSpec, run it as a one-shot batch/v1.Job, poll its terminal
condition, and emit a Backup. Wire it into the existing BackupClass
dispatch via reconcileJob/reconcileJobRestore rather than a standalone
reconciler, matching the post-BackupClass backups API (the old draft
read BackupJob.Spec.StrategyRef/StorageRef, which no longer exist).
Template context follows the in-tree convention (.Application, .Release,
.Mode, .Parameters, .Backup); the JobSpec.Template doc and regenerated
CRD are updated to match. Restore re-renders the same template with
.Mode=restore; cross-namespace restore is unsupported, mirroring
Altinity.
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
The dashboard controller used to project ApplicationDefinitions
into a set of dashboard.cozystack.io CRDs (Sidebar, Factory,
Navigation, Breadcrumb, CustomFormsOverride, CustomFormsPrefill,
CustomColumnsOverride, MarketplacePanel, TableUriMapping,
CFOMapping, BreadcrumbInside) consumed by the legacy UI.
cozystack-ui now reads ApplicationDefinitions directly, so the
controller, its API types, the generated CRDs and the related
RBAC rules are no longer used.
Removes:
- internal/controller/dashboard
- api/dashboard/v1alpha1
- packages/system/cozystack-controller/definitions/dashboard.cozystack.io_*.yaml
- dashboard.cozystack.io wiring from cmd/cozystack-controller/main.go
- dashboard.cozystack.io RBAC rules from cozystack-controller and cozy-dashboard charts
- ./api/dashboard/... codegen path from hack/update-codegen.sh
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
helm-controller's XValidation rule rejects Strategy.Name=RetryOnFailure
with RetryInterval set alongside a Remediation entry, so a future
"for safety" reintroduction of Remediation{Retries: -1} would silently
break every HR. Pin the deliberate omission.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Avoid pointer aliasing across all generated HelmReleases. r.HelmReleaseMaxHistory
is set once at startup today, but a future mutation through the field would
silently affect every in-flight HelmReleaseSpec.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin
