Go 1.26's extended new() builtin takes an expression, so new(true) and
new(false) replace ptr.To for the owner reference's Controller and
BlockOwnerDeletion flags. ptr.Deref, which has no builtin equivalent,
stays.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A key-free Secret already sitting at the canonical trust-anchor name is
left untouched and reported Ready. But tenant visibility comes from the
internal.cozystack.io/tenant-ca label, and an engine-owned Secret at that
name carries none, so the lineage webhook marks it tenantresource=false
and the tenant is locked out while the controller reports "published".
Add the missing label to the canonical-name contract deviations so the
warning names it, and collect all deviations rather than only the first
so a single warning describes every way the object falls short.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The ApplicationDefinition mapping fans out to every TenantProjection in
the cluster on every definition event, including Flux's periodic no-op
re-apply of the many *-rd definitions. The selectors-digest drift check
already suppresses the resulting writes, but not the load: each enqueued
reconcile still issues several uncached reads.
Gate the watch with a predicate that delivers an update only when
spec.secrets — the field the selectors digest is computed from — changed,
while letting creates and deletes through. Comparing the same field the
digest digests keeps the gate and the digest from drifting apart.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The package doc claimed there is no withdrawal logic and that a retired
declaration is only ever a deleted sentinel. withdrawProjection has
since become the second mechanism: it deletes a projection when the
sentinel outlives its CACert entry, which owner-reference garbage
collection never reaps because the owner still exists. State both paths.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
An entry with an empty sourceSecretName sent the reconciler to Get a
Secret with an empty name. The apiserver answers that with "resource
name may not be empty" — not NotFound — so it escaped the not-found
branch, was returned from Reconcile, and wedged the sentinel on
exponential backoff. The error path returns before any status write, so
no Ready condition was ever recorded: the silent retry the sentinel was
introduced to make visible.
Pin sourceSecretName to MinLength=1 so a live apiserver rejects it at
admission, and guard the reconcile so an object admitted by an older CRD
resolves to Ready=False/SourceInvalid instead of an error.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
withdrawProjection recognised its projection by an exact owner-reference
match while adoption uses a name match, so the two disagreed about what
counts as "ours" — and withdrawal was the stricter, which is backwards:
"safe to overwrite" is a stronger claim than "safe to delete".
Two projections a name match adopts and rewrites were read as foreign
by the exact match and left in place: one carrying a previous
incarnation's UID (delete-and-recreate under the same name), which
owner-reference garbage collection never reaps once the CACert entry is
gone, so the retired anchor stayed tenant-readable forever; and one
whose BlockOwnerDeletion flag had drifted, the very drift the adoption
path normalizes. Decide ownership with isOurProjection so both paths
answer the question the same way.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The selectors digest kept only the first 8 bytes of the SHA-256. A collision
skips the re-admission the doc calls load-bearing — the revocation direction,
where a tenant would silently keep read access after a definition stopped
selecting the anchor — and the annotation has no length limit, so store the full
hash. Also correct the withdraw-branch comment: MinItems=1 prevents a
single-entry sentinel from emptying its list, so the branch is reached when a
future non-CACert entry replaces the CACert one, not by gating a lone entry off.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The two-sentinel refusal returned without a RequeueAfter. NoRelease and
MultipleCACert can do that because their fix edits the same sentinel, which the
For() watch delivers; a contest is cleared by deleting the SIBLING sentinel, and
nothing enqueues this one when a sibling changes. So after an operator removed
the duplicate, the surviving sentinel withheld its anchor until the informer's
global resync — hours. Return RequeueAfter: resyncInterval on the contest branch
so recovery rides the resync backstop, and clarify that an anchor an earlier
uncontested reconcile already published is left in place rather than withdrawn.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The projection carries two annotations whose keys still named a publish-ca-cert
discovery leg that no longer exists in the tree. Rename them to
internal.cozystack.io/ca-cert-source and internal.cozystack.io/ca-cert-selectors
while the API is unshipped; both stay functional (the selectors digest drives
the re-admission drift check, the source ref is a drift input and a traceability
record).
Resolving the ApplicationDefinition matched on Kind alone from a List of
unspecified order. Gate the match on the release's group label too, so a release
stamped with an unexpected group cannot resolve a same-kind definition by
accident.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two gaps in the sentinel-to-canonical-name lifecycle:
Withdrawal — the reconciler published each declared entry but never deleted
one. A sentinel that keeps existing with its CACert entry removed (a chart
gating it on tls.enabled, say) left the old <release>.tenant-ca served forever,
because owner-reference GC only fires when the whole sentinel is pruned. A
sentinel that reconciles with zero CACert entries now deletes the projection at
the canonical name, but only when it still carries that sentinel's owner
reference, so a foreign Secret or another sentinel's projection is never touched.
Contention — two sentinels in one namespace with the same release label both
resolve to the single canonical name. Whichever reconciled first would publish
an arbitrary CA and the two could flap ownership of one projection. Refuse both,
mirroring the more-than-one-CACert-entry refusal, so nothing is published until
the declaration lives on exactly one sentinel.
The ClusterRole comments claimed the reconciler never deletes a Secret; corrected
to describe the owner-scoped withdrawal delete.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The only watches on the projection Secret were the source-Secret index (keyed on
spec.projections.sourceSecretName) and the sentinel itself. The projection is
named <release>.tenant-ca, which is no sentinel's source, so an in-place rewrite
of its ca.crt was corrected only by the five-minute resync — a window in which a
forged trust anchor is served.
Map the projection back to its owning sentinel through the controller owner
reference it carries, on the same dedicated metadata cache the source watch uses.
Owns(&corev1.Secret{}) would register against the manager cache, whose Secret
informer is label-scoped to the WildcardSecret replicas, so a projection carrying
no wildcard label would never be delivered — it would compile and pass a fake
client yet never fire in a real cluster.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
isOurProjection gated adoption on the ManagedLabel marker plus the Opaque
type. The marker is an ordinary label any Secret writer in the namespace can
strip: stripping it off the genuine projection made the controller disown its
own object, refuse it as a collision, and keep serving whatever bytes the same
actor wrote under ca.crt — a trust anchor stuck forged and never self-healing.
Key adoption on the owner reference back to this sentinel instead, matched by
name so a delete-and-recreate under the same name is still re-homed. Forging
the reference only causes an adopt-and-overwrite with the genuine key-free CA,
the self-healing direction; a Secret with no such reference is a stranger and
is refused as a collision, never overwritten. The Opaque type stays as a cheap
sanity guard but is no longer load-bearing — the update path is a
read-modify-write that never writes the immutable Type.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rewrite the cacert unit suite for the TenantProjection model: the
happy-path projection, the Ready=False/SourceNotFound status a missing
source now surfaces, ownership solely by the sentinel, the write-path
guards driven through a real source (private key and non-certificate
refused), the canonical-name cases, and the forged-projection heal. The
pure write-path guard tests — including the DER-reassembly guarantee —
carry over unchanged.
Replace the cacert chainsaw suite: it probes the writer policy against
the live API server (RBAC denies a tenant, the VAP denies a
non-flux writer, flux is allowed) and drives the controller end to end
(a flux-created sentinel reports SourceNotFound, then lifts a real CA
into a key-free, sentinel-owned projection). Drop the declared-leg
wiring test.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Drive the CA-extraction reconciler off the namespaced TenantProjection
sentinel a chart renders, replacing the publish-ca-cert label leg and
the cluster-scoped spec.caCert declaration leg. The sentinel names the
source Secret and key; the projected trust anchor takes an owner
reference to the sentinel, so the lineage webhook resolves the tenant
through sentinel -> HelmRelease and garbage collection prunes the
projection when the chart stops rendering the sentinel.
The source Secret is watched as metadata only, so the informer holds no
private key material, and the one source actually read goes through the
uncached reader. The sentinel's Ready status makes a missing or
unpopulated source visible with kubectl instead of a silent requeue.
The write-path guard, canonical-name contract, adoption rules and
selectors-digest re-admission are unchanged.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
releaseOwnerRef pins BlockOwnerDeletion to false so a trust anchor never
holds up the teardown of the application it belongs to, but the ownership
drift check compared every owner-reference field except that one. A sole
owner reference whose only drift was BlockOwnerDeletion flipped to true
therefore passed as already-correct and was never re-homed, leaving a
projection that blocks a foreground deletion of its HelmRelease — the
opposite of what the flag is set for.
Include BlockOwnerDeletion in the owner comparison so the write path
replaces a drifted reference and restores the flag to false.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The write-path guard validated the CA value as a certificate chain but
then published a clone of the original input. pem.Decode silently skips
bytes before the first PEM block and between blocks, and the private-key
header guard only matches a "-----BEGIN ... PRIVATE KEY-----" line, so a
raw DER or JWK private key placed ahead of or between real certificates
passes both checks and is copied verbatim into the tenant-readable
projection.
Assemble the projection from the re-encoded certificate DER instead of
cloning the input, so preamble and interstitial key material are
structurally unreachable. The empty-value, wrong-type, non-parseable and
trailing-byte rejections are unchanged.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Declare CNPG's operator-created CA Secret as the postgres trust-anchor
source (spec.caCert names {{ .release }}-ca), and add the generic
internal.cozystack.io/tenant-ca selector to spec.secrets so the key-free
<release>.tenant-ca projection the extraction controller publishes becomes
readable by the tenant.
postgres is the first converged engine, and the hardest: CNPG creates the
CA Secret itself, key-bearing, asynchronously, and offers no way to label
it, so it exercises the name-driven leg end to end. The test drives the
shipped ApplicationDefinition rather than a fixture, so the wiring and the
controller cannot drift apart unseen: a CNPG-shaped key-bearing
<release>-ca appears, and the projection must be produced key-free AND
selected for tenant visibility by the shipped selector.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Every engine in the catalog issues a per-release CA, but the object that
holds ca.crt almost always holds a private key next to it, so any RBAC
path that hands a tenant the trust anchor hands over key material too.
Add a controller that publishes, for each application release, a
canonical Opaque Secret <release>.tenant-ca holding only ca.crt. That
name is claimed by none of the operators the platform ships, which
matters: <release>-ca-cert, the obvious candidate, is already taken and
means opposite things across engines — a key-free CA on one, a
key-BEARING Secret on another — so projecting onto it could only be done
by overwriting live key material.
The certificate is found two ways. A Secret labelled
internal.cozystack.io/publish-ca-cert is how a chart opts in, and it is
watched, so a rotation propagates at once. For the engines whose operator
owns the CA Secret and will not label it, the application's caCert
declaration names that Secret, which is read by name and re-read on a
resync. The two legs differ only in how the source is found — extraction,
sanitization, ownership and collision handling are one shared write path,
because forking it is where a key-leak bug would hide.
Exactly one whitelisted key is ever copied, and every byte written is
re-checked for a PEM private-key header first: on the name-driven leg the
source is key-bearing by construction. The projection is owner-referenced
to the release, so deleting the application collects it with no prune
logic, and a Secret the controller did not create is never overwritten —
a Secret carrying key material on the canonical name is refused with a
Warning Event rather than adopted.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What
Removes the `network.cozystack.io` API group — `ExposureClass` +
`ServiceExposure` (#3081) — and its reconciler, in favor of native
`Service` `type: LoadBalancer` + `loadBalancerClass`.
## Why
Full rationale and discussion in #3164. For **managed applications** the
chart already owns the Service, so:
- `type: LoadBalancer` + `loadBalancerClass` + an admin-provisioned pool
covers vendor-neutral external exposure without a dedicated API group
and controller;
- the `externalIPs` (node-IP) default never went through
`ServiceExposure` (its backend is a no-op) and is left unchanged;
- the only current consumer, the host ingress, keeps rendering `type:
LoadBalancer` and just drops the `ServiceExposure`; the admin provisions
the address pool / announcer out of band, as the install docs already
assume.
`ServiceExposure`'s one unique feature — dynamic per-namespace scoping
of a shared VIP pool — is a convenience the managed-app path does not
need, and it does not justify a second tenant-networking API group next
to `sdn.cozystack.io`.
## Changes
- Delete `api/network/v1alpha1` and
`internal/controller/serviceexposure` (+ CRD definitions, e2e bats,
controller registration, RBAC).
- Host ingress: drop the `ServiceExposure` template; add an optional
`publishing.loadBalancerClass` that sets `spec.loadBalancerClass` on the
host-ingress LoadBalancer Service.
- The `externalIPs` path and default behavior are unchanged.
## Verification
- `go build ./...` — green.
- `helm unittest` — `core/platform` 72/72, `extra/ingress` 25/25 (incl.
host-vs-tenant `loadBalancerClass` scoping and the immutable-field
migration guard).
## Migration
None required. `network.cozystack.io` (#3081) has never shipped in a
release — it is on `main` only (not backported to `release-1.5`; absent
from v1.4.x / v1.5.x), so no upgrading cluster has these CRDs and this
removes it before it is ever released.
Clusters that track `main` and explicitly set `publishing.exposureClass`
(opt-in, only since 2026-06-30) should provision their MetalLB / Cilium
address pool themselves and set `publishing.loadBalancerClass`; any
orphaned `cozystack-<class>` pool can be deleted by hand. Default
clusters (externalIPs path) are unaffected.
## Status
**Draft / RFC for the #3164 discussion** — this removes an API group
that landed only days ago, so nothing here should merge before the
design call. cc @lexfrei (author of #3081) and @lllamnyp (#3164).
Deferred: when Kubernetes actually removes `Service.spec.externalIPs`,
the node-IP default migrates via a MetalLB `IPAddressPool` with no
`L2Advertisement` (allocate-but-don't-announce) — a separate change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for configuring the ingress host Service with a native
`loadBalancerClass` value.
* Expanded chart tests to verify class wiring and immutability behavior
across install and upgrade scenarios.
* **Bug Fixes**
* Removed the legacy exposure-class/service-exposure flow and related
controller behavior.
* Updated templates and CRD packaging so the platform no longer renders
or depends on those deprecated resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove the network.cozystack.io API group (ExposureClass + ServiceExposure)
and its reconciler. Managed-application charts own their Service, so native
type: LoadBalancer + loadBalancerClass + an admin-provisioned pool covers
external exposure without a dedicated API group and controller. The host
ingress -- the only consumer -- keeps rendering type: LoadBalancer and gains
an optional publishing.loadBalancerClass to pick the LB controller; the
externalIPs (node-IP) default path is unchanged.
Rationale in cozystack#3164.
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The operator-provided wildcard MVP serves the root tenant only: the root
ingress controller and Gateway read a TLS Secret from their own namespace.
Child tenants run their own ingress controller / Gateway in a separate,
default-deny namespace and cannot read that Secret cross-namespace.
Add a controller that mirrors the operator wildcard Secret into every
tenant namespace that owns a TLS termination point (its
namespace.cozystack.io/ingress or .../gateway label equals its own name).
The source is identified by name from the platform values channel
(_cluster.wildcard-secret-name) with no extra operator input; replicas
carry a managed-by label and refresh on rotation. An individual replica is
removed when its namespace stops terminating TLS; every replica is torn
down only on an explicit disable (clearing wildcard-secret-name). A missing
or mistyped source, or a transiently-absent values channel, keeps existing
replicas, so tenant TLS never drops on a transient gap. The controller's
Secret cache is scoped to managed replicas and the values channel, not
every cluster Secret. Only the platform controller's ServiceAccount gains
Secret writes — no tenant RBAC is widened, and every consumer reads only
its own-namespace copy.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
ComputePools derived a pool's Available with SubtractWithNonNegativeResult over
the union of the budget and carve-out keys, so a bounded child carving a
resource its parent does not bound surfaced that resource as 0 in the parent
pool's Available. The controller then wrote a tenant-quota-allocated clamping
the parent namespace to zero of a resource it never limited, blocking its
workloads (observed on a cluster: a memory-bounded child froze its cpu-only
parent's memory). Mask Available to the budget's resource names so only
resources the pool root actually bounds are enforced; mirrors the fix already
applied to Pool.Overcommitted.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
Pool.Overcommitted reported a resource as overcommitted whenever a sub-tenant
carved out a resource the pool root does not bound. Admission explicitly allows
a child to bound a resource its parent leaves unbounded, and an unbounded budget
can never be overcommitted, so this produced a spurious QuotaOvercommitted
warning event on every reconcile. Restrict the check to resources present in
the pool budget.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
Take a tenant's declared budget from its chart-rendered tenant-quota
ResourceQuota (.spec.hard) instead of the raw tenant.spec.resourceQuotas
values. The chart expands the shorthand values into real quota resource
names and applies the cluster's request/limit allocation ratios, so
reading the rendered object keeps the controller-owned allocated quota in
exactly the same key space — the two quotas compose correctly and the
controller never replicates that cluster-configurable transformation.
Also skip pools that need no tightening beyond the chart quota (a lone
tenant with no sub-tenants and no carve-outs): the controller writes an
allocated quota only when a budget is actually shared or carved up, so a
single-tenant namespace is left exactly as the chart renders it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Add a tenant quota reconciler to cozystack-controller that realises the
hierarchical (sub-tree) quota model at runtime. Each tenant's declared
quota is the budget for its whole sub-tree: a child that declares its own
quota carves a fixed slice out of the parent budget, while a child without
a quota shares the parent's remaining pool.
The controller maintains a per-namespace 'tenant-quota-allocated'
ResourceQuota (Kubernetes enforces the most restrictive quota, so it binds
without fighting Flux over the chart-rendered 'tenant-quota'), aggregating
sub-tree usage so a pool's members collectively stay within the budget. A
--tenant-quota-buffer-percent flag temporarily inflates pool budgets so
workloads already over a freshly-introduced quota keep running during
rollout; overcommitted pools emit a QuotaOvercommitted event.
The model and the selector-spans-namespaces/aggregated-usage approach are
adapted from OpenShift's ClusterResourceQuota controllers (openshift/api,
library-go, cluster-policy-controller, apiserver-library-go), Apache-2.0,
Copyright Red Hat, Inc. and the OpenShift Authors.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
## What this PR does
Lays the foundation for migrating tenant LoadBalancer exposure off the
deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off
MetalLB onto Cilium-native announcement — the mechanism-agnostic shape
prescribed when the earlier chart-side approach (#2468) was reverted in
#2603.
It adds a new `network.cozystack.io/v1alpha1` API group and a controller
that translates intent into whichever LoadBalancer backend the cluster
admin configured:
- `ExposureClass` (cluster-scoped, admin): binds a logical class name to
a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP
address scope — the StorageClass/IngressClass pattern.
- `ServiceExposure` (namespaced, tenant intent): "expose this Service
via class X"; status reports the resolved backend and assigned IPs.
- A `serviceexposure` controller renders the backend's pool + announcer
once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium
`CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to
the namespaces that opted in. Backends are pluggable behind a small Go
interface, so adding BGP or another cloud LB is one implementation, not
an API change. The controller never mutates the target Service — the
chart still owns its shape.
Platform wiring: `publishing.exposureClass.*` renders the admin
`ExposureClass` and the host ingress publishes through it. The default
(no class set, `externalIPs`) leaves every existing cluster's behaviour
byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes
a single `ExposureClass.spec.backend` change.
This is the foundation only — Part of #2586. Converting per-app
exposure, flipping Cilium `l2announcements` on, changing the default,
and removing MetalLB are deferred to later phases.
### Release note
```release-note
feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added new `ExposureClass` and `ServiceExposure` custom resources with
support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB),
including optional L2 and address configuration.
* Added Helm wiring and templates to enable service/ingress publishing
via `ServiceExposure`, selecting a configured (or default)
`ExposureClass`.
* **Bug Fixes**
* Improved reconciliation, finalizer-based cleanup, and status updates
for readiness and assigned IP reporting, including safer handling when
backends or managed resources change.
* **Tests**
* Added end-to-end and unit test coverage for backend rendering,
readiness, and cleanup flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add the serviceexposure controller to the cozystack-controller binary. It
resolves a ServiceExposure's ExposureClass to a backend and renders the
backend-specific pool and announcer CRs, then reports the assigned IPs in
status.
Backends are pluggable behind a small interface, one per file:
- externalIPs: renders nothing (the node owns the address) — preserves the
historical default behaviour.
- metallb: IPAddressPool + L2Advertisement in cozy-metallb, scoped to the
target Service's namespace.
- cilium: CiliumLoadBalancerIPPool (cilium.io/v2) + CiliumL2AnnouncementPolicy
(cilium.io/v2alpha1), scoped to the Service via serviceSelector.
- robotlb: renders nothing (the cloud LB owns allocation and announcement).
The rendered CRs belong to foreign API groups not in the Go scheme, so they
are emitted as unstructured objects. They carry no OwnerReference (the
Cilium CRs are cluster-scoped and the MetalLB CRs live in a different
namespace than the owning ServiceExposure), so ownership is tracked via a
management label plus owner annotations, cleanup is driven by a finalizer,
and the controller refuses to take over an identically-named resource it
does not manage. The controller never mutates the target Service, leaving
its shape to the chart that renders it.
Grant the controller write access to metallb.io and cilium.io pool /
announcer resources.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Operators who already hold a wildcard TLS certificate (purchased, or
issued by a corporate CA) had no way to make a tenant Gateway serve
under it: the only cert modes were http01 (per-listener ACME) and
dns01 (controller-minted wildcard via ACME).
Add a third mode, existingSecret, that references a pre-existing TLS
Secret in the TenantGateway's own namespace via spec.wildcardSecretRef.
In this mode the controller mints no ACME Issuer and no Certificate;
the wildcard, apex, and per-child-apex HTTPS listeners reference the
supplied Secret directly. A missing wildcardSecretRef fails fast with
Ready=False rather than producing a Gateway pointing at a nonexistent
Secret. Switching into existingSecret from http01/dns01 reclaims the
now-unused Issuer and Certificates so no ACME machinery is left behind.
The non-cert listeners (http->https redirect, TLS passthrough) are
unaffected and continue to render.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The HTTP-01 inheritance flow deadlocked at the route-Accepted step:
the apps/tenant chart correctly labelled the child namespace with
`namespace.cozystack.io/gateway=<owner>`, the parent Gateway's
allowedRoutes selector matched by label, but `collectHostnameClaims`
filtered routes by the static set `{tgw.Namespace} ∪ Spec.AttachedNamespaces`
— so a child route in a namespace reached purely via the inheritance
label was silently dropped. No per-listener Certificate, no per-app
HTTPS listener, no hostname match → `Accepted=False,
Reason=NoMatchingListenerHostname` indefinitely.
Caught by the new `child tenant's HTTPRoute attaches to parent's
Gateway via inheritance label` e2e bats — failed 3/3 retries on the
CI run for `afc7765e6`. The Go controller-level
TestReconcile_HTTP01CollectsHostnamesFromInheritingChildNamespaces
test pins the same contract at a faster feedback loop: an HTTPRoute
in a namespace labelled `namespace.cozystack.io/gateway=<owner>` but
NOT listed in Spec.AttachedNamespaces must still cause the
controller to render a per-listener HTTPS listener for its hostname.
The fix extends the `allowed` set in `collectHostnameClaims` to
also include namespaces matched by the gateway label selector, so the
hostname-claim collector and the Gateway's allowedRoutes selector
agree on which namespaces can attach. DNS-01 mode is unaffected — it
returns nil from `collectHostnameClaims` immediately and uses the
separate `collectInheritingChildApexes` path for SAN expansion.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
In DNS-01 mode, the parent's wildcard Certificate now covers every
inheriting child tenant's apex (`<child-apex>` + `*.<child-apex>`
as additional SANs) and the parent Gateway grows one wildcard HTTPS
listener per child apex referencing the same Certificate. Without
the expansion, a child route hostname like `harbor.alice.example.org`
fails to match the parent's listeners — `*.example.org` is a single-
label wildcard and `alice.example.org` is two labels deep — and the
TLS handshake fails the SNI check because the parent cert never
listed the child apex.
The controller discovers inheriting children by listing namespaces
labelled `namespace.cozystack.io/gateway = <tgw.Namespace>` (the
existing inheritance contract from Stage 1/3) and reading their
`namespace.cozystack.io/host` label, which apps/tenant chart writes
to the child apex. Children with no host label are skipped
(defensive); a child host equal to the parent apex is deduplicated
(cert-manager rejects duplicate dnsNames). HTTP-01 mode is
unchanged — its per-listener / per-cert flow already handles child
hostnames without any wildcard.
Listener naming: `https-child-<first-label>-<8-hex>`, where the hex
suffix is sha256(child-apex)[:8] so two children whose first DNS
label collides produce distinct listener names.
Cap: Gateway API hard-caps spec.listeners at 64 entries. The parent
uses 3 slots (http + https + https-apex) leaving ~61 wildcard
children per parent. Operators whose subtree fans out beyond that
must run the high-fanout branch on its own Gateway via
tenant.spec.gateway=true.
Tests (TDD red→green):
* TestReconcile_DNS01WildcardCertCoversInheritingChildApexes pins
the SAN list contains both `<child-apex>` and `*.<child-apex>`.
* TestReconcile_DNS01WildcardCertDeduplicatesChildApexEqualToParent
pins the dedupe path (child host == parent apex).
* TestReconcile_HTTP01WildcardCertNeverRendered pins the inverse:
HTTP-01 mode never renders a wildcard Certificate.
* TestReconcile_DNS01GatewayHasListenerPerChildApex pins the
listener-side expansion (one `*.<child-apex>` listener per
inheriting tenant, all referencing the parent's wildcard cert).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Gateway's HTTPS / TLS-passthrough listeners now match attached
namespaces by the namespace.cozystack.io/gateway label keyed on the
TenantGateway's own namespace, instead of a static
kubernetes.io/metadata.name In [list] whitelist. This restores
inheritance parity with the legacy ingress flow: child tenants of a
Gateway-owning ancestor publish through the ancestor's Gateway by
default — without their own LB IP / Certificate, and without
operator intervention to add each child by name to platform values.
Two writers populate the label:
* apps/tenant chart namespace.yaml — each tenant namespace gets a
label pointing at the nearest ancestor that owns a Gateway (self
if owning, inherited otherwise). The tenant Helm chart is the
source of truth for tenant-tree labels; the controller never
touches them (annotation-gated GC).
* cozystack-controller (ensureNamespaceLabels) — patches the label
onto every namespace in tgw.Spec.AttachedNamespaces, so cozy-*
system namespaces (cert-manager, monitoring, harbor, …) reach the
publishing Gateway alongside the tenant tree. Tracks its own
writes via the cozystack.io/gateway-attached-by annotation; on a
reconcile where an entry is removed from AttachedNamespaces, the
label is garbage-collected (annotation present + namespace no
longer expected → strip both label and annotation).
Also adds the gateway-attach label to tenant-root itself via
cozystack-basics/templates/tenant-root.yaml when
_cluster.gateway-enabled is true, so the Gateway in tenant-root's
own namespace passes its own selector for the controller-owned
http→https redirect HTTPRoute.
Tests (TDD, red→green):
* TestReconcile_HTTPSListenerUsesGatewayLabelSelector pins the new
MatchLabels shape on HTTPS listeners.
* TestReconcile_LabelsAttachedNamespaces pins the patch path: every
namespace in spec.AttachedNamespaces (plus tgw.Namespace) carries
namespace.cozystack.io/gateway=<owner> after reconcile.
* TestReconcile_LabelGCRemovesDroppedAttachedNamespaces pins the GC
path: removing an entry between reconciles strips the label.
* TestReconcile_LabelGCDoesNotStripHelmOwnedLabels pins the safety
contract: labels lacking the controller annotation (Helm-owned
tenant labels) are left alone.
* tenant-root_test.yaml adds the positive case for the conditional
label render under _cluster.gateway-enabled=true.
The HTTP listener (port 80) keeps the narrower static-name selector
— it allows only the tenant namespace plus cozy-cert-manager (for
HTTP-01 ACME challenges) and must NOT accept routes from app
namespaces or child tenants, since unattended routes on the HTTP
listener serve plaintext and leak credentials.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The dashboard controller used to project ApplicationDefinitions
into a set of dashboard.cozystack.io CRDs (Sidebar, Factory,
Navigation, Breadcrumb, CustomFormsOverride, CustomFormsPrefill,
CustomColumnsOverride, MarketplacePanel, TableUriMapping,
CFOMapping, BreadcrumbInside) consumed by the legacy UI.
cozystack-ui now reads ApplicationDefinitions directly, so the
controller, its API types, the generated CRDs and the related
RBAC rules are no longer used.
Removes:
- internal/controller/dashboard
- api/dashboard/v1alpha1
- packages/system/cozystack-controller/definitions/dashboard.cozystack.io_*.yaml
- dashboard.cozystack.io wiring from cmd/cozystack-controller/main.go
- dashboard.cozystack.io RBAC rules from cozystack-controller and cozy-dashboard charts
- ./api/dashboard/... codegen path from hack/update-codegen.sh
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Replace rename-only wording on spec.targetApplicationRef form labels so
users discover that the field also supports restoring into a different
application, not just renaming.
- Drop VMInstance-specific keys from the spec.options form label; the
restore options are driver-specific, so hardcoding one driver's keys is
misleading once other drivers land.
- Render a single "Same as backup" fallback for an omitted
spec.targetApplicationRef on the details page instead of the noisy
"-.-/-" produced by concatenating three missing path fallbacks.
- De-duplicate the time-block / time-icon / time-value component IDs
within the restorejob-details factory by prefixing them with
created-, started-at-, and completed-at-.
- Mark non-CRD-backed stock-project-factory-*-details sidebars (kube-*,
plan, backupjob, backup, restorejob) as static in
upsertMultipleSidebars so they pick up consistent managed-by labels
instead of being left unmanaged.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replace http.DefaultClient with a package-level *http.Client with an
explicit 10-second timeout. Avoids sharing the process-wide default
transport with other libraries.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Pass bucket names from BucketClaim.Status.BucketName into the PromQL
query as a bucket=~"name1|name2" filter. This prevents O(N²) load
where N WorkloadMonitors each fetch all buckets globally.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Wrap resolvePrometheusURL and queryAllBucketMetrics in a
len(bucketClaimList.Items) > 0 guard. Avoids unnecessary namespace
GET and HTTP request to Prometheus on every reconcile of non-bucket
WorkloadMonitors (postgres, redis, kubernetes, etc.).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Replace 2×N per-bucket HTTP requests with a single query that fetches
all SeaweedFS bucket size metrics at once:
{__name__=~"SeaweedFS_s3_bucket_(size|physical_size)_bytes"}
Results are keyed by bucket name in memory, then looked up per
BucketClaim. This reduces HTTP round-trips from 2N+1 to 2 (one
namespace lookup + one Prometheus query) regardless of bucket count.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Move resolvePrometheusURL call before the BucketClaim loop and pass
the URL as parameter. Avoids redundant namespace lookups and reduces
reconcile time from 2×N+1 to 2×N HTTP calls (where N = BucketClaims).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Change queryPrometheusMetric to return (int64, bool) so callers can
emit s3-storage-bytes=0 for empty buckets while omitting the field
entirely when monitoring is not configured.
Also:
- Add RBAC marker for core/namespaces GET (used by resolvePrometheusURL)
- Log namespace read errors instead of silently returning empty URL
- Add TestQueryPrometheusMetricZeroValue test
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Bucket existence is already tracked via LifetimeHours. The s3-buckets
count resource produced a meaningless "s3-buckets-Hours" fallback type
in billing. Only storage size metrics (s3-storage-bytes,
s3-physical-storage-bytes) are now set on bucket Workloads.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Initialize workload.Labels map inside CreateOrUpdate mutate function
to prevent nil map panic when existing Workload has no labels
- Use strconv.ParseFloat instead of resource.ParseQuantity for
Prometheus metric values, which may use scientific notation
(e.g. "1.048576e+06") that ParseQuantity does not support
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Replace static --prometheus-url flag with dynamic resolution from
namespace.cozystack.io/monitoring label. Each tenant namespace knows
which tenant hosts its monitoring stack, so the controller constructs
the vmselect URL automatically. This correctly handles multi-tenant
setups where different tenants may use different monitoring instances.
- Remove PrometheusURL field from WorkloadMonitorReconciler struct
- Remove --prometheus-url flag and prometheusUrl chart value
- Add resolvePrometheusURL() that reads namespace label
- queryPrometheusMetric() now accepts prometheusBaseURL as parameter
- Add tests for resolvePrometheusURL with and without label
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Set replicas/minReplicas to 0 in bucket WorkloadMonitor (buckets
have no pods, minReplicas=1 would mark monitor as not operational)
- Remove dead IsNotFound check on List (List returns empty, not 404)
- Trim trailing slash from PrometheusURL before path append
- Add strings import for TrimRight
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Check resp.StatusCode before parsing Prometheus response
- Limit response body read to 1 MB via io.LimitReader
- Use strings.HasPrefix in test instead of fragile slice indexing
- Add TestQueryPrometheusMetricServerError for 500 responses
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Query SeaweedFS_s3_bucket_physical_size_bytes alongside the logical
size metric. Physical size includes all replicas and reflects actual
disk usage, while logical size reflects what the user stored.
Refactor queryBucketSizeBytes into generic queryPrometheusMetric
to reuse for both metrics.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Aleksei Sviridkin