mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-29 16:17:20 +09:00
11bac54f931682dd94de4081bb9e8a7eef60bef1
161
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
9121456ba8 |
chore(cozystack-controller): name the Service and ReferenceGrant reads
Deciding whether a TLSRoute forwards anywhere reads the backend Service and the ReferenceGrant admitting a cross-namespace reference to it, and both are watched rather than fetched on demand. The read-everything rule at the foot of the ClusterRole covers them today, so nothing is broken. Naming them keeps the feature working the day that rule is narrowed, where an informer starting without a grant fails with Forbidden and takes the manager with it. The kubebuilder markers in the package generate nothing: controller-gen runs over ./api/... alone and writes no rbac artifact, so the shipped role is hand-written and only a test connects the two. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
2bf9ae4a27 |
feat(gateway)!: serve passthrough hostnames and stop terminating them
Each tlsPassthroughListeners entry renders a mode=Passthrough listener on its own port, answering the entry's hostname by SNI and admitting TLSRoutes from the publishing tenant's namespace alone. It declares no allowedRoutes.kinds: at the pinned Cilium v1.19.5 a listener that spells its kinds out has them applied to every route on the Gateway rather than to itself, so naming TLSRoute here would reject every HTTPRoute on the Gateway. A hostname a passthrough listener answers no longer earns an HTTPS-terminate listener or a Gateway-issued certificate. That is the break, and it is keyed on service rather than on declaration: the same Cilium builds a passthrough filter chain per route out of the backends that survived resolution, so a listener no TLSRoute attaches to, and a route whose backendRefs resolve to nothing, both match no ClientHello and leave the terminate listener holding the name. Eligibility asks what v1.19.5 asks and no more, the reference allowed and the Service present, and fails closed on a read it cannot complete. Services and ReferenceGrants are watched, so a withdrawal deferred on a missing backend lands on the pass that sees it arrive, and the Service watch takes only the creates and deletes, existence being all that is read. Which hostname a route claims is answered the same way. spec.hostnames is optional on a TLSRoute, and ComputeHosts substitutes the hostname of each TLS listener the route's parentRefs select for a route that declares none, so such a route carries the SNI and takes the name over exactly as one that spells it out. A declared hostname broader than the listener's is that substitution from the other side: the route is served on the listener's name and not on the wildcard it wrote, so a claim covering a reserved name counts as a claim on it. The substitution reaches only the listeners that would admit the route. A native-port listener takes the publishing tenant's namespace alone, so Cilium never puts a route from elsewhere on it and never serves that route on its hostname; borrowing the name anyway refuses the route for a namespace, over a hostname its own object does not carry, in the same pass that sheds a terminate listener because that same route is serving a name it borrowed legitimately. A route that names the listener in its sectionName keeps the borrow, since there the refusal is the answer to what it asked for. A route the Gateway renders no listener for hears it from this controller instead of from Cilium alone: the hostname a passthrough listener took, the hostname nothing answers, the sectionName or port naming a listener that answers something else, the sectionName naming no rendered passthrough listener, and the native-port listener that admits the tenant alone, each with its own Gateway API reason and a message naming the hostnames that lost. An HTTPRoute pinning a rendered passthrough listener by sectionName is refused there too, since such a listener forwards a stream it never terminates and takes TLSRoute alone by protocol. Nothing else tells that route: the port-443 listeners name HTTPRoute in their kinds to keep every port-443 set uniform, which the Gateway reports back as InvalidRouteKinds rather than as a grant, and the pin reads a listener's kinds against every route on the Gateway, so it reports the route accepted. The field is confined to certMode http01, stated as the modes that can carry it so one added later is refused until someone rules on it. The wildcard modes cover the whole apex with one terminate listener the pinned Cilium cannot keep apart from a passthrough listener beneath it, and edge terminates TLS at the class provider and renders no TLS listener at all. An HTTPRoute claiming a wildcard is refused under http01 before the race and the overlap are read. The HTTP-01 challenge issues no wildcard certificate, so the terminate listener rendered for the claim could never carry the name, and the listener name built from the hostname's first label carried the asterisk, which the Gateway CRD admits in a hostname and refuses in a name, so the apiserver refused the whole Gateway write and every other listener on it froze where it stood. The renderer now writes that label as "wildcard", so every hostname the CRD admits yields a name it admits, and no name that was already admissible moves. That refusal is a statement about the terminate listener, so it is judged where that listener is decided rather than ahead of everything else. The port-80 listener carries no hostname and serves a wildcard claim as readily as any other, so a route that selects it, by pinning the port or by naming the listener, never needed the listener HTTP-01 cannot supply and is served rather than refused. Judged first, the refusal reached that route too and reported it unserved while Cilium was serving it in plaintext. A parentRef port on an HTTPRoute is read the way servableOn reads it for a TLSRoute. The terminate listener sits on 443, so a route pinning another port attaches to no listener the render would give it, and it used to earn a listener and an ACME certificate for a route that attached to nothing. Port 80 is the plain-HTTP listener, which admits the publishing tenant alone: a route from there pinning it is served plain and earns neither a listener nor a refusal. A route status write that fails is told apart by kind. A conflict or a route that vanished between the list and the write is a race between the two status writers this Gateway has: the reconcile requeues and retries the write without pinning Ready=False on the tenant, and the status pass runs after the http-to-https redirect, so such a race cannot withhold the one object that guard exists to provide. A write the apiserver will refuse on every pass still fails the reconcile, after every other route has been written, because the condition is the only object saying why a hostname is not served, and a pass that reported success over a lost write left the route reading Accepted=True until the resync. The retryable half is logged on the way out. Declining to pin Ready=False leaves the requeue as the only thing acting on it, and the requeue says nothing: the tenant goes on reading Ready, no event is recorded, and returning no error opts out of the backoff that would widen the interval, so a write failing on every pass retries at a fixed period indefinitely with nothing anywhere to show for it. A TLSRoute claiming a hostname broader than the listener's is pulled into the eligibility question for each name it covers, because the pinned Cilium serves it on the listener's own name. That is right only where the listener answering a name is named exactly that name. Under a wildcard entry the substitution hands such a route the wildcard, so its chain carries "*.db.<apex>" while the terminate chain carries the exact name beneath, and Envoy matches an exact server name ahead of a wildcard, so nothing collides; pulling the route in there withdrew a served endpoint and handed its name to the database behind the passthrough listener. The pull-in is now keyed on an exact listener match. Every condition this controller writes on an HTTPRoute, and the one it writes on a TLSRoute declaring no hostnames, is written under http01, and the whole-apex modes collect no claims, so a switch of certMode left those verdicts standing over listeners the new mode does not render. The whole-apex pass now drops this controller's entry from HTTPRoutes and from hostname-less TLSRoutes; the edge refusal stays confined to routes that declare hostnames, since a refusal written on a lent hostname would have no path back. Within http01 the same verdicts outlive their listeners for a second reason, and no mode change is needed to reach it. The status pass walks the routes that attached rather than the ones that claimed a hostname, and retracts its entry from the difference, because a claim set is not a route's own property once a hostname can be borrowed: a TLSRoute declaring none takes the hostname of every passthrough listener its parentRef selects, so removing that listener empties the claim set while the route stands unchanged, and an HTTPRoute reaches the same state by dropping its hostnames. Judged on claims alone such a route keeps whatever the last pass that did judge it left behind, describing a listener the Gateway no longer has. An HTTPRoute pinned by sectionName to a passthrough listener this Gateway does not render is refused with NoMatchingParent, the reading a TLSRoute in the same shape already got, instead of earning a terminate listener it could never attach to. A route served on the port-80 listener, by parentRef port or by naming it, earns no terminate listener and is refused nothing, but it publishes the name from its namespace all the same, so it stays in the hostname race. Every recount now ranks it with the field it recounts, because a recount that left it out crowned the namespace that had lost to it and dropped a loss the first count recorded: a third route refused in a third namespace, or a passthrough entry declared for the name, flipped the loser to Accepted=True. Selecting the port-80 listener by name is judged as selecting it by port: from a namespace the listener admits the route is served plain, from anywhere else the listener turns it away and the route reads NotAllowedByListeners, and a sectionName and a parentRef port that select different listeners select none and read NoMatchingParent, which is what Gateway API requires when both are given. Before this a route naming the listener from a refused namespace earned a terminate listener and a certificate it could never attach to. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d685fb30ec |
fix(gateway)!: require tenant routes to declare their hostnames (#3475)
## What this PR does The tenant route hostname policy skipped the shape it most needed to check. Its CEL opened with `!has(object.spec.hostnames) ||`, so an HTTPRoute or TLSRoute declaring no hostnames satisfied the first operand and got admitted with nothing compared against the tenant apex. An empty list passed too: `has()` is true on `[]`, and `all()` over an empty list is vacuously true. Such a route is not inert. Under Gateway API it inherits the hostnames of every listener it attaches to, and `parentRefs` can name a Gateway in another namespace, so it covers whatever that listener covers. Reported in #3467. The predicate now requires the field to be present and non-empty before validating it. Both policies rendered from that template share one CEL string, so HTTPRoute and TLSRoute get the change together, and both match `CREATE` and `UPDATE`, so an existing route that drops its hostnames is denied exactly as a new one is. The issue is titled after TLSRoute, but the reachable shape is mostly HTTPRoute. TLSRoute allows a hostname-less route only on the deprecated `v1alpha2`; at `v1` and `v1alpha3` the bundle this repo vendors marks `spec.hostnames` required and rejects an empty list on `minItems: 1`, while `v1alpha2` carries neither constraint. HTTPRoute `v1` carries neither either, so it accepts both shapes. All four are checkable in `packages/system/gateway-api-crds/templates/crds-experimental.yaml`. Checking that turned up a stale claim in the template header, which said TLSRoute had no `v1` in Gateway API 1.4 to 1.5 and used it to justify the rule naming only `v1alpha2`. Gateway API 1.5.1, which this repo vendors, serves TLSRoute at `v1` as the storage version. The rule still covers every served version because `matchConstraints.matchPolicy` defaults to `Equivalent`, and that default is now written down, because setting `matchPolicy: Exact` reads like a tightening and would silently stop the policy seeing `v1` and `v1alpha3` TLSRoutes. I corrected the text and left the coverage alone: the wrong sentence predates this branch, but my change is what makes the file contradict itself, so the text is mine to fix while widening version coverage is a separate job. A `ValidatingAdmissionPolicy` has no cross-object lookup, so it cannot resolve the parent Gateway to see which hostnames a route would inherit. Requiring the route to name its own is what keeps every admitted hostname comparable against the apex. That has a cost: a route relying on listener inheritance now has to spell its hostnames out, and a tenant that hand-writes one gets a denial where it used to get an admission. ### The controller change belongs in the same PR `renderHTTPRedirect` built the http-to-https redirect HTTPRoute with no hostnames on purpose, to catch every host on port 80. It lands in a `tenant-*` namespace, so the stricter policy denies it, `reconcileHTTPToHTTPSRedirect` returns that error, and since it runs before the status update the TenantGateway never reaches Ready. That redirect is what keeps app routes attaching by hostname from serving plaintext on port 80. Neither half of this change is safe alone. The redirect now names the tenant apex and its wildcard. Both entries are needed: a wildcard hostname is a suffix match over any number of labels, so `*.<apex>` covers subdomains at every depth but not the bare `<apex>`. `renderWildcardCertificate` already builds the same pair. Coverage narrows in one direction. A request on port 80 with a Host outside the tenant apex now falls through to a 404 instead of a 301. That narrowing costs no covered host: the port-80 listener admits routes only from the Gateway's own namespace and cert-manager's, and the tightened policy requires every route in the tenant namespace to declare its hostnames inside the apex, so any host published from this namespace already falls under the apex or its wildcard. An inheriting child tenant does not widen that set either. Its hostnames would reach port 443 through a listener on this same Gateway, and `cozystack-gateway-hostname-policy` compares every listener against the host label of the Gateway's own namespace, so a child apex outside that label never gets its listener: the Gateway write carrying it is refused at admission whole, in `reconcileGateway` and before the redirect is rendered. A child apex under this one passes that check and the wildcard already covers it. That holds at merge base as well, so it is not something this branch changes. Deriving hostnames from `spec.apex` makes that field load-bearing for admission, so it gains `MinLength=1`. An empty apex used to reconcile to Ready, since the redirect declared no hostnames at all; it would now render `""` and `"*."` and be rejected on the HTTPRoute hostname schema, surfacing as a validation error naming the route rather than the field. The renderer checks it too, because the CRD and the controller binary roll out separately and an older CRD in the cluster will not carry the bound. That schema bound is a breaking change to an existing field and the API review gate flags it as one, so here is what it actually invalidates. The chart that renders these CRs already fails on an empty `_namespace.host`, and in DNS-01 or existingSecret mode an empty apex already broke earlier, in `renderGateway`, on a listener with `hostname: "*."`. Nothing that works today becomes invalid. What the bound rejects is a TenantGateway written by hand, past the chart, with an empty apex, and I would rather that be refused at admission naming the field than at reconcile time in controller logs, which is why the bound sits in the CRD as well as the renderer. Naming those hostnames also meant excluding the redirect from `collectHostnameClaims`. A route that attaches to the Gateway and declares hostnames otherwise reads as an app publishing them, and each claimed hostname becomes an HTTPS listener named after its first label. The first label of `*.<apex>` is `*`, which the Gateway CRD's listener-name pattern rejects, so the Gateway write fails and the reconcile stops before anything downstream. The exclusion matches on namespace, name and controller ownership rather than on the labels the renderer stamps: those labels are writable by anyone who can create a route, and the claim set also drives conflict resolution, so a label-keyed exception would let any route opt out of being marked `Accepted=False` when it loses a hostname race. Name and ownership are each required and each is pinned by its own test, since ownership alone would exclude any other controller-owned route and name alone would extend the exception to a route planted under that name by someone else. ### Radius Only namespaces starting with `tenant-` are matched. The three first-party TLSRoutes the platform ships, `kubernetes-api` in `default` plus `cdi-uploadproxy` and `vm-exportproxy`, set hostnames explicitly and none sits in a `tenant-*` namespace, so nothing changes for them. I went through every producer of routes in the tree, in three places rather than one. No Helm template emits an absent or empty hostname list in any configuration a tenant can reach: the vendored openbao, harbor, opencost and flux-operator route templates can render one, but each is behind a flag that defaults off with no values path to turn it on, and openbao would fail CRD validation first because its default route apiVersion requires hostnames. Exactly one Go site builds a route, the redirect fixed here. And one e2e fixture did rely on the old behaviour: the Gateway suite's parentRef probe declared no hostnames and is applied into `tenant-test`, which the policy matches, so it now reads the apex from the namespace label and names a hostname under it, the same way the neighbouring cases in that suite already did. cert-manager also writes routes into tenant namespaces, since it creates the Challenge in the Certificate's own namespace and the HTTP-01 solver route beside it. That route declares the hostname it is validating, so the policy admits it. ### Related issues Two open issues have a text half that this PR happens to settle, and neither of them closes here. #3788 reports that the port-80 allow-list contains the namespace its own comment says it excludes; the comment and the README paragraph now describe what that filter actually does, while the behavioural exit it names stays open. #3789 reports that the TLSRoute rule names only the deprecated API version; that premise is now written down in the template header, including what has to change before a bundle drops `v1alpha2`, but the rule still names `v1alpha2` alone. #3787 covers the wildcard-hostname listener name that the apiserver rejects, which is what makes the claim-set exclusion here load-bearing rather than tidy. ### Screenshots N/A (no UI changes). ### Downstream repositories I walked the trigger map in `docs/agents/contributing.md` against the diff, file by file. No package is added, renamed or removed; no `values.yaml` key, `values.schema.json` field, version enum or default moves; no `kind` or `plural` changes; no namespace or release-asset name changes; nothing under `hack/` moves, is renamed, or changes what a make target does. One row deserves a sentence rather than a blanket denial, because the diff does change a CRD schema: `spec.apex` on `gateway.cozystack.io/v1alpha1 TenantGateway` gains a `minLength` bound, which lands in the generated `gateway.cozystack.io_tenantgateways.yaml`. The provider hand-types `Package` under `cozystack.io` and `Plan`/`RestoreJob` under `backups.cozystack.io`; `gateway.cozystack.io` is not among them, and a tightening on an existing field adds no new field for it to offer typed access to. The `packages/extra/gateway/README.md` edit sits in the `## Security model` section, below the `## Parameters` block that `cozyvalues-gen` regenerates, so it survives `make generate`. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(gateway)!: the tenant route hostname policy now requires an HTTPRoute or TLSRoute in a `tenant-*` namespace to declare a non-empty `spec.hostnames`. A route that omitted the field, or set it to an empty list, was previously admitted without any hostname being checked against the tenant apex, even though such a route inherits the hostnames of the listener it attaches to rather than matching nothing. A route in a `tenant-*` namespace relying on that inheritance is now rejected and has to name its hostnames. The controller-owned http-to-https redirect route names the tenant apex and its wildcard accordingly, so a request reaching port 80 with a Host outside the tenant apex now gets a 404 instead of a 301. `TenantGateway.spec.apex` additionally gains a `minLength: 1` bound, because the redirect derives its hostnames from that field; a TenantGateway written past the chart with an empty apex is now rejected at admission instead of failing later in the controller. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added edge-terminated gateway support with plain-HTTP listeners and TLS passthrough handling. - HTTP-to-HTTPS redirects now explicitly cover the tenant apex and subdomains. - Gateway configurations require a non-empty apex hostname. - HTTPRoute, TLSRoute, and GRPCRoute resources must declare hostnames. - **Bug Fixes** - Invalid hostname-less routes are now rejected consistently. - Controller-managed redirects no longer trigger unnecessary hostname claims, listeners, or certificates. - Switching certificate modes now cleans up obsolete ACME resources and route status. - **Documentation** - Updated security and gateway guidance for hostname validation, edge termination, route compatibility, and certificate handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
18d02d813a |
fix(cacert): make the comments match the reconciler (#4064)
## What this PR does Comments in the CA-extraction package claim guards the code does not have. No behaviour change; one test is renamed to match the property it pins. - `ManagedLabel` said the reconciler only ever writes over a Secret carrying that label. It does not. `isOurProjection` decides that from the Secret type and the owner reference, and never looks at the label, which any writer in the namespace can strip. - `managedByCozystackLabel` said the reconciler never writes it. It deletes it whenever the selectors digest changes, which is what gets the projection re-admitted by the lineage webhook. - `projectionSuffix` said the dotted canonical name cannot collide with an operator-owned object. A release the apps API created cannot carry a dot, since application names are DNS-1035 labels and `release.prefix` is restricted to `^[a-z0-9-]*$`. The guarantee stops there: the controller reads the release from the `helm.toolkit.fluxcd.io/name` label, where a dot is legal, so a hand-written HelmRelease sits outside it. On the validated path the dot can only come from the suffix an operator or chart appends, which nothing validates at all, and since a canonical name ends in `.tenant-ca` and carries that one dot, a suffix ending in `.tenant-ca` is the only one that reaches a canonical name. The comment says all of that now. - The test pinning that name claimed the property for any engine suffix, in its doc and in its own name, while its loop samples only dot-free ones. It is now `TestProjectionNameCannotCollideWithADotFreeEngineSuffix`, and its doc says what the loop covers. ### Screenshots Not a UI change. ### Downstream repositories Comments and one test name in two Go files. No package, values key, schema field, CRD, metric, release asset or `hack/` target changes, so nothing in the trigger map fires. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified certificate reconciliation behavior, including label handling, projection naming, and source certificate scenarios. * Documented naming guarantees and their API boundaries more precisely. * **Tests** * Updated test descriptions and failure messaging to clarify coverage for dot-free engine suffixes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
4b2507b7a1 |
docs(gateway): align gateway prose with the code
The comment on the shared CEL string said what the hostname requirement is but not what it is worth per kind. Two of the three schemas leave spec.hostnames optional at every version the vendored bundle serves, so for HTTPRoute and GRPCRoute this policy is the only rejection a hostname-less route meets; tlsroutes requires the field at v1 and v1alpha3 and not at v1alpha2, so there the policy closes the gap for as long as v1alpha2 is served. State that where the requirement is stated. The comment on the port-80 allowedRoutes filter credited the cert-manager namespace entry for letting ACME complete, which the paragraph above it retracts: the solver route lives in the tenant namespace, and that is what keeps ACME working. The same sentence claimed the filter keeps app routes off port 80, and the Layer 1 paragraph of the gateway README claimed it too. It does not. A tenant that owns its Gateway is its own gateway namespace, so its app routes are admitted by this very filter; what it excludes is cozy-* and inheriting children. Both copies now say that. Nothing said why naming the apex on the redirect costs no coverage, which is the question a reader has once the route stops matching every host. It holds because of the other half of this change: the port-80 listener admits routes only from the tenant namespace and cert-manager's, and the tightened route policy requires every route there to declare hostnames inside the apex. An inheriting child does not widen that set either, and the comment now says so at the point the hostnames are chosen: the gateway hostname policy compares every listener against the host label of the Gateway's own namespace, so a child apex outside it never gets its listener, the Gateway write carrying it being refused at admission whole, in reconcileGateway and before this route is rendered, while a child apex under this one passes that check and the wildcard already covers it. The empty-apex guard in renderHTTPRedirect, and the test pinning it, both read as though the guard is what spares an operator an unreadable schema error. It is, but the condition stated for it named the number of attached routes, which does not enter into it. In HTTP-01 the listener set is built from route hostnames alone, so reconcileGateway never reads the apex ahead of the guard however many routes attach. DNS-01 and existingSecret render an apex listener, and any tlsPassthroughServices entry builds <service>.<apex>, so in those the operator meets the equivalent error against the Gateway first. State that condition once, at the guard, and let the test defer to it rather than carry a second copy that can drift; the test stays green either way because the fake client does not validate against the CRD schema. The e2e suite explained omitting the TLSRoute empty-list probe by the v1 schema rejecting an empty list. Both halves of that are true and the conclusion does not follow: the probe runs at v1alpha2, which marks spec.hostnames neither required nor minItems, so an empty list reaches admission there exactly as an absent field does. What makes the omission safe is that all three policies render one byte-identical CEL string, so the HTTPRoute pair proves that term already. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b9ecb2e39f |
fix(tenantgateway): name the apex on the http-to-https redirect route
The controller-owned redirect route carried no spec.hostnames, relying on the port-80 listener to match every host. Name the tenant apex and its wildcard instead. A route in a tenant namespace has to declare hostnames inside its own apex to pass the route-hostname admission policy, and a route that names none has nothing to compare against that apex. Naming them also states the coverage the route wants rather than inheriting whatever its listener happens to carry. Both hostnames are required: a wildcard is a suffix match over any number of labels, so "*.<apex>" covers every subdomain at every depth but not the bare "<apex>". Coverage narrows in one direction worth stating: a request arriving on port 80 with a Host outside the tenant apex now falls through to a 404 instead of a 301. Skip this route when collecting hostname claims. Before it carried hostnames it contributed nothing to that set, and letting its hostnames through instead provisions listeners and certificates for them, which fails. The check matches on name and controller ownership, not on the labels the renderer stamps, since those are writable by anyone who can create a route. Deriving hostnames from spec.apex makes that field load-bearing for admission, so it gains MinLength=1 and a matching check in the renderer for clusters whose CRD is older than the binary. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
773d8dabce |
fix(gateway): withdraw TLSRoute acceptance when the mode drops passthrough
Edge terminates TLS at the class provider, so its Gateway carries plain HTTP listeners only and a route pinned to a passthrough section has nothing left to attach to. The api, vm-exportproxy and cdi-uploadproxy endpoints of publishing.exposedServices are the ones that pin one. Hostname claims are empty for every whole-apex mode, so the status pass ran over an empty ref set and left the Accepted=True written while the tenant ran HTTP-01 standing on a route the switch had just unpublished. Whoever went looking for the dead endpoint read that condition as proof the route was fine. The rule this settles on is that the controller owns a condition only where it judges. Edge judges every attached route by the absence of the listener it needs and writes Accepted=False with Gateway API's NoMatchingParent, the reason defined for a parentRef whose port or sectionName matches no listener. The other whole-apex modes render those listeners and serve them but collect no claims, so they hold no opinion: they drop the entry rather than leave a withdrawal standing on a listener that is serving, naming a class provider that no longer terminates anything. HTTP-01 collects claims and has already judged every route this pass could speak about. A route declaring no hostnames is judged by none of them, because claims are gathered inside a loop over spec.hostnames and TLSRoute v1alpha2 sets no minItems on that field. Withdrawing one would write a condition with no path back, since HTTP-01 renders the listener again and judges only the routes that produced claims, so both directions skip it. The tests read the controller name from the constant the controller stamps rather than restating it, so a helper cannot go looking for a name that is no longer written and report absence it never checked. The switch already deletes the redirect route, the Issuer and the wildcard Certificate it invalidates. The statuses it owns were the one thing left behind. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5976ddf02f |
fix(gateway): normalise the GatewayClass name and refuse a blank one
The class lists arrive over the values channel comma-joined, so every reader trims each entry as it splits them. The scalar those entries are compared against was not trimmed anywhere. A class name carrying a stray space therefore failed `has` against its own padded self, and the two readers of that scalar disagreed about the same class: the gateway chart put the tenant on an edge-terminated class, while the cluster-wide ACME issuers still pinned their solver to a listener an edge Gateway does not render. That stops every HTTP-01 certificate in the cluster from issuing, with nothing in any status naming the cause. Normalise once at the single writer so both readers see a clean value, and keep a defensive trim at each reader. The tenant's own choice does not travel that channel — it comes from tenant.spec.gatewayClass — so it is trimmed where the gateway chart reads it. Unset and whitespace-only are different events and no longer share an outcome. Unset means the choice was not made, and falling back to the bundled class stays its documented behaviour. Whitespace-only means a choice was made and is garbage, in the field that decides where TLS terminates: resolving it quietly would take a tenant the operator meant to put behind an edge terminator and publish it directly, with a real certificate on a real address. Both the platform default and the tenant's own choice now refuse it by name. The kind check beside that refusal is part of the same fix, not a tidy up. Through Helm's values path an unquoted numeric scalar arrives as a float64, and the whitespace check compares it against a string, so `className: 123` dies at that comparison — one line before the trim — with an error naming neither the key nor the file. Refusing whitespace without checking the kind first would therefore have made the numeric case worse than it was. `kindIs "string"` is the right test because it never has to name the type that actually arrived. The order of the guard's steps is part of the guard. The value is read before anything normalises it, so a default that fires on any falsy value cannot launder 0 or false into the empty string and resolve them to the bundled class ahead of the kind check. Unset is the one case that still falls back, and each of those four inputs is pinned, so a normalisation drifting back ahead of a check fails rather than quietly covering only the loudest bad input. Also correct a cleanup log line: the wildcard Certificate is deleted for every mode that is not DNS-01, so naming HTTP-01 in that message is wrong for existingSecret and for edge alike. And state the listener name invariant plainly: the cap is 253, not the 63 of a DNS label, and the longest name the builders produce is 84. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
7851969448 |
docs(gateway): document GatewayClass selection and edge TLS termination
An operator setting up a class whose provider terminates TLS upstream had no page answering their questions: which platform values bound the choice, what a tenant writes to make it, what a move between classes does to certificates and to the replicated wildcard Secret, and how the mechanism fails when a configured class name matches nothing installed. Add docs/gateway-classes.md covering that surface. Describe the mode and its consequences in the chart README alongside the other three, including the one constraint stated nowhere else: the publishing tenant cannot select a class whose TLS termination differs from the platform default, because the cluster-wide ACME issuers derive their solver pin from that default alone. State the blast radius where it actually applies. The endpoints an edge Gateway stops publishing exist only where those routes are rendered at all, and both pages say what the transition does to the route conditions the controller owns, so a reader is not left inferring it from the code. The security model names the selector each listener really carries. The port-80 listener selects on the built-in kubernetes.io/metadata.name label, which kube-apiserver writes and a namespace cannot forge; the HTTPS and TLS-passthrough listeners select on namespace.cozystack.io/gateway, written by the tenant chart and the controller, so that guarantee does not extend to them. What closes the exposure is that Namespace is cluster-scoped with no tenant verb on it, and that the route-hostname policies bound what a labelled namespace could publish. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
57c51f58ee |
feat(gateway): stop replicating the wildcard key into edge namespaces
The wildcard-secret controller replicates the operator's TLS Secret, private key included, into every namespace that owns a termination point, and a namespace owning a Gateway counted as one without further question. A Gateway whose class ends TLS at its provider's edge terminates nothing locally, so the replica has no consumer there. Read certMode off the tenant's own TenantGateway rather than a cluster-wide value, because the class is a per-tenant choice. Keep the key whenever the answer is not a confident no: a namespace that also owns an ingress controller still terminates locally, a namespace holding more than one TenantGateway keeps it while any of them is not at the edge, and an unreadable list or a namespace whose TenantGateway has not appeared yet keeps it too, so a transient API error never withdraws a key a Gateway may still be serving. Watch TenantGateway, so the withdrawal follows the class change rather than waiting for the periodic resync. The watched set is a value the tests assert on, because the controller-runtime builder exposes nothing about what it watched: a dropped watch would otherwise leave both suites green while the withdrawal quietly degraded to that resync, leaving the private key in a namespace that stopped terminating TLS. Pruning the replica is not the same as leaving the namespace key-free. The Certificate objects go, but the Secrets they issued stay, as they do on any other mode transition. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
2ef0e8d3a1 |
feat(gateway): add an edge cert mode for upstream-terminated classes
Some GatewayClasses have their provider terminate TLS in front of the Gateway and reach it over plain HTTP. On such a class the platform issued origin certificates nobody serves, and where ACME could not complete, the Gateway's listeners never became ready. Add certMode=edge to TenantGateway. The controller renders the apex, its wildcard and every inheriting child apex as plain HTTP listeners carrying the same label-based allowedRoutes as the HTTPS listeners of the other modes, so application HTTPRoutes attach by hostname unchanged. It mints no Issuer and no Certificate, and removes the http-to-https redirect route, because the edge performs that redirect itself. TLSPassthroughServices are not rendered: an HTTP-only edge cannot carry a TLS listener, which leaves the endpoints relying on one unpublished through such a Gateway. The hostname-less port-80 listener is left out. It carries the ACME challenge and the redirect in the other modes and has neither here, so it would only be a catch-all admitting any hostname. Leaving it out also keeps a hostname-pinned listener at index 0, which is what Cilium's first-listener-wins namespace check reads before the fix in cilium/cilium#45693. Moving into the mode garbage-collects the Issuer and Certificate objects the controller owned. The TLS Secrets they issued stay behind: cert-manager runs with enableCertificateOwnerRef false, so a Certificate does not own the Secret it issued into. Every mode transition already behaves that way. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
fb740db0b0 |
docs(security): separate the fail-open Gateway-listener policy from the fail-closed route policies
The self-assessment summarised the whole hostname policy set as fail-closed, while its own table row states that the Gateway-listener policy passes a Gateway whose namespace carries no host label. Say which policies are which. The port-443 kinds comment in the reconciler test also tied the GRPCRoute admission rule to the port-80 listener; the rule matches the kind wherever it attaches, and the port-443 exclusion is a least-privilege choice, not the reason the rule exists. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
af19aa933c |
docs(security): record GRPCRoute in the route hostname policy set
The port-443 kinds set still excludes GRPCRoute, TCPRoute and UDPRoute. cilium#45559 governs only that every port-443 listener carries the same set, not what is in it, so the contents are a least-privilege choice: nothing the platform ships needs gRPC, TCP or UDP routing on port 443. TCPRoute and UDPRoute additionally carry no hostname and no admission rule gates them. The gateway README described HTTPS and TLS-passthrough listeners as carrying different kinds. They carry the same one, which is what cilium#45559 requires. The threat model and the self-assessment enumerate which route kinds the hostname policies cover, and both listed two. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
9ea4a961ec |
fix(cacert): make the comments match the reconciler
Comments in the CA-extraction package described guards the code does not have. ManagedLabel: the anti-clobber decision is made by isOurProjection, from the Secret type and an owner reference back to the sentinel, not from the marker, which no decision keys on beyond the drift check. managedByCozystackLabel: the reconciler does mutate it. It deletes the label on a selectors-digest change, which is what hands the projection back to the lineage webhook for a fresh verdict. projectionSuffix: the projection name is kept clear of engine CA Secrets by a dot, and a release name cannot carry one, but the suffix an operator appends can and nothing validates it. A suffix ending in ".tenant-ca" is the only one that reaches a canonical name. TestProjectionNameCannotCollideWithADotFreeEngineSuffix: both its doc and its name claimed the property for any engine suffix while the loop samples only dot-free ones, which is the same overstatement one file over. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
3d076099a2 |
docs(postgres): name the trust anchor as the tenant actually sees it
The README told a tenant to fetch <release>.tenant-ca, but the release name carries the postgres- prefix the application definition applies, so the object is postgres-<name>.tenant-ca. Following the old text returned NotFound. Also drop two comments promising a companion ValidatingAdmissionPolicy that pins sentinel writes to helm-controller. No such policy ships: the proposal for it was rejected because authorization by username duplicates RBAC in a more brittle form. The RBAC boundary is the whole control. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Timofei Larkin
|
95e4faf68e |
feat(workloadmonitor): bucket metrics endpoint override
The controller discovers where to query SeaweedFS bucket size metrics via the namespace.cozystack.io/monitoring label, which assumes the monitoring stack that scrapes SeaweedFS runs in the same cluster. Some deployments host SeaweedFS and its metrics in a separate cluster, so add a --seaweedfs-metrics-endpoint flag to cozystack-controller (and a matching cozystackController.seaweedfsMetricsEndpoint chart value): an explicit base URL of a Prometheus-compatible query API, with scheme, host, optional port and path prefix; /api/v1/query is appended. Basic auth may be embedded as URL userinfo. The flag is validated and normalized at startup, so a malformed URL fails the rollout rather than the queries. Empty keeps label-based discovery unchanged. Bucket sizes feed billing, so metrics failures no longer collapse to "no size": previously any query failure yielded an empty result map and the reconciler overwrote Workload resources with it, silently wiping the recorded sizes. The reconciler now retains the last known s3-storage-bytes and s3-physical-storage-bytes values, emits a BucketMetricsUnavailable warning event on the WorkloadMonitor, and returns the error from Reconcile so it lands in the reconcile error metrics. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Aleksei Sviridkin
|
37a91f98b8 |
feat(cozystack-controller): extract application CA into a key-free tenant Secret
Every catalog engine issues a per-release CA, but the object holding ca.crt usually holds a private key beside it (cert-manager's tls.key, CloudNativePG's ca.key), so granting a tenant read on it hands over key material too. Add a controller that publishes the trust anchor as one canonical, key-free Secret per release, "<release>.tenant-ca", holding only ca.crt. A chart declares the anchor by rendering a namespaced TenantProjection sentinel (internal.cozystack.io/v1alpha1) that names the source Secret and key. The controller rebuilds ca.crt from the parsed certificate DER rather than copying the source, refuses any value carrying private key material, and owns the projection by owner reference so a foreign Secret at the same name is never overwritten. No tenant RBAC grants any verb on internal.cozystack.io; a ValidatingAdmissionPolicy pins writes to helm-controller as defence in depth. The controller does not grant tenant read access — the lineage webhook does, from the ApplicationDefinition's spec.secrets. This side stamps the engine-agnostic internal.cozystack.io/tenant-ca label and forces a re-admission when those selectors change. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrei Kvapil
|
fbbe4af7df |
refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass (#3218)
## What Removes the `network.cozystack.io` API group — `ExposureClass` + `ServiceExposure` (#3081) — and its reconciler, in favor of native `Service` `type: LoadBalancer` + `loadBalancerClass`. ## Why Full rationale and discussion in #3164. For **managed applications** the chart already owns the Service, so: - `type: LoadBalancer` + `loadBalancerClass` + an admin-provisioned pool covers vendor-neutral external exposure without a dedicated API group and controller; - the `externalIPs` (node-IP) default never went through `ServiceExposure` (its backend is a no-op) and is left unchanged; - the only current consumer, the host ingress, keeps rendering `type: LoadBalancer` and just drops the `ServiceExposure`; the admin provisions the address pool / announcer out of band, as the install docs already assume. `ServiceExposure`'s one unique feature — dynamic per-namespace scoping of a shared VIP pool — is a convenience the managed-app path does not need, and it does not justify a second tenant-networking API group next to `sdn.cozystack.io`. ## Changes - Delete `api/network/v1alpha1` and `internal/controller/serviceexposure` (+ CRD definitions, e2e bats, controller registration, RBAC). - Host ingress: drop the `ServiceExposure` template; add an optional `publishing.loadBalancerClass` that sets `spec.loadBalancerClass` on the host-ingress LoadBalancer Service. - The `externalIPs` path and default behavior are unchanged. ## Verification - `go build ./...` — green. - `helm unittest` — `core/platform` 72/72, `extra/ingress` 25/25 (incl. host-vs-tenant `loadBalancerClass` scoping and the immutable-field migration guard). ## Migration None required. `network.cozystack.io` (#3081) has never shipped in a release — it is on `main` only (not backported to `release-1.5`; absent from v1.4.x / v1.5.x), so no upgrading cluster has these CRDs and this removes it before it is ever released. Clusters that track `main` and explicitly set `publishing.exposureClass` (opt-in, only since 2026-06-30) should provision their MetalLB / Cilium address pool themselves and set `publishing.loadBalancerClass`; any orphaned `cozystack-<class>` pool can be deleted by hand. Default clusters (externalIPs path) are unaffected. ## Status **Draft / RFC for the #3164 discussion** — this removes an API group that landed only days ago, so nothing here should merge before the design call. cc @lexfrei (author of #3081) and @lllamnyp (#3164). Deferred: when Kubernetes actually removes `Service.spec.externalIPs`, the node-IP default migrates via a MetalLB `IPAddressPool` with no `L2Advertisement` (allocate-but-don't-announce) — a separate change. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for configuring the ingress host Service with a native `loadBalancerClass` value. * Expanded chart tests to verify class wiring and immutability behavior across install and upgrade scenarios. * **Bug Fixes** * Removed the legacy exposure-class/service-exposure flow and related controller behavior. * Updated templates and CRD packaging so the platform no longer renders or depends on those deprecated resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
3dbc7d3662 |
refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass
Remove the network.cozystack.io API group (ExposureClass + ServiceExposure) and its reconciler. Managed-application charts own their Service, so native type: LoadBalancer + loadBalancerClass + an admin-provisioned pool covers external exposure without a dedicated API group and controller. The host ingress -- the only consumer -- keeps rendering type: LoadBalancer and gains an optional publishing.loadBalancerClass to pick the LB controller; the externalIPs (node-IP) default path is unchanged. Rationale in cozystack#3164. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
c841c70866 |
feat(operator): replicate operator wildcard certificate to tenant namespaces
The operator-provided wildcard MVP serves the root tenant only: the root ingress controller and Gateway read a TLS Secret from their own namespace. Child tenants run their own ingress controller / Gateway in a separate, default-deny namespace and cannot read that Secret cross-namespace. Add a controller that mirrors the operator wildcard Secret into every tenant namespace that owns a TLS termination point (its namespace.cozystack.io/ingress or .../gateway label equals its own name). The source is identified by name from the platform values channel (_cluster.wildcard-secret-name) with no extra operator input; replicas carry a managed-by label and refresh on rotation. An individual replica is removed when its namespace stops terminating TLS; every replica is torn down only on an explicit disable (clearing wildcard-secret-name). A missing or mistyped source, or a transiently-absent values channel, keeps existing replicas, so tenant TLS never drops on a transient gap. The controller's Secret cache is scoped to managed replicas and the values channel, not every cluster Secret. Only the platform controller's ServiceAccount gains Secret writes — no tenant RBAC is widened, and every consumer reads only its own-namespace copy. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Алексей Артамонов
|
184324430f |
fix(controller): don't clamp pool resources the parent budget leaves unbounded
ComputePools derived a pool's Available with SubtractWithNonNegativeResult over the union of the budget and carve-out keys, so a bounded child carving a resource its parent does not bound surfaced that resource as 0 in the parent pool's Available. The controller then wrote a tenant-quota-allocated clamping the parent namespace to zero of a resource it never limited, blocking its workloads (observed on a cluster: a memory-bounded child froze its cpu-only parent's memory). Mask Available to the budget's resource names so only resources the pool root actually bounds are enforced; mirrors the fix already applied to Pool.Overcommitted. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com> |
||
|
Алексей Артамонов
|
b0556cc753 |
fix(controller): ignore unbounded resources in tenant quota overcommit check
Pool.Overcommitted reported a resource as overcommitted whenever a sub-tenant carved out a resource the pool root does not bound. Admission explicitly allows a child to bound a resource its parent leaves unbounded, and an unbounded budget can never be overcommitted, so this produced a spurious QuotaOvercommitted warning event on every reconcile. Restrict the check to resources present in the pool budget. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com> |
||
|
Andrei KvapilandClaude
|
55f1edda78 |
fix(controller): read tenant budget from rendered quota, skip inert pools
Take a tenant's declared budget from its chart-rendered tenant-quota ResourceQuota (.spec.hard) instead of the raw tenant.spec.resourceQuotas values. The chart expands the shorthand values into real quota resource names and applies the cluster's request/limit allocation ratios, so reading the rendered object keeps the controller-owned allocated quota in exactly the same key space — the two quotas compose correctly and the controller never replicates that cluster-configurable transformation. Also skip pools that need no tightening beyond the chart quota (a lone tenant with no sub-tenants and no carve-outs): the controller writes an allocated quota only when a budget is actually shared or carved up, so a single-tenant namespace is left exactly as the chart renders it. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei KvapilandClaude
|
844deb170f |
feat(controller): enforce recursive tenant quotas at runtime
Add a tenant quota reconciler to cozystack-controller that realises the hierarchical (sub-tree) quota model at runtime. Each tenant's declared quota is the budget for its whole sub-tree: a child that declares its own quota carves a fixed slice out of the parent budget, while a child without a quota shares the parent's remaining pool. The controller maintains a per-namespace 'tenant-quota-allocated' ResourceQuota (Kubernetes enforces the most restrictive quota, so it binds without fighting Flux over the chart-rendered 'tenant-quota'), aggregating sub-tree usage so a pool's members collectively stay within the budget. A --tenant-quota-buffer-percent flag temporarily inflates pool budgets so workloads already over a freshly-introduced quota keep running during rollout; overcommitted pools emit a QuotaOvercommitted event. The model and the selector-spans-namespaces/aggregated-usage approach are adapted from OpenShift's ClusterResourceQuota controllers (openshift/api, library-go, cluster-policy-controller, apiserver-library-go), Apache-2.0, Copyright Red Hat, Inc. and the OpenShift Authors. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
31c15b282f |
feat(network): mechanism-agnostic LoadBalancer exposure (ExposureClass + ServiceExposure) (#3081)
## What this PR does Lays the foundation for migrating tenant LoadBalancer exposure off the deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off MetalLB onto Cilium-native announcement — the mechanism-agnostic shape prescribed when the earlier chart-side approach (#2468) was reverted in #2603. It adds a new `network.cozystack.io/v1alpha1` API group and a controller that translates intent into whichever LoadBalancer backend the cluster admin configured: - `ExposureClass` (cluster-scoped, admin): binds a logical class name to a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP address scope — the StorageClass/IngressClass pattern. - `ServiceExposure` (namespaced, tenant intent): "expose this Service via class X"; status reports the resolved backend and assigned IPs. - A `serviceexposure` controller renders the backend's pool + announcer once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium `CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to the namespaces that opted in. Backends are pluggable behind a small Go interface, so adding BGP or another cloud LB is one implementation, not an API change. The controller never mutates the target Service — the chart still owns its shape. Platform wiring: `publishing.exposureClass.*` renders the admin `ExposureClass` and the host ingress publishes through it. The default (no class set, `externalIPs`) leaves every existing cluster's behaviour byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes a single `ExposureClass.spec.backend` change. This is the foundation only — Part of #2586. Converting per-app exposure, flipping Cilium `l2announcements` on, changing the default, and removing MetalLB are deferred to later phases. ### Release note ```release-note feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added new `ExposureClass` and `ServiceExposure` custom resources with support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB), including optional L2 and address configuration. * Added Helm wiring and templates to enable service/ingress publishing via `ServiceExposure`, selecting a configured (or default) `ExposureClass`. * **Bug Fixes** * Improved reconciliation, finalizer-based cleanup, and status updates for readiness and assigned IP reporting, including safer handling when backends or managed resources change. * **Tests** * Added end-to-end and unit test coverage for backend rendering, readiness, and cleanup flows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
shreyaabaranwal
|
6acf34aef2 |
test(gateway): pin full port-443 kinds contract across all cert modes (cilium#45559)
Signed-off-by: shreyaabaranwal <shreyabaranwal229@gmail.com> |
||
|
shreyaabaranwal
|
786c098ac6 |
fix(gateway): pin identical allowedRoutes.kinds on port 443 listeners (cilium#45559)
Signed-off-by: shreyaabaranwal <shreyabaranwal229@gmail.com> |
||
|
Aleksei Sviridkin
|
664704d7b6 |
feat(controller): reconcile ServiceExposure into backend LB resources
Add the serviceexposure controller to the cozystack-controller binary. It resolves a ServiceExposure's ExposureClass to a backend and renders the backend-specific pool and announcer CRs, then reports the assigned IPs in status. Backends are pluggable behind a small interface, one per file: - externalIPs: renders nothing (the node owns the address) — preserves the historical default behaviour. - metallb: IPAddressPool + L2Advertisement in cozy-metallb, scoped to the target Service's namespace. - cilium: CiliumLoadBalancerIPPool (cilium.io/v2) + CiliumL2AnnouncementPolicy (cilium.io/v2alpha1), scoped to the Service via serviceSelector. - robotlb: renders nothing (the cloud LB owns allocation and announcement). The rendered CRs belong to foreign API groups not in the Go scheme, so they are emitted as unstructured objects. They carry no OwnerReference (the Cilium CRs are cluster-scoped and the MetalLB CRs live in a different namespace than the owning ServiceExposure), so ownership is tracked via a management label plus owner annotations, cleanup is driven by a finalizer, and the controller refuses to take over an identically-named resource it does not manage. The controller never mutates the target Service, leaving its shape to the chart that renders it. Grant the controller write access to metallb.io and cilium.io pool / announcer resources. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4262658f0f |
feat(tenantgateway): add existingSecret cert mode for operator wildcard
Operators who already hold a wildcard TLS certificate (purchased, or issued by a corporate CA) had no way to make a tenant Gateway serve under it: the only cert modes were http01 (per-listener ACME) and dns01 (controller-minted wildcard via ACME). Add a third mode, existingSecret, that references a pre-existing TLS Secret in the TenantGateway's own namespace via spec.wildcardSecretRef. In this mode the controller mints no ACME Issuer and no Certificate; the wildcard, apex, and per-child-apex HTTPS listeners reference the supplied Secret directly. A missing wildcardSecretRef fails fast with Ready=False rather than producing a Gateway pointing at a nonexistent Secret. Switching into existingSecret from http01/dns01 reclaims the now-unused Issuer and Certificates so no ACME machinery is left behind. The non-cert listeners (http->https redirect, TLS passthrough) are unaffected and continue to render. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
320172137e |
fix(tenantgateway): include inheriting child namespaces in hostname claim collection
The HTTP-01 inheritance flow deadlocked at the route-Accepted step:
the apps/tenant chart correctly labelled the child namespace with
`namespace.cozystack.io/gateway=<owner>`, the parent Gateway's
allowedRoutes selector matched by label, but `collectHostnameClaims`
filtered routes by the static set `{tgw.Namespace} ∪ Spec.AttachedNamespaces`
— so a child route in a namespace reached purely via the inheritance
label was silently dropped. No per-listener Certificate, no per-app
HTTPS listener, no hostname match → `Accepted=False,
Reason=NoMatchingListenerHostname` indefinitely.
Caught by the new `child tenant's HTTPRoute attaches to parent's
Gateway via inheritance label` e2e bats — failed 3/3 retries on the
CI run for `afc7765e6`. The Go controller-level
TestReconcile_HTTP01CollectsHostnamesFromInheritingChildNamespaces
test pins the same contract at a faster feedback loop: an HTTPRoute
in a namespace labelled `namespace.cozystack.io/gateway=<owner>` but
NOT listed in Spec.AttachedNamespaces must still cause the
controller to render a per-listener HTTPS listener for its hostname.
The fix extends the `allowed` set in `collectHostnameClaims` to
also include namespaces matched by the gateway label selector, so the
hostname-claim collector and the Gateway's allowedRoutes selector
agree on which namespaces can attach. DNS-01 mode is unaffected — it
returns nil from `collectHostnameClaims` immediately and uses the
separate `collectInheritingChildApexes` path for SAN expansion.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
99fe080def |
feat(tenantgateway): DNS-01 SAN expansion + listeners for child apexes
In DNS-01 mode, the parent's wildcard Certificate now covers every inheriting child tenant's apex (`<child-apex>` + `*.<child-apex>` as additional SANs) and the parent Gateway grows one wildcard HTTPS listener per child apex referencing the same Certificate. Without the expansion, a child route hostname like `harbor.alice.example.org` fails to match the parent's listeners — `*.example.org` is a single- label wildcard and `alice.example.org` is two labels deep — and the TLS handshake fails the SNI check because the parent cert never listed the child apex. The controller discovers inheriting children by listing namespaces labelled `namespace.cozystack.io/gateway = <tgw.Namespace>` (the existing inheritance contract from Stage 1/3) and reading their `namespace.cozystack.io/host` label, which apps/tenant chart writes to the child apex. Children with no host label are skipped (defensive); a child host equal to the parent apex is deduplicated (cert-manager rejects duplicate dnsNames). HTTP-01 mode is unchanged — its per-listener / per-cert flow already handles child hostnames without any wildcard. Listener naming: `https-child-<first-label>-<8-hex>`, where the hex suffix is sha256(child-apex)[:8] so two children whose first DNS label collides produce distinct listener names. Cap: Gateway API hard-caps spec.listeners at 64 entries. The parent uses 3 slots (http + https + https-apex) leaving ~61 wildcard children per parent. Operators whose subtree fans out beyond that must run the high-fanout branch on its own Gateway via tenant.spec.gateway=true. Tests (TDD red→green): * TestReconcile_DNS01WildcardCertCoversInheritingChildApexes pins the SAN list contains both `<child-apex>` and `*.<child-apex>`. * TestReconcile_DNS01WildcardCertDeduplicatesChildApexEqualToParent pins the dedupe path (child host == parent apex). * TestReconcile_HTTP01WildcardCertNeverRendered pins the inverse: HTTP-01 mode never renders a wildcard Certificate. * TestReconcile_DNS01GatewayHasListenerPerChildApex pins the listener-side expansion (one `*.<child-apex>` listener per inheriting tenant, all referencing the parent's wildcard cert). Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
30302eabe4 |
feat(tenantgateway): switch Gateway allowedRoutes to label selector
The Gateway's HTTPS / TLS-passthrough listeners now match attached namespaces by the namespace.cozystack.io/gateway label keyed on the TenantGateway's own namespace, instead of a static kubernetes.io/metadata.name In [list] whitelist. This restores inheritance parity with the legacy ingress flow: child tenants of a Gateway-owning ancestor publish through the ancestor's Gateway by default — without their own LB IP / Certificate, and without operator intervention to add each child by name to platform values. Two writers populate the label: * apps/tenant chart namespace.yaml — each tenant namespace gets a label pointing at the nearest ancestor that owns a Gateway (self if owning, inherited otherwise). The tenant Helm chart is the source of truth for tenant-tree labels; the controller never touches them (annotation-gated GC). * cozystack-controller (ensureNamespaceLabels) — patches the label onto every namespace in tgw.Spec.AttachedNamespaces, so cozy-* system namespaces (cert-manager, monitoring, harbor, …) reach the publishing Gateway alongside the tenant tree. Tracks its own writes via the cozystack.io/gateway-attached-by annotation; on a reconcile where an entry is removed from AttachedNamespaces, the label is garbage-collected (annotation present + namespace no longer expected → strip both label and annotation). Also adds the gateway-attach label to tenant-root itself via cozystack-basics/templates/tenant-root.yaml when _cluster.gateway-enabled is true, so the Gateway in tenant-root's own namespace passes its own selector for the controller-owned http→https redirect HTTPRoute. Tests (TDD, red→green): * TestReconcile_HTTPSListenerUsesGatewayLabelSelector pins the new MatchLabels shape on HTTPS listeners. * TestReconcile_LabelsAttachedNamespaces pins the patch path: every namespace in spec.AttachedNamespaces (plus tgw.Namespace) carries namespace.cozystack.io/gateway=<owner> after reconcile. * TestReconcile_LabelGCRemovesDroppedAttachedNamespaces pins the GC path: removing an entry between reconciles strips the label. * TestReconcile_LabelGCDoesNotStripHelmOwnedLabels pins the safety contract: labels lacking the controller annotation (Helm-owned tenant labels) are left alone. * tenant-root_test.yaml adds the positive case for the conditional label render under _cluster.gateway-enabled=true. The HTTP listener (port 80) keeps the narrower static-name selector — it allows only the tenant namespace plus cozy-cert-manager (for HTTP-01 ACME challenges) and must NOT accept routes from app namespaces or child tenants, since unattended routes on the HTTP listener serve plaintext and leak credentials. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d2e851f63b |
feat: Gateway API support via Cilium
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Andrei KvapilandClaude
|
4d5486cd84 |
chore(controller): remove dead dashboard controller
The dashboard controller used to project ApplicationDefinitions into a set of dashboard.cozystack.io CRDs (Sidebar, Factory, Navigation, Breadcrumb, CustomFormsOverride, CustomFormsPrefill, CustomColumnsOverride, MarketplacePanel, TableUriMapping, CFOMapping, BreadcrumbInside) consumed by the legacy UI. cozystack-ui now reads ApplicationDefinitions directly, so the controller, its API types, the generated CRDs and the related RBAC rules are no longer used. Removes: - internal/controller/dashboard - api/dashboard/v1alpha1 - packages/system/cozystack-controller/definitions/dashboard.cozystack.io_*.yaml - dashboard.cozystack.io wiring from cmd/cozystack-controller/main.go - dashboard.cozystack.io RBAC rules from cozystack-controller and cozy-dashboard charts - ./api/dashboard/... codegen path from hack/update-codegen.sh Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Myasnikov Daniil
|
6105882856 |
fix(dashboard): address review feedback on RestoreJob views
- Replace rename-only wording on spec.targetApplicationRef form labels so users discover that the field also supports restoring into a different application, not just renaming. - Drop VMInstance-specific keys from the spec.options form label; the restore options are driver-specific, so hardcoding one driver's keys is misleading once other drivers land. - Render a single "Same as backup" fallback for an omitted spec.targetApplicationRef on the details page instead of the noisy "-.-/-" produced by concatenating three missing path fallbacks. - De-duplicate the time-block / time-icon / time-value component IDs within the restorejob-details factory by prefixing them with created-, started-at-, and completed-at-. - Mark non-CRD-backed stock-project-factory-*-details sidebars (kube-*, plan, backupjob, backup, restorejob) as static in upsertMultipleSidebars so they pick up consistent managed-by labels instead of being left unmanaged. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
310b0ece6b |
feat(dashboard): add restorejob list and add page and sidebar link
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
2c141d3f38 |
feat(platform): add resourcePreset labels
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
ZverGuyandClaude
|
3803e6e070 |
fix(controller): use dedicated HTTP client instead of http.DefaultClient
Replace http.DefaultClient with a package-level *http.Client with an explicit 10-second timeout. Avoids sharing the process-wide default transport with other libraries. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
c72fadec3e |
fix(controller): scope Prometheus query to only matched bucket names
Pass bucket names from BucketClaim.Status.BucketName into the PromQL query as a bucket=~"name1|name2" filter. This prevents O(N²) load where N WorkloadMonitors each fetch all buckets globally. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
8183b4669a |
perf(controller): skip Prometheus calls when no BucketClaims matched
Wrap resolvePrometheusURL and queryAllBucketMetrics in a len(bucketClaimList.Items) > 0 guard. Avoids unnecessary namespace GET and HTTP request to Prometheus on every reconcile of non-bucket WorkloadMonitors (postgres, redis, kubernetes, etc.). Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
475d24b029 |
perf(controller): batch all bucket metrics into single Prometheus query
Replace 2×N per-bucket HTTP requests with a single query that fetches
all SeaweedFS bucket size metrics at once:
{__name__=~"SeaweedFS_s3_bucket_(size|physical_size)_bytes"}
Results are keyed by bucket name in memory, then looked up per
BucketClaim. This reduces HTTP round-trips from 2N+1 to 2 (one
namespace lookup + one Prometheus query) regardless of bucket count.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
|
||
|
ZverGuyandClaude
|
ed9808fad6 |
perf(controller): resolve Prometheus URL once per reconcile, not per BucketClaim
Move resolvePrometheusURL call before the BucketClaim loop and pass the URL as parameter. Avoids redundant namespace lookups and reduces reconcile time from 2×N+1 to 2×N HTTP calls (where N = BucketClaims). Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
260ce84e5f |
fix(controller): distinguish empty buckets from monitoring unavailable
Change queryPrometheusMetric to return (int64, bool) so callers can emit s3-storage-bytes=0 for empty buckets while omitting the field entirely when monitoring is not configured. Also: - Add RBAC marker for core/namespaces GET (used by resolvePrometheusURL) - Log namespace read errors instead of silently returning empty URL - Add TestQueryPrometheusMetricZeroValue test Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
a6f74f3198 |
refactor(controller): remove s3-buckets count resource from bucket workloads
Bucket existence is already tracked via LifetimeHours. The s3-buckets count resource produced a meaningless "s3-buckets-Hours" fallback type in billing. Only storage size metrics (s3-storage-bytes, s3-physical-storage-bytes) are now set on bucket Workloads. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
40620b2911 |
fix(controller): fix nil map panic and scientific notation parsing
- Initialize workload.Labels map inside CreateOrUpdate mutate function to prevent nil map panic when existing Workload has no labels - Use strconv.ParseFloat instead of resource.ParseQuantity for Prometheus metric values, which may use scientific notation (e.g. "1.048576e+06") that ParseQuantity does not support Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
7f7f4b218d |
refactor(controller): resolve Prometheus URL from namespace labels
Replace static --prometheus-url flag with dynamic resolution from namespace.cozystack.io/monitoring label. Each tenant namespace knows which tenant hosts its monitoring stack, so the controller constructs the vmselect URL automatically. This correctly handles multi-tenant setups where different tenants may use different monitoring instances. - Remove PrometheusURL field from WorkloadMonitorReconciler struct - Remove --prometheus-url flag and prometheusUrl chart value - Add resolvePrometheusURL() that reads namespace label - queryPrometheusMetric() now accepts prometheusBaseURL as parameter - Add tests for resolvePrometheusURL with and without label Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
0e6b8f7ba1 |
fix(controller): address review findings for BucketClaim reconciler
- Set replicas/minReplicas to 0 in bucket WorkloadMonitor (buckets have no pods, minReplicas=1 would mark monitor as not operational) - Remove dead IsNotFound check on List (List returns empty, not 404) - Trim trailing slash from PrometheusURL before path append - Add strings import for TrimRight Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
bfdfe989e0 |
fix(controller): add HTTP status check and limit response body size
- Check resp.StatusCode before parsing Prometheus response - Limit response body read to 1 MB via io.LimitReader - Use strings.HasPrefix in test instead of fragile slice indexing - Add TestQueryPrometheusMetricServerError for 500 responses Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
04f340ab84 |
feat(controller): add physical storage size metric for S3 buckets
Query SeaweedFS_s3_bucket_physical_size_bytes alongside the logical size metric. Physical size includes all replicas and reflects actual disk usage, while logical size reflects what the user stored. Refactor queryBucketSizeBytes into generic queryPrometheusMetric to reuse for both metrics. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |
||
|
ZverGuyandClaude
|
0aab19f500 |
feat(controller): add SeaweedFS bucket size metrics via Prometheus
Query SeaweedFS_s3_bucket_size_bytes from a Prometheus-compatible API to populate s3-storage-bytes resource on bucket Workloads. The Prometheus URL is configurable via --prometheus-url flag. When set, bucket WorkloadMonitors are requeued every 60s to keep sizes current. When Prometheus is not configured, buckets still get tracked with s3-buckets=1 for existence-based billing. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: ZverGuy <maximbel2003@gmail.com> |