mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 14:02:56 +09:00
Aleksei Sviridkin
b167fd112a
## What this PR does Lets a tenant publish through a GatewayClass whose provider terminates TLS upstream of the Gateway, and gives the platform the two knobs that bound the choice. The first consumer is the Cloudflare Tunnel controller from #3858, which ends TLS at the Cloudflare edge and reaches the Gateway over the tunnel, but nothing here is tunnel-specific. Until now every cert mode rendered HTTPS listeners with `certificateRefs` and tied the TenantGateway's `Ready` to their issuance. On such a class the platform issued origin certificates nobody serves, and where ACME could not complete, the tenant's `gateway` release never became Ready at all. **Who picks the class.** The platform names a default in `gateway.className` and the set a tenant may choose from in `gateway.tenantSelectableClasses`; a tenant picks its own with `tenant.spec.gatewayClass`. That set is an allowlist rather than free choice because Tenant is a tenant-writable object while GatewayClass is cluster-scoped: without it a tenant could attach its Gateway to a class an admin keeps for internal-only traffic. Naming anything outside the set fails that tenant's own gateway release with a message listing what is allowed, and reaches no other tenant. **Where TLS ends.** That is a property of the class, not of the tenant, so it is declared once in `gateway.edgeTerminatedClasses`, and whichever class a tenant lands on decides its `certMode`. The two therefore cannot be configured out of step: a tenant on a tunnel class cannot keep issuing origin certificates because a separate switch still says otherwise. In `edge` the controller renders the apex, its wildcard and every inheriting child apex as plain HTTP listeners carrying the same label-based `allowedRoutes` as the HTTPS listeners of the other modes, so app HTTPRoutes attach by hostname unchanged. It mints no Issuer, no Certificate and no http-to-https redirect route, and skips the TLS-passthrough listeners an HTTP-only edge cannot serve, which leaves the tenant Kubernetes API, VM export and CDI upload endpoints unpublished through that Gateway. Those TLSRoutes each pinned a listener that no longer exists, so the controller flips the acceptance it wrote for them to `Accepted=False` with reason `NoMatchingParent`. Moving a tenant into the mode removes the ACME objects it owned, and the cluster-wide ACME solver drops its `sectionName: http` pin when the publishing Gateway itself sits on such a class. Pointing `gateway.className` at an edge class is therefore a supported topology (the whole platform behind one terminator), and its cost is that those three endpoints stop being published for the publishing tenant rather than for one tenant among many. Nothing refuses it, because refusing it would forbid the topology; `docs/gateway-classes.md` states the cost where an operator will meet it. Two consequences worth stating plainly. The wildcard-secret controller replicated the operator's TLS Secret, private key included, into every namespace owning a termination point, and a Gateway owner counted as one; a Gateway on an edge class does not, so that tenant's replica is pruned while its neighbours keep theirs. And listing a class as edge-terminated is a claim nothing validates: name the bundled `cilium` and its Gateways publish every app hostname over plain HTTP on a public address. A hard `fail` there would forbid a legitimate topology, a Cilium Gateway behind an external terminator, so it is stated in the platform values, the chart README and the CRD field description instead. **Documentation.** `docs/gateway-classes.md` is the operator-facing page for the whole surface: the three platform values and what each bounds, what a tenant writes to pick a class, how `certMode` is derived and in what precedence, what moving a tenant between classes does to its Issuer, its Certificates, the TLS Secrets those leave behind, the redirect route and the wildcard replica, and what each way of getting the names wrong actually looks like: a class name matching nothing installed renders fine and never programs, a misspelling in `edgeTerminatedClasses` silently leaves the tenant issuing certificates, a default moved out from under a tenant that pinned the old name fails that tenant's release. The chart README keeps the per-mode reference and gains the one constraint it had not stated: the publishing tenant cannot select a class that terminates TLS differently from the platform default, because the cluster-wide ACME issuers derive their solver pin from that default alone. Verified on a v1.6.0-rc.2 dev cluster (Talos, k8s 1.36, cilium 1.19.5) by building the controller from this branch: the CRD enum gained `edge`, the Gateway came up with `edge` / `edge-apex` / `edge-child-*` HTTP listeners and nothing else, no Issuer, no Certificate, no redirect route, and the LoadBalancer Service exposed 80/TCP only. That run also caught a defect no unit test could. The first implementation kept the hostname-less `:80` listener at index 0, and on cilium 1.19.5 `CheckGatewayAllowedForNamespace` is first-listener-wins ([cilium#42159](https://github.com/cilium/cilium/issues/42159), fixed in 1.19.6), so an inheriting tenant's HTTPRoute came back `NotAllowedByListeners` even though a later listener admitted it. Dropping that listener, which in edge mode carries neither ACME nor the redirect, flipped the route to `Accepted=True` and removed a catch-all at the same time. The live check predates the redesign to per-tenant classes, so it exercised the mode rather than the selection path; the selection is covered by unit tests across the four charts and both controllers. Moving a live Gateway from one class to another is covered nowhere. `Gateway.spec.gatewayClassName` has no immutability rule in the shipped CRD, so the apiserver accepts the update; what the outgoing class controller then does with the LoadBalancer Service it provisioned was not exercised by the live run and no unit test reaches it. ### Screenshots Not a UI change. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: `networking/gateway-api.md` documents the platform `gateway.*` rows and the cert modes, so `className`, `tenantSelectableClasses`, `edgeTerminatedClasses` and `tenant.spec.gatewayClass` need a doc PR there; `docs/gateway-classes.md` from this PR is what that port starts from. Not filed yet. - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: the provider hand-writes the Tenant schema, so `spec.gatewayClass` needs an attribute there; not filed yet. - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Two boxes stay unticked on purpose: the website and provider follow-ups are real and not filed yet. I will open them once the shape here settles in review. ### Release note ```release-note feat(gateway): tenant Gateways can now run on a GatewayClass other than the bundled Cilium one. The platform sets the default in `gateway.className` and bounds tenant choice with `gateway.tenantSelectableClasses`; a tenant selects its own with `tenant.spec.gatewayClass`. Classes whose provider terminates TLS upstream are listed in `gateway.edgeTerminatedClasses`, and a tenant Gateway on such a class serves plain HTTP listeners, issues no certificates, and no longer receives a replica of the operator wildcard Secret. Such a Gateway also has no TLS-passthrough listeners, so the `api`, `vm-exportproxy` and `cdi-uploadproxy` entries of `publishing.exposedServices` are not published through it, and their TLSRoutes report `Accepted=False` with reason `NoMatchingParent` ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added edge certificate mode with HTTP-only listeners, no certificate issuance, and no TLS passthrough. * Added configurable GatewayClass selection for tenant-owned Gateways, with platform defaults, allowlists, and validation. * Added platform configuration for edge-terminating and tenant-selectable GatewayClasses. * **Bug Fixes** * Improved wildcard certificate replication for edge-terminated Gateways. * Ensured safe cleanup and restoration when switching certificate modes. * **Documentation** * Updated Gateway, tenant, and certificate configuration guidance for edge termination and GatewayClass selection. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.