169 Commits
Author SHA1 Message Date
Aleksei Sviridkin ad3f15d4f3 feat(gateway)!: reserve a passthrough listener's name by the spec alone
The TenantGateway controller withheld the HTTPS-terminate listener for a
name a passthrough listener answers only once a TLSRoute was attached
there and resolved a backend, mirroring how Cilium builds its filter
chains. That kept a served endpoint up on Cilium 1.19.5, but it tied
every route's status to one implementation's translator and needed
Service and ReferenceGrant watches to land the deferred decision.

The shipped operator accepts a wildcard terminate listener beside a
narrower passthrough listener on 443 and keeps conflicting every other
overlap. A terminate listener for the passthrough name itself is not
narrower, so on 443 the rule now protects the dataplane outright; on a
native port Gateway API would serve the pair, and the rule holds as a
policy that keeps a platform name with its passthrough listener. Reading
it off the spec makes the outcome the same on every implementation.

Under http01 an HTTPRoute claiming such a name gets no listener and no
certificate whether or not a TLSRoute is attached. A route whose every
hostname is reserved reads Accepted=False with
NoMatchingListenerHostname; one that also claims a free hostname stays
Accepted and is served there, as Gateway API accepts a route whose
hostnames intersect at least one listener. dns01 and existingSecret are
unchanged: their wildcard listener still covers such a name, and the
passthrough listener answers it by its exact server name.

The chart writes the passthrough services on the publishing tenant
alone, so the names reserved by default are the ones the platform
publishes there through its own TLSRoutes. The shape an upgrade
changes is an HTTPRoute on such a name whose platform TLSRoute forwards
nowhere, because it never attached (the package owning the service is
not installed) or because none of its backends resolve: that route kept
a terminate listener and now gets none.

The backend resolution, its watches and the RBAC rules that served them
go away.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-10 06:32:42 +03:00
Aleksei Sviridkin bdca165f4c feat(gateway): report a GatewayClass that does not serve TLSRoute
A GatewayClass lists the features it serves in status.supportedFeatures,
and TLS passthrough is TLSRoute. On a class that leaves it out the
passthrough listeners are accepted and serve nothing, so the
TenantGateway now reads Ready=False/GatewayClassUnsupported naming the
class instead of Ready. A class that lists no features at all is not
judged: implementations predating the field leave it empty.

The class writes that list after it is accepted and may change it when
its controller is upgraded, so a GatewayClass event requeues the
TenantGateways on it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-09 17:53:25 +03:00
Aleksei Sviridkin d624049a3a feat(gateway): allow tlsPassthroughListeners under dns01 and existingSecret
The refusal existed because Cilium 1.19.5 served every Gateway port
from one Envoy listener matched by SNI alone, so the "*.<apex>"
terminate listener of the wildcard modes and a passthrough listener
under the apex could reach each other's backend. The shipped Cilium
1.20.2 gives each port its own Envoy listener when SNI overlaps across
ports, and Gateway API treats listeners on different ports as
compatible, so the pair is served as declared. edge still refuses the
field: it renders no TLS listener for an entry to bind to.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-09 17:53:25 +03:00
Aleksei Sviridkin 2af8b0c229 ci(build): fall back to direct, retry and cache Go module downloads (#4660)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

One transient error from proxy.golang.org is enough to fail an image
build or the `Unit & controller tests` job today. The error is an HTTP/2
stream reset (`stream error ... INTERNAL_ERROR`), and nothing retries or
falls back after it. This PR adds three things: a direct fallback, a
bounded retry and a module cache.

Image builds now fall back to fetching modules from their origin when
the proxy fails. They get `GOPROXY=https://proxy.golang.org|direct` as a
build arg from `hack/common-envs.mk`, and every Go builder stage
declares `ARG GOPROXY`. With a pipe Go goes direct on any proxy error.
With Go's default comma it does that only on a 404 or 410. The value
comes from `IMAGE_GOPROXY` and not from the host's `GOPROXY`, because
buildkit can't reach a mirror on localhost or a `file://` proxy. Every
workflow job that runs Go on the runner sets the same value as env.

Explicit module downloads in Dockerfiles are retried three times with a
growing pause. The loop is inline, in one form that the contract test
pins. I tried a shared script from a named build context first, but
buildx refuses any `--build-context` for the four Dockerfiles still
pinned to the `dockerfile:1.2` frontend. The images also build from
different contexts, so no single file can be copied in. Raising those
pins changes how those images build and needs a real build to check, so
I left it for a follow-up. Two alpine stages, token-proxy and
shadowbox's `ss-server`, left the download to `go build`. The alpine
golang images have no git, so the direct fallback can't fetch for them,
and they now get the same download step. The runner jobs use
`hack/go-mod-download.sh` with the same bound.

The unit test job now starts from a module cache instead of downloading
everything. It restores `GOMODCACHE` with `actions/cache`, keyed on
every `go.sum` in the tree, and then prefetches all tracked modules
through the retry, so a cache miss still works. A cache saved by a PR
run is visible only to that PR, but one saved on the default branch can
be restored by any PR
([docs](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache)).
That is why the build cache warmer on main writes the same key before
its image build. This way the first run of a PR, which is the one that
flakes, starts warm. The warmer restores only an exact key, so old
module versions don't pile up, and its Go steps can't fail the image
build. Both runner labels are Linux, so the key matches. Both also check
out under `/home/ubuntu/_work`, so `go env GOMODCACHE` should give the
same cache path, but I have not seen it on the warmer's runner yet. The
release jobs in `tags.yaml` and `promote-rc.yaml` get the GOPROXY env
only, no cache.

`hack/go-module-fetch-contract.bats` covers all three. Each test is red
on main, and I checked each guard with a mutation.

Closes #4591

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
ci(build): Go module downloads in image builds and CI jobs now fall back to direct fetching and retry on transient proxy.golang.org errors. The unit test job reuses a module cache written from main.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Reliability**
- Go module downloads now fall back to direct access when the public
proxy encounters errors.
- Builds and automated checks retry failed module downloads, helping
them recover from temporary network issues.
- **Performance**
- CI caches Go modules to reduce repeated downloads and speed up builds
and tests.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-05 14:14:47 +03:00
Aleksei Sviridkin 9f72f0bda9 feat(api): refuse a WorkloadMonitor with an empty selector
An empty selector matches every Pod, PVC, Service and DataVolume in the
namespace, so one such monitor goes non-operational on any import there.
No chart renders one, but the schema marked the field required without
requiring any label, and the API accepted it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 14:02:37 +03:00
Aleksei Sviridkin 99f3d3b2b7 feat(controller): tag a DataVolume cause with a condition reason
WorkloadsReady=False covers different situations: a stopped VM has
no pod and its monitor reports it not operational on purpose, while a
disk whose DataVolume never populates is broken. A consumer that wants
to react to one and not the other had only the message text to go by.

The WorkloadMonitor status gains a Reason next to Message, set to
DataVolumeNotReady by the DataVolume check, and the Application API
carries it into the WorkloadsReady condition in place of the generic
reason whenever a monitor that is not operational names one.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 14:02:33 +03:00
Aleksei Sviridkin 48ece79ffc feat(controller): name the DataVolume that keeps a disk not ready
For a vm-disk, WorkloadsReady is the one condition that reports
whether CDI populated the disk, and its generic "One or more workloads
are not operational" gives an operator nothing to act on: a disk has
no workloads, and the stuck DataVolume and its phase are exactly what
they need to know.

The WorkloadMonitor status gains a message that the controller fills
with each selected DataVolume that is not ready and its phase, and the
Application API uses the messages of the monitors that are not
operational in WorkloadsReady instead of the generic text whenever one
of them names a cause.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 14:02:32 +03:00
Aleksei Sviridkin 548d33f01f build(images): fall back to direct and retry Go module downloads
A transient proxy.golang.org HTTP/2 stream reset fails an image build
outright: Go's default GOPROXY falls back to direct only on a 404 or
410. Image builds now get GOPROXY=https://proxy.golang.org|direct as a
build arg from hack/common-envs.mk, declared by every Go builder stage,
so any proxy error falls back to fetching from the origin. The value
comes from IMAGE_GOPROXY rather than the host's GOPROXY, which may
name a mirror that buildkit cannot reach. The alpine golang images
carry no git, so in their stages a module hosted on GitHub cannot be
fetched directly and the retry is the cover.

Explicit `go mod download` steps retry three times with a growing
pause. The loop is inline, in one form the contract test pins: the
images build from different contexts, so no shared script can be
copied in, and a named build context in the shared buildx arguments
makes buildx refuse every Dockerfile pinned to a frontend older than
dockerfile:1.4.

The two alpine stages that left the download to `go build` get the
same download step, because without git neither the fallback nor a
retry covered them.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-01 19:37:00 +03:00
Mattia Eleuteri e510d49b3d feat(monitoring): re-render node agents when a monitoring label changes
The node fluent-bit builds its tenant routes from a lookup of namespace
labels, and helm-controller upgrades a release only when its chart or
values digest changes. A tenant turning monitoring on therefore got no
route, and a removed one kept its route, until an unrelated change
upgraded monitoring-agents.

The reconciler digests the label across all namespaces and requests a
forced upgrade of the release when the digest moves. It digests every
labelled namespace rather than repeating the chart's selection, so it
cannot drift from the chart; the cost is an occasional upgrade that
renders the same config.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-09-29 17:10:55 +02:00
Aleksei Sviridkin f301cb551b feat(api): add configurable resources and replicas for cozystack-engine components (#2956)
## What this PR does

Exposes standard Helm `resources` values for the three core workloads in
the `cozystack-engine` package, and makes `cozystack-controller`
replicas configurable (was hardcoded to `1`).

**Changes by component:**

| Component | `resources` | `replicas` |
|---|---|---|
| `cozystack-api` | ✅ NEW (`cozystackAPI.resources`) | already
configurable |
| `cozystack-controller` | ✅ NEW (`cozystackController.resources`) | ✅
NEW (`cozystackController.replicas`, default `1`) |
| `lineage-controller-webhook` | ✅ NEW
(`lineageControllerWebhook.resources`) | already configurable |

Each template uses `{{- with .Values.xxx.resources }}` so the
`resources:` block is only rendered when values are provided. Default
`resources: {}` produces no output, so resource requests/limits stay
opt-in.

**Leader election:** `cozystack-controller` now always runs with leader
election — the chart passes `--leader-elect` unconditionally and ships a
namespaced `Role`/`RoleBinding` for the Lease in `cozy-system` (where
controller-runtime creates it). Raising `cozystackController.replicas`
therefore adds standbys instead of a second active writer over the
reconcilers that patch HelmReleases, Gateways, Certificates,
ResourceQuotas and Secrets; this holds whatever `replicas` is. A restart
can pause reconciliation for up to the Lease duration (15s by default),
because `LeaderElectionReleaseOnCancel` is off. With `replicas: 1` the
change for an existing install is the flag plus the two RBAC objects.

**Why:** Without these values, operators cannot set QoS guarantees and
must resort to `kubectl patch`, Flux Kustomize SSA patches, or Kyverno
mutations — all fragile under Helm upgrades.

Closes #2955

### Release note

```release-note
feat(api): add configurable `resources` (requests/limits) for cozystack-api, cozystack-controller, and lineage-controller-webhook.
`cozystack-controller` replicas is now configurable via `cozystackController.replicas` (default: 1).
The `cozystack-controller` Deployment now always runs with leader election and creates a Lease in `cozy-system`, so raising `replicas` adds standbys rather than a second active controller; a restart can pause reconciliation for up to the Lease duration (15s by default).
An empty `resources: {}` renders no resources block, so requests and limits remain opt-in.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Container resource requests and limits can now be configured for the
API, controller, and webhook services; defaults remain unchanged.
* The controller replica count is configurable, and leader election is
enabled for multi-replica operation.
* Controller leader-election access is limited to the `cozy-system`
namespace.

* **Documentation**
  * Documented the webhook resource configuration option.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 20:56:43 +03:00
Nikita Aboltin cb8728dde1 docs: note the measured idle usage next to the empty resources defaults
The resources keys stay empty so requests and limits are opt-in, and the comment
records the measured idle usage next to them, so a limit that does get set is not
placed below what the component needs at idle.

Signed-off-by: Nikita Aboltin <aboltin.nikita@rwb.ru>
2026-09-28 16:43:44 +03:00
Nikita Aboltin 8732dc96cf feat(api): add configurable resources and replicas for cozystack-engine components
- cozystack-api, cozystack-controller, lineage-controller-webhook: conditional
  resources block via `{{- with .Values.*.resources }}`, defaulting to `{}` so the
  default render stays unchanged
- cozystack-controller: replicas comes from values (was hardcoded 1); the chart
  now passes --leader-elect unconditionally and grants the Lease verbs through a
  namespaced Role and RoleBinding in cozy-system (the Lease lives in the manager's
  own namespace), so raising replicas adds a standby instead of a second active
  writer over the reconcilers that patch HelmReleases, Gateways, Certificates,
  ResourceQuotas and Secrets

Signed-off-by: Nikita Aboltin <aboltin.nikita@rwb.ru>
2026-09-28 15:19:45 +03:00
Aleksei Sviridkin d9cbbec5ff fix(build): cross-compile image builders on the build platform
The remaining Go builder stages, plus the kube-ovn, console and
piraeus-server builders, ran on the target platform. A multi-platform
build therefore ran the whole compiler under QEMU: slow, and for Go
unreliable, as the migration-controller Dockerfile already records a
"found bad pointer in Go heap" crash from exactly that. The builders
now run on $BUILDPLATFORM and produce the target architecture from
TARGETARCH.

Two builders only produced the right architecture because they were
emulated. token-proxy ran `go build` with no GOOS/GOARCH, and kube-ovn
calls upstream `make build-go`, which hardcodes GOARCH=amd64 even on an
arm64 build. On a native builder both would have shipped host binaries
inside the other architecture's image, and nothing fails until a node
of that architecture starts the container. token-proxy now passes the
target explicitly; kube-ovn picks build-go or its arm64 twin
build-go-arm from TARGETARCH.

The console builder emits static JS and linstor-server's debian/control
declares every package Architecture: all, so both run natively without
a target.

A bats test requires every stage that compiles to sit on a BUILDPLATFORM
FROM and to read TARGETARCH or TARGETPLATFORM on a RUN or ENV line. A
bare ARG declaration does not count, which is what let token-proxy
through.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-26 15:05:06 +03:00
Aleksei Sviridkin ec7387109e build(go): move the toolchain and builder images to Go 1.27.1
Go 1.26.4 carries standard-library advisories in net/http, net/url,
crypto/tls, encoding/asn1 and html/template that govulncheck reports
as reachable from the first-party binaries. The shipped stdlib comes
from the builder image, so the builder tags move together with the
go directive; the digests are the multi-arch indexes that golang:1.27,
1.27-alpine and 1.27-bookworm resolve to at 1.27.1.

api/apps/v1alpha1 keeps go 1.26.4. It is a published types-only
module, no binary here links its stdlib, and raising its directive
would force every consumer onto Go 1.27.

linstor-csi v1.11.2 pins grpc v1.81.0, which no longer builds on
Go 1.27: golang.org/x/net drops http2.TrailerPrefix there. Its image
now raises grpc to v1.83.2 before building, as the kamaji image does
for its advisories; the upstream unit tests pass on it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 17:23:18 +03:00
renovate[bot] a7dfbe6f8c chore(deps): update container image & action digests
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:04 +03:00
Aleksei Sviridkin 9121456ba8 chore(cozystack-controller): name the Service and ReferenceGrant reads
Deciding whether a TLSRoute forwards anywhere reads the backend Service
and the ReferenceGrant admitting a cross-namespace reference to it, and
both are watched rather than fetched on demand. The read-everything
rule at the foot of the ClusterRole covers them today, so nothing is
broken. Naming them keeps the feature working the day that rule is
narrowed, where an informer starting without a grant fails with
Forbidden and takes the manager with it.

The kubebuilder markers in the package generate nothing: controller-gen
runs over ./api/... alone and writes no rbac artifact, so the shipped
role is hand-written and only a test connects the two.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-10 18:47:30 +03:00
Aleksei Sviridkin 50e1e2b2ed feat(gateway): add tlsPassthroughListeners to the TenantGateway API
A tlsPassthroughServices entry publishes a hostname on port 443, which
suits a service whose clients speak HTTP and mis-serves one whose
clients expect a database on its own port. The new field takes a name,
a native port and the SNI hostname to answer on it, so an engine can be
reached where its drivers already look.

The cross-field rules are enforced at admission as well as in the
controller. renderGateway is the first thing that renders, so a spec
the apiserver accepts and the renderer refuses aborts everything behind
it, and one mistyped hostname would freeze the Gateway and every route
status where they stand. The controller keeps its own copy because it
and the CRD roll out separately. Rules with no admission form, and why
each one has none, are named on the type.

tlsPassthroughServices and apex gain bounds at the same time, for the
CEL cost estimator rather than for their own sake: the new rules
concatenate apex and scan the services list, and an unbounded string or
list makes the install-time estimate exceed the per-CRD budget and the
apiserver refuse the whole definition. Every bound is a kind the
apiserver ratchets, so a stored value that violates one keeps accepting
writes that leave it alone.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-10 14:21:05 +03:00
Aleksei Sviridkin d685fb30ec fix(gateway)!: require tenant routes to declare their hostnames (#3475)
## What this PR does

The tenant route hostname policy skipped the shape it most needed to
check. Its CEL opened with `!has(object.spec.hostnames) ||`, so an
HTTPRoute or TLSRoute declaring no hostnames satisfied the first operand
and got admitted with nothing compared against the tenant apex. An empty
list passed too: `has()` is true on `[]`, and `all()` over an empty list
is vacuously true.

Such a route is not inert. Under Gateway API it inherits the hostnames
of every listener it attaches to, and `parentRefs` can name a Gateway in
another namespace, so it covers whatever that listener covers. Reported
in #3467.

The predicate now requires the field to be present and non-empty before
validating it. Both policies rendered from that template share one CEL
string, so HTTPRoute and TLSRoute get the change together, and both
match `CREATE` and `UPDATE`, so an existing route that drops its
hostnames is denied exactly as a new one is.

The issue is titled after TLSRoute, but the reachable shape is mostly
HTTPRoute. TLSRoute allows a hostname-less route only on the deprecated
`v1alpha2`; at `v1` and `v1alpha3` the bundle this repo vendors marks
`spec.hostnames` required and rejects an empty list on `minItems: 1`,
while `v1alpha2` carries neither constraint. HTTPRoute `v1` carries
neither either, so it accepts both shapes. All four are checkable in
`packages/system/gateway-api-crds/templates/crds-experimental.yaml`.

Checking that turned up a stale claim in the template header, which said
TLSRoute had no `v1` in Gateway API 1.4 to 1.5 and used it to justify
the rule naming only `v1alpha2`. Gateway API 1.5.1, which this repo
vendors, serves TLSRoute at `v1` as the storage version. The rule still
covers every served version because `matchConstraints.matchPolicy`
defaults to `Equivalent`, and that default is now written down, because
setting `matchPolicy: Exact` reads like a tightening and would silently
stop the policy seeing `v1` and `v1alpha3` TLSRoutes. I corrected the
text and left the coverage alone: the wrong sentence predates this
branch, but my change is what makes the file contradict itself, so the
text is mine to fix while widening version coverage is a separate job.

A `ValidatingAdmissionPolicy` has no cross-object lookup, so it cannot
resolve the parent Gateway to see which hostnames a route would inherit.
Requiring the route to name its own is what keeps every admitted
hostname comparable against the apex. That has a cost: a route relying
on listener inheritance now has to spell its hostnames out, and a tenant
that hand-writes one gets a denial where it used to get an admission.

### The controller change belongs in the same PR

`renderHTTPRedirect` built the http-to-https redirect HTTPRoute with no
hostnames on purpose, to catch every host on port 80. It lands in a
`tenant-*` namespace, so the stricter policy denies it,
`reconcileHTTPToHTTPSRedirect` returns that error, and since it runs
before the status update the TenantGateway never reaches Ready. That
redirect is what keeps app routes attaching by hostname from serving
plaintext on port 80. Neither half of this change is safe alone.

The redirect now names the tenant apex and its wildcard. Both entries
are needed: a wildcard hostname is a suffix match over any number of
labels, so `*.<apex>` covers subdomains at every depth but not the bare
`<apex>`. `renderWildcardCertificate` already builds the same pair.

Coverage narrows in one direction. A request on port 80 with a Host
outside the tenant apex now falls through to a 404 instead of a 301.
That narrowing costs no covered host: the port-80 listener admits routes
only from the Gateway's own namespace and cert-manager's, and the
tightened policy requires every route in the tenant namespace to declare
its hostnames inside the apex, so any host published from this namespace
already falls under the apex or its wildcard.

An inheriting child tenant does not widen that set either. Its hostnames
would reach port 443 through a listener on this same Gateway, and
`cozystack-gateway-hostname-policy` compares every listener against the
host label of the Gateway's own namespace, so a child apex outside that
label never gets its listener: the Gateway write carrying it is refused
at admission whole, in `reconcileGateway` and before the redirect is
rendered. A child apex under this one passes that check and the wildcard
already covers it. That holds at merge base as well, so it is not
something this branch changes.

Deriving hostnames from `spec.apex` makes that field load-bearing for
admission, so it gains `MinLength=1`. An empty apex used to reconcile to
Ready, since the redirect declared no hostnames at all; it would now
render `""` and `"*."` and be rejected on the HTTPRoute hostname schema,
surfacing as a validation error naming the route rather than the field.
The renderer checks it too, because the CRD and the controller binary
roll out separately and an older CRD in the cluster will not carry the
bound.

That schema bound is a breaking change to an existing field and the API
review gate flags it as one, so here is what it actually invalidates.
The chart that renders these CRs already fails on an empty
`_namespace.host`, and in DNS-01 or existingSecret mode an empty apex
already broke earlier, in `renderGateway`, on a listener with `hostname:
"*."`. Nothing that works today becomes invalid. What the bound rejects
is a TenantGateway written by hand, past the chart, with an empty apex,
and I would rather that be refused at admission naming the field than at
reconcile time in controller logs, which is why the bound sits in the
CRD as well as the renderer.

Naming those hostnames also meant excluding the redirect from
`collectHostnameClaims`. A route that attaches to the Gateway and
declares hostnames otherwise reads as an app publishing them, and each
claimed hostname becomes an HTTPS listener named after its first label.
The first label of `*.<apex>` is `*`, which the Gateway CRD's
listener-name pattern rejects, so the Gateway write fails and the
reconcile stops before anything downstream. The exclusion matches on
namespace, name and controller ownership rather than on the labels the
renderer stamps: those labels are writable by anyone who can create a
route, and the claim set also drives conflict resolution, so a
label-keyed exception would let any route opt out of being marked
`Accepted=False` when it loses a hostname race. Name and ownership are
each required and each is pinned by its own test, since ownership alone
would exclude any other controller-owned route and name alone would
extend the exception to a route planted under that name by someone else.

### Radius

Only namespaces starting with `tenant-` are matched. The three
first-party TLSRoutes the platform ships, `kubernetes-api` in `default`
plus `cdi-uploadproxy` and `vm-exportproxy`, set hostnames explicitly
and none sits in a `tenant-*` namespace, so nothing changes for them.

I went through every producer of routes in the tree, in three places
rather than one. No Helm template emits an absent or empty hostname list
in any configuration a tenant can reach: the vendored openbao, harbor,
opencost and flux-operator route templates can render one, but each is
behind a flag that defaults off with no values path to turn it on, and
openbao would fail CRD validation first because its default route
apiVersion requires hostnames. Exactly one Go site builds a route, the
redirect fixed here. And one e2e fixture did rely on the old behaviour:
the Gateway suite's parentRef probe declared no hostnames and is applied
into `tenant-test`, which the policy matches, so it now reads the apex
from the namespace label and names a hostname under it, the same way the
neighbouring cases in that suite already did.

cert-manager also writes routes into tenant namespaces, since it creates
the Challenge in the Certificate's own namespace and the HTTP-01 solver
route beside it. That route declares the hostname it is validating, so
the policy admits it.

### Related issues

Two open issues have a text half that this PR happens to settle, and
neither of them closes here. #3788 reports that the port-80 allow-list
contains the namespace its own comment says it excludes; the comment and
the README paragraph now describe what that filter actually does, while
the behavioural exit it names stays open. #3789 reports that the
TLSRoute rule names only the deprecated API version; that premise is now
written down in the template header, including what has to change before
a bundle drops `v1alpha2`, but the rule still names `v1alpha2` alone.
#3787 covers the wildcard-hostname listener name that the apiserver
rejects, which is what makes the claim-set exclusion here load-bearing
rather than tidy.

### Screenshots

N/A (no UI changes).

### Downstream repositories

I walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file. No package is added, renamed or removed; no
`values.yaml` key, `values.schema.json` field, version enum or default
moves; no `kind` or `plural` changes; no namespace or release-asset name
changes; nothing under `hack/` moves, is renamed, or changes what a make
target does.

One row deserves a sentence rather than a blanket denial, because the
diff does change a CRD schema: `spec.apex` on
`gateway.cozystack.io/v1alpha1 TenantGateway` gains a `minLength` bound,
which lands in the generated `gateway.cozystack.io_tenantgateways.yaml`.
The provider hand-types `Package` under `cozystack.io` and
`Plan`/`RestoreJob` under `backups.cozystack.io`; `gateway.cozystack.io`
is not among them, and a tightening on an existing field adds no new
field for it to offer typed access to. The
`packages/extra/gateway/README.md` edit sits in the `## Security model`
section, below the `## Parameters` block that `cozyvalues-gen`
regenerates, so it survives `make generate`.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(gateway)!: the tenant route hostname policy now requires an HTTPRoute or TLSRoute in a `tenant-*` namespace to declare a non-empty `spec.hostnames`. A route that omitted the field, or set it to an empty list, was previously admitted without any hostname being checked against the tenant apex, even though such a route inherits the hostnames of the listener it attaches to rather than matching nothing. A route in a `tenant-*` namespace relying on that inheritance is now rejected and has to name its hostnames. The controller-owned http-to-https redirect route names the tenant apex and its wildcard accordingly, so a request reaching port 80 with a Host outside the tenant apex now gets a 404 instead of a 301. `TenantGateway.spec.apex` additionally gains a `minLength: 1` bound, because the redirect derives its hostnames from that field; a TenantGateway written past the chart with an empty apex is now rejected at admission instead of failing later in the controller.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added edge-terminated gateway support with plain-HTTP listeners and
TLS passthrough handling.
- HTTP-to-HTTPS redirects now explicitly cover the tenant apex and
subdomains.
  - Gateway configurations require a non-empty apex hostname.
  - HTTPRoute, TLSRoute, and GRPCRoute resources must declare hostnames.

- **Bug Fixes**
  - Invalid hostname-less routes are now rejected consistently.
- Controller-managed redirects no longer trigger unnecessary hostname
claims, listeners, or certificates.
- Switching certificate modes now cleans up obsolete ACME resources and
route status.

- **Documentation**
- Updated security and gateway guidance for hostname validation, edge
termination, route compatibility, and certificate handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-07 18:49:04 +04:00
IvanHunters b6f14373b5 fix(marketplace): scope controller OCIRepository writes and split validate YAML safely
Close the two non-blocking notes from the latest review, mirroring fixes already
applied on the other side:
- The tap materializer's update/patch on ocirepositories moves from the
  cluster-wide cozystack-controller ClusterRole to a namespaced Role in
  cozy-system (get/list/watch stay cluster-wide for the informer), matching the
  API-side scoping. The materializer only ever writes tap sources in cozy-system.
- cozypkg validate now splits multi-document YAML with the apimachinery reader
  instead of a naive ^---$ regex, so a PackageSource whose block scalar contains
  a separator-looking line is not fractured — the gate's validator now decodes
  the same content the materializer applies. Covered by a block-scalar test.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-07 10:58:38 +03:00
IvanHunters 1fbceee10e fix(marketplace): operator RBAC for tap materialization and dot-safe artifact naming
Second review round found the write-path RBAC fix was only half applied and the
extracted naming helper diverged from its producer:

- The tap materializer runs as the cozystack-controller ServiceAccount, whose
  ClusterRole granted source.toolkit.fluxcd.io/ocirepositories only
  get/list/watch. Its finalizer and materialized-revision writes (r.Update on
  the OCIRepository) were therefore Forbidden, so a dashboard-connected tap
  never materialized. Grant the controller update/patch on ocirepositories.
- naming.ArtifactName replaced dots only in the PackageSource name, but the
  reconciler replaces them in the variant and component too, so a tapped repo
  with a dotted variant or component name dropped its packages from the catalog
  and tripped a false appdef-dangling warning. Replace dots in all three
  segments and add a direct parity test with dotted names.
- Pin cosign-installer and the cosign release in the index gate so the
  signature verifier is reproducible.
- Add a unit test for the tap rollback objectExists predicate.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-07 10:58:36 +03:00
Aleksei Sviridkin b9ecb2e39f fix(tenantgateway): name the apex on the http-to-https redirect route
The controller-owned redirect route carried no spec.hostnames, relying
on the port-80 listener to match every host. Name the tenant apex and
its wildcard instead.

A route in a tenant namespace has to declare hostnames inside its own
apex to pass the route-hostname admission policy, and a route that
names none has nothing to compare against that apex. Naming them also
states the coverage the route wants rather than inheriting whatever its
listener happens to carry.

Both hostnames are required: a wildcard is a suffix match over any
number of labels, so "*.<apex>" covers every subdomain at every depth
but not the bare "<apex>".

Coverage narrows in one direction worth stating: a request arriving on
port 80 with a Host outside the tenant apex now falls through to a 404
instead of a 301.

Skip this route when collecting hostname claims. Before it carried
hostnames it contributed nothing to that set, and letting its hostnames
through instead provisions listeners and certificates for them, which
fails. The check matches on name and controller ownership, not on the
labels the renderer stamps, since those are writable by anyone who can
create a route.

Deriving hostnames from spec.apex makes that field load-bearing for
admission, so it gains MinLength=1 and a matching check in the renderer
for clusters whose CRD is older than the binary.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 18:08:02 +03:00
Aleksei Sviridkin 2ef0e8d3a1 feat(gateway): add an edge cert mode for upstream-terminated classes
Some GatewayClasses have their provider terminate TLS in front of the
Gateway and reach it over plain HTTP. On such a class the platform
issued origin certificates nobody serves, and where ACME could not
complete, the Gateway's listeners never became ready.

Add certMode=edge to TenantGateway. The controller renders the apex,
its wildcard and every inheriting child apex as plain HTTP listeners
carrying the same label-based allowedRoutes as the HTTPS listeners of
the other modes, so application HTTPRoutes attach by hostname
unchanged. It mints no Issuer and no Certificate, and removes the
http-to-https redirect route, because the edge performs that redirect
itself. TLSPassthroughServices are not rendered: an HTTP-only edge
cannot carry a TLS listener, which leaves the endpoints relying on one
unpublished through such a Gateway.

The hostname-less port-80 listener is left out. It carries the ACME
challenge and the redirect in the other modes and has neither here, so
it would only be a catch-all admitting any hostname. Leaving it out
also keeps a hostname-pinned listener at index 0, which is what
Cilium's first-listener-wins namespace check reads before the fix in
cilium/cilium#45693.

Moving into the mode garbage-collects the Issuer and Certificate
objects the controller owned. The TLS Secrets they issued stay behind:
cert-manager runs with enableCertificateOwnerRef false, so a
Certificate does not own the Secret it issued into. Every mode
transition already behaves that way.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-04 14:26:54 +03:00
Aleksei Sviridkin 3d076099a2 docs(postgres): name the trust anchor as the tenant actually sees it
The README told a tenant to fetch <release>.tenant-ca, but the release
name carries the postgres- prefix the application definition applies, so
the object is postgres-<name>.tenant-ca. Following the old text returned
NotFound.

Also drop two comments promising a companion ValidatingAdmissionPolicy
that pins sentinel writes to helm-controller. No such policy ships: the
proposal for it was rejected because authorization by username duplicates
RBAC in a more brittle form. The RBAC boundary is the whole control.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-08-17 03:10:24 +03:00
Aleksei Sviridkin 62204d4903 test(cozystack-controller): guard the Secret grant by effective permission
The suite's "does not grant a blanket wildcard verb on secrets" assert
was a notContains over a rule object. helm-unittest compares every field
it is handed by value equality, and `any: true` changes which fields are
compared rather than how, so no spelling of that assert bounds what a
verbs list may hold: `verbs: ["get", "*"]` satisfies all of them.

Measured against the rendered ClusterRole, seven realistic spellings of
a widened Secret grant left that assert green: a wildcard in resources,
a wildcard in apiGroups, a "*" appended to the declared verb list, an
extra field beside the verbs, deletecollection spelled out, and two
where the grant is added as a separate rule. Those last two leave the
whole suite green - appending apiGroups:['*'] resources:['*']
verbs:['*'] keeps the positive pin satisfied while the controller gains
every verb on every Secret cluster-wide.

Replace it with a bats guard over the rendered manifests. It unions the
verbs that every ClusterRole and Role rule confers on core Secrets,
expanding "*" on each of the three axes, and compares that union with
the declared set. Equality rather than an upper bound: the controller is
cluster-scoped, so a widening reaches every tenant namespace, and a
narrowing breaks the two reconcilers that share the grant.

The positive pin stays in tests/rbac_test.yaml. Value equality is the
right primitive for pinning a rule as written and the wrong one for
bounding what the rule may become.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-08-12 18:37:09 +03:00
Timofei Larkin 95e4faf68e feat(workloadmonitor): bucket metrics endpoint override
The controller discovers where to query SeaweedFS bucket size metrics
via the namespace.cozystack.io/monitoring label, which assumes the
monitoring stack that scrapes SeaweedFS runs in the same cluster. Some
deployments host SeaweedFS and its metrics in a separate cluster, so
add a --seaweedfs-metrics-endpoint flag to cozystack-controller (and a
matching cozystackController.seaweedfsMetricsEndpoint chart value): an
explicit base URL of a Prometheus-compatible query API, with scheme,
host, optional port and path prefix; /api/v1/query is appended. Basic
auth may be embedded as URL userinfo. The flag is validated and
normalized at startup, so a malformed URL fails the rollout rather
than the queries. Empty keeps label-based discovery unchanged.

Bucket sizes feed billing, so metrics failures no longer collapse to
"no size": previously any query failure yielded an empty result map
and the reconciler overwrote Workload resources with it, silently
wiping the recorded sizes. The reconciler now retains the last known
s3-storage-bytes and s3-physical-storage-bytes values, emits a
BucketMetricsUnavailable warning event on the WorkloadMonitor, and
returns the error from Reconcile so it lands in the reconcile error
metrics.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-23 15:35:00 +03:00
Aleksei Sviridkin 6ecb36ffc7 feat(cozystack-controller): extract application CA into a key-free tenant Secret (#3407)
## What this PR does

Adds an engine-agnostic controller that publishes a key-free
`<release>.tenant-ca` Secret (only `ca.crt`) for a managed application,
so a tenant can verify the application's TLS without ever seeing a
private key. The source is declared, not guessed: a chart renders a
namespaced `TenantProjection` sentinel (group `internal.cozystack.io`)
naming the CA Secret to lift; the controller watches the sentinel,
extracts `ca.crt`, and writes the projection owner-referenced to the
sentinel, so garbage collection is native. The projection carries the
`internal.cozystack.io/tenant-ca` label the lineage webhook turns into
tenant visibility. Postgres is the first consumer.

The trust boundary is RBAC: no tenant role grants any verb on
`internal.cozystack.io`, so a tenant cannot forge a sentinel. A
companion `ValidatingAdmissionPolicy` that pins the writer to
helm-controller ships separately as defence in depth.

This is the controller-and-CRD slice, split out from #3299 for focused
review. It is the only piece required for the feature to work; the
render-time helper, the writer policy, the capability-gate, and the
release-prefix validation land as their own PRs.

### Downstream repositories

Walked the trigger map against the diff. This adds an internal CRD and
controller and a sentinel to the Postgres chart; no app is added or
renamed and no `values.schema.json` changes, so the typed provider and
Ansible are unaffected. The one to confirm is the Postgres README edit —
if the website mirrors it, it needs a refresh.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:

### Release note

```release-note
feat(cozystack-controller): publish a key-free per-application CA trust anchor (`<release>.tenant-ca`) that tenants read to verify managed-application TLS, declared through a new `TenantProjection` CRD. Postgres is the first consumer.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added tenant-facing CA certificate projections for PostgreSQL
deployments.
* Tenant secrets now provide only the CA certificate, excluding private
keys.
  * Added support for configuring and monitoring tenant CA projections.

* **Documentation**
* Updated PostgreSQL TLS instructions to retrieve certificates through
tenant secrets.

* **Bug Fixes**
* Improved protection against invalid certificates, secret collisions,
and accidental exposure of private key material.
* Added validation to ensure projected certificates remain synchronized
with their sources.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-23 14:14:28 +03:00
cozystack-ci[bot] 235d38dae1 Prepare release v1.6.0 (promoted from v1.6.0-rc.4)
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-07-22 13:00:10 +00:00
Aleksei Sviridkin 37a91f98b8 feat(cozystack-controller): extract application CA into a key-free tenant Secret
Every catalog engine issues a per-release CA, but the object holding
ca.crt usually holds a private key beside it (cert-manager's tls.key,
CloudNativePG's ca.key), so granting a tenant read on it hands over key
material too. Add a controller that publishes the trust anchor as one
canonical, key-free Secret per release, "<release>.tenant-ca", holding
only ca.crt.

A chart declares the anchor by rendering a namespaced TenantProjection
sentinel (internal.cozystack.io/v1alpha1) that names the source Secret
and key. The controller rebuilds ca.crt from the parsed certificate DER
rather than copying the source, refuses any value carrying private key
material, and owns the projection by owner reference so a foreign Secret
at the same name is never overwritten. No tenant RBAC grants any verb on
internal.cozystack.io; a ValidatingAdmissionPolicy pins writes to
helm-controller as defence in depth.

The controller does not grant tenant read access — the lineage webhook
does, from the ApplicationDefinition's spec.secrets. This side stamps the
engine-agnostic internal.cozystack.io/tenant-ca label and forces a
re-admission when those selectors change.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-21 23:59:48 +03:00
cozystack-ci[bot] d95685c427 Prepare release v1.6.0-rc.4
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-07-21 10:31:23 +00:00
Andrei Kvapil fbbe4af7df refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass (#3218)
## What

Removes the `network.cozystack.io` API group — `ExposureClass` +
`ServiceExposure` (#3081) — and its reconciler, in favor of native
`Service` `type: LoadBalancer` + `loadBalancerClass`.

## Why

Full rationale and discussion in #3164. For **managed applications** the
chart already owns the Service, so:

- `type: LoadBalancer` + `loadBalancerClass` + an admin-provisioned pool
covers vendor-neutral external exposure without a dedicated API group
and controller;
- the `externalIPs` (node-IP) default never went through
`ServiceExposure` (its backend is a no-op) and is left unchanged;
- the only current consumer, the host ingress, keeps rendering `type:
LoadBalancer` and just drops the `ServiceExposure`; the admin provisions
the address pool / announcer out of band, as the install docs already
assume.

`ServiceExposure`'s one unique feature — dynamic per-namespace scoping
of a shared VIP pool — is a convenience the managed-app path does not
need, and it does not justify a second tenant-networking API group next
to `sdn.cozystack.io`.

## Changes

- Delete `api/network/v1alpha1` and
`internal/controller/serviceexposure` (+ CRD definitions, e2e bats,
controller registration, RBAC).
- Host ingress: drop the `ServiceExposure` template; add an optional
`publishing.loadBalancerClass` that sets `spec.loadBalancerClass` on the
host-ingress LoadBalancer Service.
- The `externalIPs` path and default behavior are unchanged.

## Verification

- `go build ./...` — green.
- `helm unittest` — `core/platform` 72/72, `extra/ingress` 25/25 (incl.
host-vs-tenant `loadBalancerClass` scoping and the immutable-field
migration guard).

## Migration

None required. `network.cozystack.io` (#3081) has never shipped in a
release — it is on `main` only (not backported to `release-1.5`; absent
from v1.4.x / v1.5.x), so no upgrading cluster has these CRDs and this
removes it before it is ever released.

Clusters that track `main` and explicitly set `publishing.exposureClass`
(opt-in, only since 2026-06-30) should provision their MetalLB / Cilium
address pool themselves and set `publishing.loadBalancerClass`; any
orphaned `cozystack-<class>` pool can be deleted by hand. Default
clusters (externalIPs path) are unaffected.

## Status

**Draft / RFC for the #3164 discussion** — this removes an API group
that landed only days ago, so nothing here should merge before the
design call. cc @lexfrei (author of #3081) and @lllamnyp (#3164).

Deferred: when Kubernetes actually removes `Service.spec.externalIPs`,
the node-IP default migrates via a MetalLB `IPAddressPool` with no
`L2Advertisement` (allocate-but-don't-announce) — a separate change.



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for configuring the ingress host Service with a native
`loadBalancerClass` value.
* Expanded chart tests to verify class wiring and immutability behavior
across install and upgrade scenarios.

* **Bug Fixes**
* Removed the legacy exposure-class/service-exposure flow and related
controller behavior.
* Updated templates and CRD packaging so the platform no longer renders
or depends on those deprecated resources.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-16 17:44:28 +02:00
Aleksei Sviridkin 249f7b5a5e feat(platform, ingress): propagate operator wildcard certificate to per-tenant termination points (#2990)
## What this PR does

The operator-provided wildcard certificate
(`publishing.certificates.wildcardSecretName`) currently serves the root
tenant only: everything is same-namespace, so the root ingress
controller and Gateway read the operator's TLS Secret directly. Child
tenants run their own ingress controller / Gateway in their own
namespace under a default-deny policy and cannot read that Secret across
namespaces, so they fall back to per-host ACME.

This propagates the operator wildcard to per-tenant termination points
by replicating the Secret into each tenant namespace that owns one,
keeping the root-MVP invariant intact: the wildcard is deliberately
distributed to tenant namespaces, but no tenant RBAC is widened — every
consumer reads only its own-namespace copy.

A new controller in `cozystack-controller` does the replication with no
extra operator input. It reads the same platform values channel the
consumers read (`cozy-system/cozystack-values`): the wildcard Secret
name from `_cluster.wildcard-secret-name` and the publishing namespace
from `_cluster.expose-ingress`. It mirrors that Secret into every tenant
namespace whose `namespace.cozystack.io/ingress` or
`namespace.cozystack.io/gateway` label equals its own name — i.e. the
namespace runs its own controller / Gateway. Because the source is
derived from the same value that makes the consumers reference it, the
replica is created whenever the consumers expect it — no manual
labelling and no upgrade hazard where a child controller references a
Secret that nothing creates.

Replicas are refreshed on rotation and pruned when a namespace stops
terminating TLS. Teardown of every replica happens on exactly one
trigger: clearing `wildcardSecretName` (explicit disable). A source that
is merely absent or mistyped, or a transiently-missing values channel,
leaves existing replicas in place, so a brief gap, a delete+recreate
rotation, or a misconfigured publishing namespace never drops tenant
TLS. The platform controller's own ServiceAccount gains Secret writes;
tenant RBAC is untouched.

The controller does not cache every Secret in the cluster: the manager's
Secret informer is scoped to managed replicas and the values channel
only, and the operator source / foreign collisions are read through the
uncached APIReader. The source is not watched (its name is dynamic), so
an in-place rotation is picked up by a bounded periodic resync while the
publishing tenant serves the source with no lag.

On the consumer side, each tenant ingress controller now sets
`--default-ssl-certificate` to the wildcard Secret in its own namespace
(`.Release.Namespace/<name>`), so a child tenant serves the replica
instead of minting per-host ACME. The Gateway path already renders
`certMode: existingSecret` with a same-namespace `wildcardSecretRef` for
every tenant, so it consumes the replica with no chart change.

### Why not clustersecret-operator

The issue proposed the shipped `clustersecret-operator`. It materializes
inline template data into selected namespaces; it cannot mirror an
existing source Secret, and it is not installed in any default bundle.
Driving it would mean embedding the certificate and private key as
literal data in a cluster-scoped object rendered from a Helm `lookup` —
that broadcasts key material the design keeps off any shared channel,
and a `lookup` is invisible to the Flux helm-controller digest so it
would not re-render on rotation. Its namespace selector also cannot
express "the namespace whose owner label equals its own name", so it
would over-replicate the key into every tenant namespace. A small
dedicated reconciler targets exactly the termination namespaces and
keeps the key off the values channel.

### Coverage caveat

Replication delivers the certificate bytes, not SAN coverage: a single
`*.<root-host>` does not match `*.<child-apex>`. Operators serving
child-tenant subdomains supply a certificate whose SANs cover those
apexes; the docs spell this out. The replica carries the wildcard
private key into each terminating tenant namespace — the same exposure
as the per-host ACME Secret it replaces, now shared across tenants.

### Tests

- Go fake-client tests for the reconciler: termination-only replication
(ingress-owning and gateway-owning namespaces only, never inheriting
ones, never over the source), copy markers, rotation, prune on namespace
demotion, the non-destructive source-deletion contract, replica
self-heal after out-of-band deletion, the publishing-namespace
self-replication guard, disable-via-config teardown, source rename,
foreign-Secret collision safety (no overwrite, no requeue, does not
block siblings), transient-error requeue, non-TLS source ignored,
absent-channel keeps replicas, publishing-namespace mismatch keeps and
requeues, the cache-scope pin, the enqueue-wiring map, an
`ownsTerminationPoint` table, and the active-only requeue contract.
- helm-unittest for the ingress `default-ssl-certificate` wiring
(including a child-tenant controller pointing at its own-namespace
replica) and for the cozystack-controller ClusterRole's new Secret-write
rule.

### Screenshots

N/A — no UI changes.

### Release note

```release-note
feat(platform): the operator-provided wildcard TLS certificate (`publishing.certificates.wildcardSecretName`) is now served by per-tenant ingress controllers and Gateways automatically — the platform controller replicates it into each tenant namespace that terminates TLS, with no extra operator input and no cross-namespace Secret read
```

Closes #2820

Part of #2811


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **New Features**
* Wildcard TLS Secret now replicates from the platform to tenant
namespaces that terminate TLS via their own ingress controller or
Gateway, including child-tenant scenarios.

* **Documentation**
* Clarified configuration, timing, and SAN matching requirements for
wildcard replication.
* Updated gateway and ingress-nginx default certificate documentation
for replica mirroring behavior.

* **Bug Fixes**
* Improved handling for missing/invalid source secrets (no unnecessary
pruning) and safe collision behavior.

* **Tests**
  * Added comprehensive reconciliation/replication test coverage.
  * Added Helm unittest target for the controller chart.

* **Chores**
  * Updated controller RBAC to support wildcard TLS Secret replication.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-15 11:32:08 +03:00
renovate[bot] 5b9b420ffe chore(deps): update container image & action digests 2026-07-10 05:29:23 +00:00
Andrei Kvapil 3dbc7d3662 refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass
Remove the network.cozystack.io API group (ExposureClass + ServiceExposure)
and its reconciler. Managed-application charts own their Service, so native
type: LoadBalancer + loadBalancerClass + an admin-provisioned pool covers
external exposure without a dedicated API group and controller. The host
ingress -- the only consumer -- keeps rendering type: LoadBalancer and gains
an optional publishing.loadBalancerClass to pick the LB controller; the
externalIPs (node-IP) default path is unchanged.

Rationale in cozystack#3164.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-07 12:39:30 +03:00
Aleksei Sviridkin c841c70866 feat(operator): replicate operator wildcard certificate to tenant namespaces
The operator-provided wildcard MVP serves the root tenant only: the root
ingress controller and Gateway read a TLS Secret from their own namespace.
Child tenants run their own ingress controller / Gateway in a separate,
default-deny namespace and cannot read that Secret cross-namespace.

Add a controller that mirrors the operator wildcard Secret into every
tenant namespace that owns a TLS termination point (its
namespace.cozystack.io/ingress or .../gateway label equals its own name).
The source is identified by name from the platform values channel
(_cluster.wildcard-secret-name) with no extra operator input; replicas
carry a managed-by label and refresh on rotation. An individual replica is
removed when its namespace stops terminating TLS; every replica is torn
down only on an explicit disable (clearing wildcard-secret-name). A missing
or mistyped source, or a transiently-absent values channel, keeps existing
replicas, so tenant TLS never drops on a transient gap. The controller's
Secret cache is scoped to managed replicas and the values channel, not
every cluster Secret. Only the platform controller's ServiceAccount gains
Secret writes — no tenant RBAC is widened, and every consumer reads only
its own-namespace copy.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-01 10:47:14 +03:00
Andrei KvapilandClaude 844deb170f feat(controller): enforce recursive tenant quotas at runtime
Add a tenant quota reconciler to cozystack-controller that realises the
hierarchical (sub-tree) quota model at runtime. Each tenant's declared
quota is the budget for its whole sub-tree: a child that declares its own
quota carves a fixed slice out of the parent budget, while a child without
a quota shares the parent's remaining pool.

The controller maintains a per-namespace 'tenant-quota-allocated'
ResourceQuota (Kubernetes enforces the most restrictive quota, so it binds
without fighting Flux over the chart-rendered 'tenant-quota'), aggregating
sub-tree usage so a pool's members collectively stay within the budget. A
--tenant-quota-buffer-percent flag temporarily inflates pool budgets so
workloads already over a freshly-introduced quota keep running during
rollout; overcommitted pools emit a QuotaOvercommitted event.

The model and the selector-spans-namespaces/aggregated-usage approach are
adapted from OpenShift's ClusterResourceQuota controllers (openshift/api,
library-go, cluster-policy-controller, apiserver-library-go), Apache-2.0,
Copyright Red Hat, Inc. and the OpenShift Authors.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-06-30 16:12:17 +02:00
Aleksei Sviridkin 664704d7b6 feat(controller): reconcile ServiceExposure into backend LB resources
Add the serviceexposure controller to the cozystack-controller binary. It
resolves a ServiceExposure's ExposureClass to a backend and renders the
backend-specific pool and announcer CRs, then reports the assigned IPs in
status.

Backends are pluggable behind a small interface, one per file:

- externalIPs: renders nothing (the node owns the address) — preserves the
  historical default behaviour.
- metallb: IPAddressPool + L2Advertisement in cozy-metallb, scoped to the
  target Service's namespace.
- cilium: CiliumLoadBalancerIPPool (cilium.io/v2) + CiliumL2AnnouncementPolicy
  (cilium.io/v2alpha1), scoped to the Service via serviceSelector.
- robotlb: renders nothing (the cloud LB owns allocation and announcement).

The rendered CRs belong to foreign API groups not in the Go scheme, so they
are emitted as unstructured objects. They carry no OwnerReference (the
Cilium CRs are cluster-scoped and the MetalLB CRs live in a different
namespace than the owning ServiceExposure), so ownership is tracked via a
management label plus owner annotations, cleanup is driven by a finalizer,
and the controller refuses to take over an identically-named resource it
does not manage. The controller never mutates the target Service, leaving
its shape to the chart that renders it.

Grant the controller write access to metallb.io and cilium.io pool /
announcer resources.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-25 20:47:19 +03:00
Aleksei Sviridkin 13f1f9ece1 feat(api): add network.cozystack.io ExposureClass and ServiceExposure
Introduce a new network.cozystack.io/v1alpha1 API group with two kinds
that form the mechanism-agnostic LoadBalancer exposure model:

- ExposureClass (cluster-scoped): the admin binds a logical class to a
  concrete backend (externalIPs, metallb, cilium, robotlb) and its VIP
  address scope, following the StorageClass / IngressClass pattern.
- ServiceExposure (namespaced): the tenant-facing intent to expose a
  Service externally via a named class, with status reporting the
  resolved backend and assigned IPs.

Wire api/network into the controller-gen CRD generation so deepcopy and
the two CRD manifests are produced alongside the existing groups.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-25 17:57:38 +03:00
cozystack-ci[bot] e1aaa90ed3 Prepare release v1.5.0
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-06-22 10:55:37 +00:00
myasnikovdaniil aeaedca4a4 ci: consolidated platform & e2e stabilization batch (#2948)
## What

Consolidated landing branch for a set of interdependent platform/CI
stabilization fixes. It began as the source PRs in the table below (in
dependency order) and has since grown with review-driven fixes and a few
production-behavior changes — the full current inventory is under
**Scope** below. Landing them as one unit lets CI run against the real
combined state instead of a fragile bottom-up merge train.

## Source PRs (dependency order)

| # | Commit | Source PR | Addresses |
|---|--------|-----------|-----------|
| 1 | fix(capi): startupProbe on capi-controller-manager | #2946 | capi
crashloop during cert provisioning |
| 2 | test(e2e): bucket.bats port-forward + S3 client reliability |
#2944 | flaky bucket test |
| 3 | fix(seaweedfs): restore -lock BucketClass, s3 svc name, drop
volumeSizeLimitMB | #2943 | missing bucket/access classes |
| 4 | test(e2e): in-cluster Cilium endpoint-leak healer (install + apps)
| #2874 | cilium "IP already in use" leak |
| 5 | refactor(build): mode=max registry cache + main-only warmer |
#2938 | build cache |
| 6 | ci(build): isolate each PR build on its own ephemeral runner VM |
#2939 | shared-buildkit contention |
| 7 | fix(e2e): LINSTOR post-install waits on a single 15m deadline |
#2928 | LINSTOR provisioning timeouts |
| 8 | test(metallb): assert digest-pinned image form, not version
literal | #2873 | brittle metallb assertion |

## Why consolidated

These form a dependency DAG (verified from CI logs): e.g. 2943 needs
2946, 2938 needs 2874, 2928 needs 2938/2939, 2873 needs 2928. Tested
bottom-up, each lower PR runs with none of the fixes above it and can't
go green alone. This branch carries all of them, so CI runs against the
real combined state.

## Verification status

CI is green on the latest head (`5653c30`): full-suite E2E passes
end-to-end and is reproducible (2 of 3 runs on this SHA green). The one
red run was a LINSTOR tie-breaker / DRBD-metadata infra flake on the
sandbox, unrelated to the diff — a different environmental subsystem
failed each run (details in the comments below). The earlier 3-hour
crust-gather snapshot hang is fixed in `5653c30`.

## Scope beyond the original 8

The branch has grown past the 8 source PRs above with review-driven
fixes and a few production-behavior changes. For reviewer transparency,
the full set:

**Review fixes (@lexfrei review):**
- **B1 (blocker)** — `8f41910` converts the SeaweedFS
`-lock`/`-readonly` BucketClasses and the s3 service-name override into
`patches/`, wired into `make update`, so a re-vendor no longer drops
them.
- **FU1** — `cc99230` grants the cilium leak-healer `delete` on pods
(both delete remedies were RBAC-forbidden).
- **FU3** — `014de3f` adds a unit test for the HelmRelease update
conflict-retry path.
- **FU4** — `6deddc6` makes the leak-healer refuse a disconnect for any
non-terminal owner phase, not just `Running`.
- **FU7** — `f3914d2` sets `persist-credentials: false` on the
build-cache checkout (SHA-pinning is handled repo-wide in #2849).

**Production-behavior changes (not in the table above):**
- VPA `updateMode: Auto → Initial` for etcd (`6e9ff90`) and monitoring
(`0997105`), plus `vmselect`/`vmstorage` `minAllowed` floors — stops
install-time eviction churn.
- cozystack-api: `retry.RetryOnConflict` on the Application→HelmRelease
update path (`728f676`).

**E2E robustness follow-ups:** per-test crust-gather snapshots
(`50e5f94`), EtcdBackupSchedule wait (`763b85e`), tenant-node
single-deadline wait (`9e43d45`), harbor BucketClaim 10m budget
(`d7f244f`), tenant API via LoadBalancer (`9beda4e`), crust-gather pin
(`95f42aa`).

**Not addressed (by decision):** FU2 — the s3 Service keeps
`seaweedfs.name`-based naming via patch (rendered output unchanged;
cozystack renders `fullname == seaweedfs`, so it matches the
`componentName`/`fullname` siblings in practice). FU6 — fork PR build
push is handled by the existing mirror-to-same-repo-branch workflow.

## Relates to

#2946 #2944 #2943 #2874 #2938 #2939 #2928 #2873 — if this lands, those
can be closed; otherwise they remain the granular per-PR review path.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added CI build cache warming for main branch builds.
* Added a best-effort in-cluster Cilium endpoint self-heal watchdog for
e2e installs.
* Extended SeaweedFS COSI with object-lock support plus separate
readonly access.

* **Improvements**
* Reduced e2e install flakiness with a shared readiness deadline, safer
waits, and improved cleanup/diagnostics capture.
* Prevented install-time churn by switching VPA update mode to
**Initial**.
  * Added container startup probes and updated SeaweedFS volume sizing.

* **Tests**
* Improved e2e robustness (etcd backup schedule waiting, digest-pinned
image checks, and corrected S3 port-forwarding).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-22 09:19:22 +05:00
Myasnikov DaniilandClaude Opus 4.8 d3958c5bcb refactor(build): mode=max registry cache with a main-only warmer (#2938)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-18 10:40:05 +05:00
Aleksei Sviridkin 4262658f0f feat(tenantgateway): add existingSecret cert mode for operator wildcard
Operators who already hold a wildcard TLS certificate (purchased, or
issued by a corporate CA) had no way to make a tenant Gateway serve
under it: the only cert modes were http01 (per-listener ACME) and
dns01 (controller-minted wildcard via ACME).

Add a third mode, existingSecret, that references a pre-existing TLS
Secret in the TenantGateway's own namespace via spec.wildcardSecretRef.
In this mode the controller mints no ACME Issuer and no Certificate;
the wildcard, apex, and per-child-apex HTTPS listeners reference the
supplied Secret directly. A missing wildcardSecretRef fails fast with
Ready=False rather than producing a Gateway pointing at a nonexistent
Secret. Switching into existingSecret from http01/dns01 reclaims the
now-unused Issuer and Certificates so no ACME machinery is left behind.

The non-cert listeners (http->https redirect, TLS passthrough) are
unaffected and continue to render.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-10 03:08:07 +03:00
Myasnikov Daniil 9868c43e53 fix(build): read --cache-from from ghcr :latest so PR builds warm-start
PR #2711 switched PR builds to push only :pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to stop concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry (iad.ocir.io) and never publish :latest there, so every PR build's registry cache lookup 404s and rebuilds cold.

Add a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and point every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so the #2711 anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-09 14:24:36 +05:00
Aleksei Sviridkin 3756f5c459 feat(networking): Gateway API support via Cilium (supersedes #2213) (#2470)
## What this PR does

Replaces ingress-nginx as the default ingress layer for cozystack-native
services with **Cilium Gateway API**, materialised per-tenant via a new
`TenantGateway` CRD reconciled by `cozystack-controller`. Supersedes
#2213 / #2208. Stacked on top of #2468.

## Architecture

The chart renders one `gateway.cozystack.io/v1alpha1 TenantGateway` CR
per tenant that owns its own Gateway. The cozystack-controller
reconciles the actual `Gateway`, per-tenant `Issuer`, `Certificate`
(wildcard or per-listener), and HTTP→HTTPS redirect `HTTPRoute` from
there. Helm does not render `Gateway` or `Certificate` directly — that
prevents the Helm-vs-controller race on `Gateway.spec.listeners` that
route-driven listener additions would otherwise cause.

```
┌── TenantGateway CR (rendered by extra/gateway chart for owning tenants)
│
└──▶ cozystack-controller reconciles:
        ├─ Gateway (with dynamic listeners, label-based allowedRoutes)
        ├─ Issuer (per-tenant ACME account)
        ├─ Certificate(s) — wildcard + per-child SANs in DNS-01, per-listener in HTTP-01
        ├─ HTTPRoute (controller-owned http→https redirect)
        ├─ Namespace label patching (cozy-* system namespaces from spec.AttachedNamespaces)
        └─ Watches HTTPRoute / TLSRoute attachments to materialise listeners
```

## Inheritance: tenants attach by default, opt into their own Gateway

A tenant gets a dedicated Gateway (own LB Service, own LB IP, own
Certificate) only when it explicitly asks via
`tenant.spec.gateway=true`. Every other tenant in the tree publishes
through the Gateway of the nearest ancestor that owns one — same shape
as the existing `_namespace.ingress` inheritance.

The `apps/tenant` chart writes a `namespace.cozystack.io/gateway` label
on each tenant namespace, carrying either the tenant's own name (when
owning a Gateway) or the inherited ancestor name (when inheriting). The
Gateway's listener `allowedRoutes.namespaces.selector` is keyed on that
label, so the same selector admits routes from every namespace pointing
at the owner — descendants and cozy-* system namespaces alike.
cozystack-controller patches the same label onto every namespace in
`tgw.Spec.AttachedNamespaces` and garbage-collects labels it wrote when
an entry is removed from the attach list (annotation-gated GC so labels
written by the apps/tenant chart are never stripped).

In DNS-01 mode, the controller extends the parent's wildcard Certificate
with `<child-apex>` + `*.<child-apex>` SANs per inheriting child, and
adds a `*.<child-apex>` HTTPS listener per child apex referencing the
same cert. Without this expansion the parent's single-level wildcard
cannot match a child route's hostname (`harbor.alice.example.org` is two
labels past the parent's `*.example.org`).

Opt-out into own Gateway is the right choice when: a tenant needs its
own LB IP (DNS pinned, firewall rule), the apex is not derived from the
parent (custom `host` value the ancestor's cert can't cover), or the
tenant wants its own ACME account / cert authority. Otherwise leave
`gateway` unset and inherit.

## End-to-end flow

1. Platform: `gateway.enabled=true` toggles cert-manager solver,
switches per-system-app templates from Ingress to HTTPRoute / TLSRoute.
2. Per tenant: explicit `tenant.spec.gateway=true` renders a
`TenantGateway` CR and triggers the controller to materialise the owner
Gateway. Tenants with the field unset inherit through
`_namespace.gateway` propagation in the tenant chart.
3. Adding a published app under any tenant (owner or inheriting) is just
deploying its HTTPRoute. The controller picks up the hostname, adds a
per-listener Certificate (HTTP-01) or extends the wildcard cert's SANs
(DNS-01), and updates the owner Gateway's listeners.

Defaults stay legacy: `gateway.enabled=false`,
`tenant.spec.gateway=false`. Existing clusters upgrade unchanged.

## External IP allocation — mechanism-agnostic

The per-tenant Gateway's auto-created `LoadBalancer` Service draws its
IP from whatever LB allocator the cluster admin has configured at the
platform layer — same shape as ingress-nginx today. The tenant API stays
mechanism-agnostic — no `gatewayIP` field, no allocator-specific
manifest in the tenant chart. Cozystack itself ships MetalLB installed
but does not render any `IPAddressPool` / `L2Advertisement` /
`BGPAdvertisement` from this chart; admins set up the allocator that
suits their environment (MetalLB pool with L2 / BGP, Cilium LB-IPAM with
announcer, robotlb against a cloud provider, or
`Service.spec.externalIPs` pinning).

If a tenant needs a specific address (DNS already pinned, firewall rule,
etc.), the operator pre-allocates it on the admin side: either
pre-create the Service with `loadBalancerIP` set, or hand the tenant a
reference to a named admin-managed pool. Per-Service IP uniqueness is
the allocator's responsibility — same as for any other LoadBalancer
Service.

## Cert mode: HTTP-01 (default) vs DNS-01 (opt-in)

`publishing.certificates.solver` selects how the controller sources TLS
certs.

**HTTP-01 (default)** — per-app HTTPS listener + per-app Certificate.
New apps require zero platform-side config: deploy the HTTPRoute,
controller does the rest. Inheriting children's hostnames pick up
per-listener certs the same way the owner's own apps do.

**DNS-01 (opt-in)** — single wildcard Certificate (`<apex>` +
`*.<apex>`) covering every published app under the apex, plus per-child
SANs (`<child-apex>` + `*.<child-apex>`) for every inheriting tenant.
Pick this for clusters where many apps share the apex and Let's Encrypt
rate limits matter. The DNS provider account must be able to write TXT
records under every apex level the parent serves — for deeply-nested
inherited children that requires zone delegation or a wide-scope
provider credential.

| `publishing.certificates.dns01.provider` | Required
`publishing.certificates.dns01.<provider>` keys |
| --- | --- |
| `cloudflare` (default) | `cloudflare.secretName`,
`cloudflare.secretKey` |
| `route53` | `route53.region`, `route53.secretName` (and `accessKeyID`
if not on IRSA) |
| `digitalocean` | `digitalocean.secretName` |
| `rfc2136` | `rfc2136.nameserver`, `rfc2136.tsigKeyName`,
`rfc2136.secretName` |

The platform chart wires every provider's keys into `_cluster.dns01-*`;
the per-tenant gateway chart and the cluster-wide `cluster-issuers.yaml`
both read them.

## Strengthens admission coverage on apps.cozystack.io/*

The `cozystack-tenant-host-policy` probe in `gateway.bats` initially
looked like a flake — the apply that was supposed to be VAP-rejected
just succeeded. Investigation traced it to the custom REST handler at
`pkg/registry/apps/application/rest.go`. genericapiserver hands every
storage method a callback into the validating admission chain:
`createValidation` for Create, `updateValidation` for Update,
`deleteValidation` for Delete. `Update` was correctly invoking its
callback since 23e399bd, but `Create` and `Delete` accepted the
parameter and never called it — admission was silently bypassed for both
verbs on every `apps.cozystack.io` resource (Tenant, MariaDB, Postgres,
Kubernetes, …).

This PR wires the missing `createValidation` and `deleteValidation`
calls into Create and Delete. Every ValidatingAdmissionPolicy and
ValidatingWebhook against `apps.cozystack.io/*` now fires on all three
verbs as the API contract requires. Per KEP-3488 aggregated APIServers
are responsible for enforcing admission on their own kinds;
cozystack-api is configured to do so via
`genericoptions.NewRecommendedOptions`, the only thing missing was the
explicit `*Validation` callbacks in the custom REST.

## Security — five guards, grouped by what they defend

Tenants in Cozystack interact with the platform exclusively through
`apps.cozystack.io/*` resources (Tenant, Bucket, Kubernetes, …) served
by `cozystack-api`. Tenant RBAC (`cozy:tenant:*` aggregated to a
RoleBinding in the tenant's own namespace) does not grant write access
to `gateway.networking.k8s.io/*`, core `Namespaces`, or
`cozystack.io/Package`. The protections below split into three groups by
who they defend against:

**Tenant-user-input gates.** Layer 4 (`cozystack-tenant-host-policy`).
`Tenant.spec.host` is the user-supplied field that surfaces as a
security boundary at the hostname layer; gated on every Create / Update
via `cozystack-api`'s admission chain.

**Defense-in-depth.** Layers 1, 2, 5, 7. These do not protect against
tenant-user input (tenants don't hold the relevant RBAC). They guard
against bugs in cozystack-controller / Flux, supply-chain compromise of
an app chart that emits Gateway API resources, and confused-deputy
mistakes by a cluster admin. Fail-closed via `failurePolicy: Fail` +
`validationActions: [Deny]`.

The five layers themselves:

1. **Listener `allowedRoutes` label selector** —
`namespace.cozystack.io/gateway = <owner-tenant-name>`
(controller-written for cozy-* via `AttachedNamespaces`, Helm-written
for tenant namespaces via the tenant chart, both feeding into the same
Gateway selector).
2. **`cozystack-gateway-hostname-policy`** — VAP on `Gateway`
CREATE/UPDATE. Restricts listener hostnames to the namespace's
`namespace.cozystack.io/host` apex.
4. **`cozystack-tenant-host-policy`** — VAP on `Tenant` CREATE/UPDATE.
5. **`cozystack-namespace-host-label-policy`** — VAP on core `Namespace`
CREATE/UPDATE. Immutability gate on the host label.
7. **`cozystack-route-hostname-policy`** — VAP on `HTTPRoute` and
`TLSRoute` CREATE/UPDATE. Closes the hostname-hijack vector across
namespaces under inheritance — a child cannot claim a hostname outside
its own apex.

The previous Layer 3 (`cozystack-gateway-attached-namespaces-policy`)
and Layer 6 (render-time `fail` in cozystack-basics) banned `tenant-*`
entries in `publishing.gateway.attachedNamespaces` on the assumption
that the attach list was the hijack vector. Under inheritance the attach
surface is the label selector, not the static list; the hijack vector is
closed by Layers 4/5/7 independently. Both Layer 3 and Layer 6 are
dropped — kept in numbering above for continuity with the earlier review
threads.

## Foreign-takeover guards

Five reconcile paths refuse to silently rewrite a pre-existing object
that shares the controller-derived name but carries no `OwnerReference`
back to the TenantGateway: `Gateway`, redirect `HTTPRoute`, per-tenant
`Issuer`, wildcard `Certificate`, per-listener `Certificate`. An
operator who hand-pinned a Certificate or Issuer at the controller's
derived name (private CA, manual cert pinning) gets an explicit
`Ready=False/ReconcileError` condition instead of having their config
silently destroyed and the resource re-issued from a different ACME
account.

The namespace-label patching path applies the same discipline:
cozystack-controller only writes / strips
`namespace.cozystack.io/gateway` on namespaces it annotates with
`cozystack.io/gateway-attached-by`. Labels written by the apps/tenant
chart (no annotation) are never touched, so inheritance for tenant
namespaces survives every reconcile.

## Tests

- **Go controller tests** — 50+ cases in
`internal/controller/tenantgateway/`: idempotency, OwnerReference
cascade, multi-parentRef status, HTTP-01↔DNS-01 mode-transition cert
lifecycle, foreign-takeover refusals (Gateway / HTTPRoute / Issuer /
Certificate / per-listener Certificate), DNS-01 provider matrix
(cloudflare / route53 / digitalocean / rfc2136), HTTPS Kinds=[HTTPRoute]
restriction, fail-closed CEL on missing host label, status condition
wiring. Plus, for inheritance: label-based allowedRoutes selector shape,
namespace label patch + garbage-collect via owner annotation, Helm-owned
label safety, DNS-01 wildcard SAN expansion per inheriting child,
per-child wildcard listener materialisation.
- **Admission-chain tests** —
`pkg/registry/apps/application/rest_admission_test.go` pins
`createValidation`, `updateValidation`, `deleteValidation` paths via
sentinel callbacks.
- **helm-unittest** across `extra/gateway` (TenantGateway CR rendering,
both modes), `core/platform` (DNS-01 wiring), `apps/tenant`
(gateway-effective decision matrix including inheritance),
`system/cozystack-basics` (three VAPs + Layer 7 VAP + tenant-root
namespace label including the conditional
`namespace.cozystack.io/gateway: tenant-root` + cozystack-values-secret
`_namespace.gateway` propagation), `system/cert-manager-issuers` (full
DNS-01 provider matrix), `system/cozystack-api` and `system/dashboard`
(Ingress↔TLSRoute/HTTPRoute toggle), `system/monitoring` (Grafana /
Alerta Ingress↔HTTPRoute toggle).
- **e2e bats** — `hack/e2e-apps/gateway.bats` (17 cases) covering
GatewayClass Accepted, Gateway Programmed + LB Service, HTTPRoute
Accepted with status, VAP rejects cross-tenant hostname hijack, VAP
rejects `tenant.spec.host` writes from non-trusted SA
(regression-detector for the admission-chain fix), VAP rejects
`namespace.cozystack.io/host` label changes from non-trusted SA,
route-hostname VAP rejects HTTPRoute hostnames outside the apex, and
three new cases for inheritance — Package admission accepts `tenant-*`
in `attachedNamespaces` (regression guard for the dropped ban), child
tenant without explicit gateway inherits `_namespace.gateway` from a
Gateway-owning parent in lockstep across the namespace label and the
cozystack-values Secret, and the cross-namespace attach end-to-end where
a child tenant's HTTPRoute reaches Accepted on the parent's Gateway via
the inheritance label.
- **CI wiring** — `make test-controllers` target invoked from
`pull-requests.yaml` runs `go test ./internal/...`.

## Backward compatibility

The only always-on change is the flip of `cilium.envoy.enabled` +
`cilium.gatewayAPI.enabled` to `true` (extra `cilium-envoy` DaemonSet,
~100 MB RAM/node idle). Everything else is gated behind
`gateway.enabled=false` / `tenant.spec.gateway=false`. Existing clusters
see no behavioural change until an operator explicitly opts in.

The cozystack-api admission-chain fix is unconditional. Existing
operators who do not define any VAP / admission webhook on
`apps.cozystack.io` resources see no observable change. Operators who DO
have their own webhooks/VAPs against these kinds will start seeing them
fire on Create and Delete (as intended) — review your policies before
merging if you maintain custom admission for `apps.cozystack.io/*`.

Operators who set `publishing.gateway.attachedNamespaces` to include
`tenant-*` entries used to hit a render-time fail or an admission
rejection — now the apply succeeds and the listed tenant namespaces
simply pick up the gateway-attach label alongside the cozy-* system
namespaces. This matches the inheritance model and was kvaps' explicit
ask in the May 25 review.

## Release note

```release-note
- Adds Gateway API support via Cilium, materialised per-tenant via the new `gateway.cozystack.io/v1alpha1 TenantGateway` CRD reconciled by `cozystack-controller`. Opt-in via `publishing.gateway.enabled=true` at the platform level; per-tenant opt-in via `tenant.spec.gateway=true` for dedicated Gateway / LB IP, otherwise the tenant inherits the nearest ancestor's Gateway via a label-based `allowedRoutes` selector (same shape as the existing `_namespace.ingress` inheritance). Includes HTTP-01 (default) and DNS-01 (opt-in, with cloudflare / route53 / digitalocean / rfc2136 providers) cert solvers; DNS-01 extends the parent's wildcard Certificate with per-child apex SANs and adds a `*.<child-apex>` listener per inheriting tenant. Tenant API is mechanism-agnostic for IP allocation — the LoadBalancer Service draws its IP from the cluster's admin-configured allocator (MetalLB / Cilium LB-IPAM / robotlb / externalIPs).
- Strengthens admission coverage on `apps.cozystack.io/*` resources: `cozystack-api` now invokes `createValidation` and `deleteValidation` callbacks on Create and Delete (Update was already wired). Operators with custom ValidatingAdmissionPolicies or webhooks targeting `apps.cozystack.io/*` will see them fire on all three verbs as the API contract requires; review your policies before merging if you maintain such admission.
```
2026-05-26 10:29:30 +03:00
Aleksei Sviridkin d2e851f63b feat: Gateway API support via Cilium
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.

The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.

publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.

Security model -- seven independent layers, all admission-time gates
fail-closed:

  1. Listener allowedRoutes namespace whitelist via the
     kube-apiserver-written kubernetes.io/metadata.name label;
     plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
     passthrough listeners restrict allowedRoutes.kinds to
     HTTPRoute / TLSRoute respectively.
  2. cozystack-gateway-hostname-policy VAP on Gateway.
  3. cozystack-gateway-attached-namespaces-policy VAP on Package --
     blocks tenant-* in gateway.attachedNamespaces.
  4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
     writes from non-trusted callers.
  5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
     writes to namespace.cozystack.io/host from non-trusted callers.
  6. Render-time fail in cozystack-basics for tenant-* in
     gateway.attachedNamespaces.
  7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
     scoped to tenant-* namespaces.

Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).

Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.

Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.

Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.

Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.

Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.

Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:07:02 +03:00
Myasnikov Daniil ed0297eb87 refactor(build): standardize image tagging to fix concurrent PR push conflicts
Every package Makefile tagged images via `settag` which, on a non-release
commit, collapsed to `:latest`. Concurrent PR builds therefore raced on every
image's `:latest` manifest in the OCIR registry. Two outliers (metallb,
redis-operator) used hardcoded version tags and collided across PRs even when
`settag` would not have.

Replace `settag` and the `TAG=latest` fallback with an `image-tags` macro
driven by three env vars:
  - IMAGE_TAG          — build-unique tag, always pushed
  - PUBLISH_VERSIONED  — also push :<component-version>
  - PUBLISH_FLOATING   — also push :latest

PR CI sets IMAGE_TAG=pr-<N>-<sha> with both PUBLISH_* off, so every push is
unique across runs. Release CI sets IMAGE_TAG=<git-tag> with both PUBLISH_*
on, preserving the previous release output and adding a uniform :<git-tag>
tag across every image. Local builds default to IMAGE_TAG=dev with PUSH=0
already off.

Image consumers reference digests pinned into values.yaml by each Makefile,
so the tag is decorative for cluster-side pulls; the build-unique tag is
sufficient for E2E.

Drops the redundant `:<component-version>-<git-tag>` form (cilium, linstor,
matchbox, ubuntu-container-disk, kubevirt-*, cluster-autoscaler) — nothing
in the repo or downstream values.yaml consumed it. The same information is
preserved via the new uniform `:<git-tag>` plus the existing
`:<component-version>` on release.

ubuntu-container-disk builds a separate image per Kubernetes version in a
loop. Switched its primary tag to `:<k8s-ver>-<IMAGE_TAG>` so loop
iterations within a single build do not push to the same tag.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-21 15:23:18 +05:00
Andrei KvapilandClaude 4d5486cd84 chore(controller): remove dead dashboard controller
The dashboard controller used to project ApplicationDefinitions
into a set of dashboard.cozystack.io CRDs (Sidebar, Factory,
Navigation, Breadcrumb, CustomFormsOverride, CustomFormsPrefill,
CustomColumnsOverride, MarketplacePanel, TableUriMapping,
CFOMapping, BreadcrumbInside) consumed by the legacy UI.

cozystack-ui now reads ApplicationDefinitions directly, so the
controller, its API types, the generated CRDs and the related
RBAC rules are no longer used.

Removes:
- internal/controller/dashboard
- api/dashboard/v1alpha1
- packages/system/cozystack-controller/definitions/dashboard.cozystack.io_*.yaml
- dashboard.cozystack.io wiring from cmd/cozystack-controller/main.go
- dashboard.cozystack.io RBAC rules from cozystack-controller and cozy-dashboard charts
- ./api/dashboard/... codegen path from hack/update-codegen.sh

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-05-20 16:33:21 +02:00
cozystack-ci[bot] 9f655e29ff Prepare release v1.4.0-rc.2
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-05-13 21:48:39 +00:00
cozystack-ci[bot] 252c18f3e8 Prepare release v1.4.0-rc.1
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-05-13 12:29:01 +00:00
Aleksei Sviridkin 3fea7cd487 fix(security): bump first-party Dockerfile builders to golang:1.26
The root go.mod now declares go 1.26.2 (raised in the first commit
of this PR). Every Dockerfile that copies the root go.mod and builds
a cozystack-owned binary has to use a matching base image, otherwise
the in-container go toolchain has to download go 1.26 on the fly
(via GOTOOLCHAIN=auto) at every build, which slows builds and breaks
in network-restricted CI sandboxes.

Bumps:
- packages/core/installer/images/cozystack-operator
- packages/system/backup-controller/images/backup-controller
- packages/system/backupstrategy-controller/images/backupstrategy-controller
- packages/system/cozystack-api/images/cozystack-api
- packages/system/cozystack-controller/images/cozystack-controller
- packages/system/flux-plunger/images/flux-plunger
- packages/system/kubeovn-plunger/images/kubeovn-plunger
- packages/system/lineage-controller-webhook/images/lineage-controller-webhook

All eight images go from golang:1.25-alpine to golang:1.26-alpine.
Also normalizes the cozystack-operator Dockerfile FROM stage keyword
from lowercase 'as' to uppercase 'AS' while touching it.

Vendored Dockerfiles (kubevirt-cloud-provider, redis-operator,
keycloak-operator, linstor-csi, multus-cni, objectstorage,
s3manager, kamaji) are intentionally untouched: they pull upstream
sources at build time and pin their go base image to whatever the
upstream commit was tested against.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 01:53:19 +03:00
cozystack-ci[bot] b52e2801b4 Prepare release v1.3.0
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-22 07:28:09 +00:00