Commit Graph
7777 Commits
Author SHA1 Message Date
Alexey Artamonov ac82ce186c fix(keda): pin cozy.local domain and raise operator API throttle
The vendored KEDA chart defaults clusterDomain to cluster.local, but
cozystack runs on cozy.local. KEDA's metrics-apiserver dials the operator
metrics gRPC service at keda-operator.<ns>.svc.<clusterDomain>:9666 and the
operator mints that service's serving-cert SANs from the same domain, so the
default resolves to NXDOMAIN and every HPA hangs on FailedGetExternalMetric.
Pin keda.clusterDomain=cozy.local.

Also raise the operator's client-go throttle to 50 QPS / 100 burst: at the
upstream 20/30 a single-replica operator loses its leader lease under
scale-subresource polling and crash-loops (the client rate limiter starves
the lease renewal). Both were observed live on a dev cluster; guard them in
the package unittest.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-04 23:18:47 +03:00
Alexey Artamonov d4c76fadb4 feat(postgres): two-phase autoscaling migration and read-CPU metric
Make enabling autoscaling on an existing database count-preserving, and
add a second read-load signal:

- autoscaling.transition (migration phase 1): keep .spec.instances
  rendered as a floor (from replicas, which the operator stages to the
  live count) AND pause the ScaledObject, so KEDA stands up without
  contending with Flux for the field. Flip transition off (phase 2) to
  drop the static field and hand the count to KEDA — suspend the
  HelmRelease across that apply, since helm-controller uses Helm's
  three-way merge (not SSA) and would otherwise race CNPG's default-to-1.
  The procedure is documented on the values field.
- autoscaling.metric (ReadConnections | ReadCPUUtilization): the query
  helper emits either the active-connections sum or the postgres-container
  CPU sum, both + target.

active = enabled && !dryRun && !transition; the ScaledObject is paused
whenever dryRun or transition is set. Covered by helm unittest (phase-1
holds the staged count + pauses; CPU metric query) and regenerated
schema/README/Go types/postgres-rd.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-04 23:18:47 +03:00
Alexey Artamonov 237053191e feat(keda): ship database-autoscaling alerts and Grafana dashboard
Add cozystack-owned observability for managed-database read-replica
autoscaling, rendered by the keda umbrella (not the vendored subchart):

- PrometheusRule (templates/alerts.yaml), scoped to the KEDA-generated
  HPAs (keda-hpa-*): DatabaseAutoscalerScaleStuck (desired > current for
  15m — stuck scale-up with no automatic rollback), DatabaseAutoscalerAtMax
  (ScalingLimited=true for 30m — at max or quorum-raised ceiling), and
  DatabaseAutoscalerScalerErrors (KEDA cannot evaluate the trigger — never
  scale blind).
- A Grafana dashboard ConfigMap (grafana_dashboard sidecar label) with
  desired-vs-current replicas, active read connections, replication lag,
  and scaling-limited panels.

Both gated by databaseAutoscaling.alerts.enabled / .dashboard.enabled
(default true). Metric names are calibrated on a live cluster per the
proposal's PoC; the rule and panel shapes are what ships. Covered by helm
unittest (PrometheusRule with the three alerts + disable; dashboard
ConfigMap label + disable).

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-04 23:18:47 +03:00
Alexey Artamonov 7f8f2c1d86 feat(keda): add KEDA as an optional platform component
Vendor the upstream KEDA chart (kedacore/keda 2.20.2) as a new optional
system package and wire it into the platform, so database read-replica
autoscaling (which renders KEDA ScaledObjects) has a backend.

- packages/system/keda: umbrella cozy-keda + vendored chart (Makefile
  pins KEDA_CHART_VERSION with a lockstep assert), shim values enabling
  the operator / metrics-server / webhook ServiceMonitors under the
  platform Prometheus.
- core/platform: PackageSource cozystack.keda (namespace cozy-keda,
  dependsOn networking + prometheus-operator-crds; cert-manager is not
  needed, KEDA self-generates its webhook TLS) plus an optional bundle
  entry, opt-in via bundles.enabledPackages. It is not a dependsOn of any
  default package, so the platform HelmRelease never blocks on it.
- KEDA claims the cluster-wide external.metrics.k8s.io APIService
  (distinct from metrics-server's metrics.k8s.io — no conflict).

Covered by helm unittest: platform bundle enable/disable/opt-out, the
PackageSource dependsOn, and the vendored chart's external-metrics
APIService + shim-enabled ServiceMonitor.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-04 23:18:47 +03:00
Alexey Artamonov 0335276b61 feat(postgres): render a KEDA ScaledObject for read-replica autoscaling
Implement the KEDA-based read-replica autoscaler for postgres per the
merged design proposal (cozystack/community#44):

- cozy-lib.keda.scaledObject: engine-agnostic helper rendering a KEDA
  ScaledObject (scale target, effective bounds, prometheus trigger,
  scale-down behavior, optional pause), fail-closed on bad input.
- postgres: an `autoscaling` values block (enabled / minReplicas /
  maxReplicas / target / maxReplicationLagSeconds / dryRun). Under active
  autoscaling the chart drops the static .spec.instances so Flux and the
  KEDA-managed HPA no longer contend, and renders a ScaledObject whose
  minReplicaCount folds in the synchronous-quorum floor
  (max(minReplicas, maxSyncReplicas+1, 2)); when the floor exceeds
  maxReplicas, quorum wins. dry-run keeps the static count and pauses the
  ScaledObject. The WorkloadMonitor tracks the quorum floor instead of a
  stale static value.

Covered by helm unittest (cozy-lib helper: valid render, pause, three
fail-closed guards; postgres: instances dropped, quorum-floored bounds,
quorum-over-max, dry-run pause). Regenerated values.schema.json, README,
Go types and postgres-rd.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-04 23:18:47 +03:00
Aleksei Sviridkin 3a369fed99 feat(redis): add opt-in TLS for Redis and Sentinel (#2729)
## What this PR does

`tls.enabled` puts Redis and Sentinel on a TLS-only listener with a
per-release cert-manager chain, and hands the tenant the CA certificate
as `redis-<name>.tenant-ca`. Off unless set.

redis-operator v3.3.5 has no TLS support. The work is ported upstream in
freshworks-oss/redis-operator#85, which is open and not in a release, so
this carries it as three patches over the v3.3.5 tarball.
`images/redis-operator/patches/tls.diff` is the source half, cut from
that PR's commit `2ad2ed8d1`, which the patch header pins; it is applied
by the glob the image Dockerfile already runs, after the Dockerfile
strips the tarball's `manifests`, `charts`, `example` and `.github`, so
the image builds from source alone and the upstream tests that read the
CRD YAML skip instead of failing against the stale v3.3.5 copy.
`patches/crd-tls.diff` adds `spec.tls` to the vendored CRD, and with it
the CEL rule that fixes `tls.enabled` at creation; without the schema
block the apiserver prunes the field on admission and the patched binary
waits for something it never sees, and the rule cannot exist without the
block it references. `patches/rbac-tls.diff` grants Create, Get and
Update on Secrets, so the operator can publish the CA-only Secret it
addresses by name. That is the whole delta, and it is narrower than the
ported operator's own chart asks for: those three are the only Secret
calls in the patched source, and cert-manager Certificates get no grant
at all, because the operator only touches one under
`spec.tls.certManager` and this chart pins `certificateSecret` mode
instead. One change in `tls.diff` is not TLS and takes effect with TLS
off, named in the patch header: `MakeSlaveOfWithPort` connects to the
replica on its own port rather than on the master's. It changes a dial,
not a pod template, so shipping the rebuilt image rolls nothing; the
plaintext pod templates stay byte-identical to v3.3.5.

The image build registration this branch used to carry (`make -C
packages/system/redis-operator image` in the root `build:` target)
landed on main with the Valkey service (#3380), so the rebuilt operator
image is produced by CI from main as well; the digest in `values.yaml`
is still refreshed by release preparation, not here.

The chart patches re-apply at the end of `update:`, the same way
reloader, grafana-operator, ingress-nginx, seaweedfs and kamaji carry
theirs. Otherwise `rm -rf charts` takes TLS back out on the next `make
update` and nothing fails. `make update` reproduces the committed tree
byte for byte.

TLS is chosen when the instance is created. The config gets `port 0` and
the listener moves to `tls-port`, so TLS replaces plaintext rather than
running beside it, and a running instance cannot change mode: a replica
restarted onto the TLS listener cannot replicate from a master still on
plaintext, and Sentinels already on TLS would fail over to a stale
replica. The RedisFailover schema in the carried CRD patch rejects a
change of `tls.enabled`, which on an existing instance surfaces as a
failed HelmRelease upgrade while the instance keeps serving what it did;
the README points at creating a new instance and migrating. That is also
why TLS is not inferred from `external`: an externally reachable
instance is the one most likely to have clients an upgrade must not
drop. Existing releases keep their plaintext listener and are not
restarted by the rebuilt operator image.

The tenant gets `ca.crt` and nothing else, through `tenantsecrets`. The
chart's own Secrets take a dot between the release and the suffix
(`<release>.ca-tls`, `<release>.tls`, `<release>.ca-cert`): the suffixes
are dot-free, so the split is unique and release `foo`'s CA Secret can
no longer be release `foo-ca`'s leaf, the collision the hyphenated form
had. The chart renders a `TenantProjection` pointing at
`<release>.ca-tls`, and the CA-extraction controller re-encodes the one
named key from the parsed certificate, so nothing that is not a
certificate gets through. Same path as postgres, mongodb, nats and
qdrant.

No Secret of this release is granted to the tenant by name beyond the
generated password, and that includes `<release>.ca-cert`, the CA-only
copy the operator publishes for itself. It holds no key today, but the
operator has Create and Update on Secrets cluster-wide, so what sits at
that name is the operator's call rather than the platform's, and RBAC
cannot filter by key. `hack/check-redis-rd-secrets.bats` pins the whole
rendered Secret surface instead of a list of known-bad names, because an
include with no `resourceNames` and no `matchLabels` restricts nothing.

The chainsaw suite for redis gains two cases. A TLS fixture asserts the
cert-manager chain is issued, the TLS-probed Redis replicas and Sentinel
come up, the operator publishes its CA-only Secret under the widened
ClusterRole, the tenant receives exactly `ca.crt` through
`tenantsecrets` while the key-bearing Secrets stay off that path, and a
`redis-cli --tls` PING through the service name completes with the same
CA. A second case applies `tls.enabled` to a running plaintext release
and asserts the refusal: the HelmRelease upgrade fails with the schema's
message, the StatefulSet keeps its revision with both replicas Ready,
plaintext still answers and no TLS listener appears.

Certificate renewal rolls the pods from inside the operator. Redis and
Sentinel read the certificate once at startup, so the patched operator
stamps a hash of the TLS Secret's `tls.crt` and `ca.crt` into the pod
template of both the Redis StatefulSet and the Sentinel Deployment on
every reconcile, as `redis-failover.freshworks.com/tls-secret-hash`. A
renewal changes the hash; the operator's own roller replaces the Redis
pods one at a time and the Deployment controller rolls Sentinel. README
names the fallback: if the roll cannot run, the deadline is the old
certificate's expiry, past which the operator's own TLS connections fail
along with the clients.

`tls.authClients` maps to `tls-auth-clients` and defaults to `no`. The
platform mints no client certificate for the tenant and the CA key is
out of reach, so `yes` is only usable when the client certificate comes
from whoever also controls the CA. Changing the value on a running
release restarts the Redis and Sentinel pods: Redis reads the directive
from its config file only at startup, so the operator folds the
effective value into the certificate hash that drives the rollout, and
the change lands when it is made rather than at the next unrelated
restart.

Follow-ups in the carried operator patch go upstream first, to
freshworks-oss/redis-operator#85 or a successor there, and are picked up
here afterwards; the carried patch does not grow Go that upstream has
not seen. `tlsConfigFor` fetches the Secret on every call, tens of GETs
per reconcile. The renewal roll is static analysis so far: the live
check, `cmctl renew` on a running release while watching the `rfr-*` and
`rfs-*` pods roll, is still owed and belongs with the e2e follow-up.

Two facts about the shipped image rather than the patch.
`packages/system/redis-operator/values.yaml` keeps the digest of the
image built before the patch until release preparation rebuilds it, so
on `main` between this merge and the next release-prep, `tls.enabled`
renders the chain and `spec.tls` while the running operator ignores them
and serves plaintext. And the pinned v3.3.5 tarball carries GO-2026-5026
in `golang.org/x/net` v0.53.0, reached through
`PodService.UpdatePodLabels`; the port neither introduces nor fixes it,
upstream `master` is already on v0.55.0, so it lands with the next tag
bump rather than here.

Outside the patch: `cozyvalues-gen` renders `{}` in the README Value
column for `tls.authClients`: a child property without a default
inherits the parent object's `{}`, and `make generate` reproduces it
byte for byte, so it is a generator bug rather than something to
hand-edit here.

### Screenshots

No UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: cozystack/terraform-provider-cozystack#37
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

I walked the trigger map against the diff file by file.
terraform-provider-cozystack is reached, tracked as
cozystack/terraform-provider-cozystack#37 rather than a speculative PR:
`values.schema.json` gains a `tls` object, and the provider hand-writes
a schema, model and expand/flatten pair per Kind. It already models
`tls` for postgres, kafka, nats and qdrant, not for redis. Redis needs
different wording from those four, whose descriptions say to omit the
block to follow `external`, which is the one thing redis does not do.

Nothing else is reached. The website regenerates this package's
reference page from `README.md` on a stable tag, so the README changes
need no PR there. No package added, renamed or removed.
`packages/core/platform/values.yaml`, variant and bundle composition and
release asset names are untouched, and so are `hack/package.mk`,
`hack/common-envs.mk` and `hack/update-crd.sh`, which is what ccp and
external-apps-example read. The two new files under `hack/` are bats
unit tests. No node prerequisite, installer value, telemetry metric or
cozy-proxy label changes.

### Release note

```release-note
feat(redis): add opt-in TLS for Redis and Sentinel. Set `tls.enabled: true` to serve both on a TLS-only listener backed by a per-release cert-manager chain; the tenant reads the CA certificate as the `<release>.tenant-ca` tenant secret. TLS is off unless set and is not inferred from `external`, so existing releases keep serving plaintext and are not restarted by the rebuilt operator image. TLS is chosen at creation: a running instance cannot switch modes, the change is rejected and surfaces as a failed HelmRelease upgrade, so add TLS by creating a new instance and migrating to it. Requires cert-manager, which the platform's `system` bundle installs on every variant. A renewed certificate rolls the Redis and Sentinel pods through the operator. TLS support is carried as a downstream patch over redis-operator v3.3.5 until freshworks-oss/redis-operator#85 lands in a release.
```
2026-09-04 19:32:32 +04:00
Aleksei Sviridkin d800526696 feat(mariadb): add TLS support via cert-manager (#2680)
## What this PR does

Gives MariaDB a chart-managed certificate chain, makes TLS enforcement
explicit, and hands the tenant a key-free trust anchor instead of a
private key.

- `tls.issuer` selects who issues the server certificate: `operator`
(the CA mariadb-operator manages itself, the default and today's
behaviour) or `cert-manager` (the instance gets its own CA, and this is
what covers the external hostname). The field is named for the issuer
rather than for on/off because TLS is served under both — unlike the
similarly spelled field in some other charts, which really is a switch.
- `tls.required` refuses plaintext connections. It is opt-in and
defaults to false: enforcing TLS before the tenant can obtain the CA
would be backwards, and a platform upgrade must not sever connections
that work today.
- Neither issuer is inferred from `external`. Existing instances keep
the CA they have and change nothing on upgrade.
- The tenant no longer receives `<release>-ca-tls` or `<release>-tls` —
the CA private key and the server private key. It receives the
operator's `ca.crt`-only bundle, which works today, and the
ResourceDefinition additionally declares that bundle as the source for
the CA-extraction controller.

### Fixes along the way

- The TLS backup path could never have worked: it passed
`--ssl-mode=VERIFY_IDENTITY`, a MySQL-8 flag the MariaDB client rejects
during argument parsing. Replaced with `--ssl-ca` +
`--ssl-verify-server-cert`, and the test now asserts the dialect rather
than the presence of a substring.
- The leaf certificate carried `clientAuth`. Because `REQUIRE X509`
accepts any certificate signed by a trusted CA, the server's own key was
usable as a client credential. Removed.
- The CA and server private keys survived deletion of the instance:
cert-manager runs with `enableCertificateOwnerRef: false`, so removing
the `Certificate` left the Secrets behind. Both are now cleaned up by
the existing post-delete hook.
- The Secrets are labelled for the operator's watch predicate, without
which a renewed leaf would not reach the pods until a full resync.

### Why the include carries an exclude list

The trust-anchor selector matches by label with no name constraint, so
the enumerable dangerous set is excluded explicitly — both issuers'
private keys, the internal credentials, the backup keys, and the metrics
configuration, which the operator writes unconditionally with
credentials in plain text for a user holding `SELECT` on everything.

The backstop is bounded and the comment says so: matching is by exact
name with no globs, so a Secret introduced by a future operator version,
or a transient object created mid-renewal, stays in scope of the
label-only include. The primary defence remains that the controller
stamps only its own projection.

## Known limitations

1. **Merge is blocked on #3299 — and the blocking is mechanical, not
just procedural.** The CRD on `main` does not carry `spec.caCert`, and
under Flux's Server-Side Apply the API server rejects a field the schema
does not declare (`field not declared in schema`); under the older
client-side apply the same field was silently dropped, so installing
this branch without #3299 fails outright. An earlier revision of this
description claimed the field would be silently pruned; that was wrong.
The tenant still gets a usable trust anchor once merged in order,
through the operator's bundle.
2. **The deprecated backup CronJob cannot start at all**, for reasons
that predate this branch: its `secretKeyRef` names a Secret and key the
chart never renders. That is filed separately as #3347 and deliberately
not fixed here, since repairing it would start scheduled dumps on
production instances from a TLS change. The tests that pin the backup
TLS flags therefore pin a path that does not currently execute — the
suite header says so.
3. **The operator's `clusterName` is hardcoded**, so on a cluster with a
non-default domain the SANs and the name the operator dials diverge.
Pre-existing, raised separately.
4. **No e2e coverage for the managed path.** Unit coverage pins the
rendered chain, the SAN set, the exclusion set and the guards; the
end-to-end handshake is not asserted in CI.

### On review coverage

Two independent review passes were run repeatedly against this branch —
seven rounds, converging from a security finding, through comment
accuracy, to guard strength, and finally to nothing new. A third,
model-based pass reviewed the branch through `6cb1bd236` and its
findings were fixed; it could not be run against the final revision
because of an external account quota lasting until 25 July. The delta
since that revision is substantial and includes runtime code, not only
tests and docs. Recording this so the coverage is not overstated: if the
quota returns before this is ready to merge, that pass is worth running
again.

### Downstream repositories

`packages/apps/mariadb/values.schema.json` gains the `tls` object, so
`cozystack/terraform-provider-cozystack` needs the field modelled for
`cozystack_mariadb`; tracked in
cozystack/terraform-provider-cozystack#40.

### Release note

```release-note
feat(mariadb): chart-managed TLS via cert-manager, opt-in enforcement, and the CA published to tenants as a key-free trust anchor
```
2026-09-04 19:32:16 +04:00
Aleksei Sviridkin 18d02d813a fix(cacert): make the comments match the reconciler (#4064)
## What this PR does

Comments in the CA-extraction package claim guards the code does not
have. No behaviour change; one test is renamed to match the property it
pins.

- `ManagedLabel` said the reconciler only ever writes over a Secret
carrying that label. It does not. `isOurProjection` decides that from
the Secret type and the owner reference, and never looks at the label,
which any writer in the namespace can strip.
- `managedByCozystackLabel` said the reconciler never writes it. It
deletes it whenever the selectors digest changes, which is what gets the
projection re-admitted by the lineage webhook.
- `projectionSuffix` said the dotted canonical name cannot collide with
an operator-owned object. A release the apps API created cannot carry a
dot, since application names are DNS-1035 labels and `release.prefix` is
restricted to `^[a-z0-9-]*$`. The guarantee stops there: the controller
reads the release from the `helm.toolkit.fluxcd.io/name` label, where a
dot is legal, so a hand-written HelmRelease sits outside it. On the
validated path the dot can only come from the suffix an operator or
chart appends, which nothing validates at all, and since a canonical
name ends in `.tenant-ca` and carries that one dot, a suffix ending in
`.tenant-ca` is the only one that reaches a canonical name. The comment
says all of that now.
- The test pinning that name claimed the property for any engine suffix,
in its doc and in its own name, while its loop samples only dot-free
ones. It is now
`TestProjectionNameCannotCollideWithADotFreeEngineSuffix`, and its doc
says what the loop covers.

### Screenshots

Not a UI change.

### Downstream repositories

Comments and one test name in two Go files. No package, values key,
schema field, CRD, metric, release asset or `hack/` target changes, so
nothing in the trigger map fires.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Clarified certificate reconciliation behavior, including label
handling, projection naming, and source certificate scenarios.
* Documented naming guarantees and their API boundaries more precisely.

* **Tests**
* Updated test descriptions and failure messaging to clarify coverage
for dot-free engine suffixes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 19:32:01 +04:00
Aleksei Sviridkin 4b2507b7a1 docs(gateway): align gateway prose with the code
The comment on the shared CEL string said what the hostname requirement
is but not what it is worth per kind. Two of the three schemas leave
spec.hostnames optional at every version the vendored bundle serves,
so for HTTPRoute and GRPCRoute this policy is the only rejection a
hostname-less route meets; tlsroutes requires the field at v1 and
v1alpha3 and not at v1alpha2, so there the policy closes the gap for
as long as v1alpha2 is served. State that where the requirement is
stated.

The comment on the port-80 allowedRoutes filter credited the
cert-manager namespace entry for letting ACME complete, which the
paragraph above it retracts: the solver route lives in the tenant
namespace, and that is what keeps ACME working. The same sentence
claimed the filter keeps app routes off port 80, and the Layer 1
paragraph of the gateway README claimed it too. It does not. A tenant
that owns its Gateway is its own gateway namespace, so its app routes
are admitted by this very filter; what it excludes is cozy-* and
inheriting children. Both copies now say that.

Nothing said why naming the apex on the redirect costs no coverage,
which is the question a reader has once the route stops matching every
host. It holds because of the other half of this change: the port-80
listener admits routes only from the tenant namespace and
cert-manager's, and the tightened route policy requires every route
there to declare hostnames inside the apex. An inheriting child does
not widen that set either, and the comment now says so at the point
the hostnames are chosen: the gateway hostname policy compares every
listener against the host label of the Gateway's own namespace, so a
child apex outside it never gets its listener, the Gateway write
carrying it being refused at admission whole, in reconcileGateway and
before this route is rendered, while a child apex under this one
passes that check and the wildcard already covers it.

The empty-apex guard in renderHTTPRedirect, and the test pinning it,
both read as though the guard is what spares an operator an unreadable
schema error. It is, but the condition stated for it named the number
of attached routes, which does not enter into it. In HTTP-01 the
listener set is built from route hostnames alone, so reconcileGateway
never reads the apex ahead of the guard however many routes attach.
DNS-01 and existingSecret render an apex listener, and any
tlsPassthroughServices entry builds <service>.<apex>, so in those the
operator meets the equivalent error against the Gateway first. State
that condition once, at the guard, and let the test defer to it rather
than carry a second copy that can drift; the test stays green either
way because the fake client does not validate against the CRD schema.

The e2e suite explained omitting the TLSRoute empty-list probe by the
v1 schema rejecting an empty list. Both halves of that are true and
the conclusion does not follow: the probe runs at v1alpha2, which
marks spec.hostnames neither required nor minItems, so an empty list
reaches admission there exactly as an absent field does. What makes
the omission safe is that all three policies render one byte-identical
CEL string, so the HTTPRoute pair proves that term already.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 18:08:02 +03:00
Aleksei Sviridkin 900711567a fix(cozystack-basics): require route hostnames in tenant namespaces
The route hostname policy short-circuited on an absent field. Its CEL
opened with `!has(object.spec.hostnames) ||`, so an HTTPRoute,
TLSRoute or GRPCRoute declaring no hostnames satisfied the first
operand and was
admitted without a single hostname compared against the tenant apex.
An explicitly empty list reached the same outcome by a different path,
because `all()` over an empty list is vacuously true.

A route that names no hostname does not match nothing. Under Gateway
API it inherits the hostnames of every listener it attaches to, so it
covers whatever that listener covers. Require the field to be present
and non-empty before validating it.

All three policies rendered by the template share one CEL string, so
HTTPRoute, TLSRoute and GRPCRoute pick the change up together. The denial message
now names the missing-hostnames case, and the Gateway e2e grep moves to
the part of that message common to every denial it issues.

The Gateway e2e suite carried the one fixture that relied on the old
behaviour: its parentRef probe declared no hostnames and is applied
into the suite's tenant namespace. It now reads the apex from the
namespace label and names a hostname under it, as the neighbouring
cases in that suite already did.

Radius is unchanged: the policy still matches only tenant-* namespaces.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 18:08:02 +03:00
Aleksei Sviridkin b9ecb2e39f fix(tenantgateway): name the apex on the http-to-https redirect route
The controller-owned redirect route carried no spec.hostnames, relying
on the port-80 listener to match every host. Name the tenant apex and
its wildcard instead.

A route in a tenant namespace has to declare hostnames inside its own
apex to pass the route-hostname admission policy, and a route that
names none has nothing to compare against that apex. Naming them also
states the coverage the route wants rather than inheriting whatever its
listener happens to carry.

Both hostnames are required: a wildcard is a suffix match over any
number of labels, so "*.<apex>" covers every subdomain at every depth
but not the bare "<apex>".

Coverage narrows in one direction worth stating: a request arriving on
port 80 with a Host outside the tenant apex now falls through to a 404
instead of a 301.

Skip this route when collecting hostname claims. Before it carried
hostnames it contributed nothing to that set, and letting its hostnames
through instead provisions listeners and certificates for them, which
fails. The check matches on name and controller ownership, not on the
labels the renderer stamps, since those are writable by anyone who can
create a route.

Deriving hostnames from spec.apex makes that field load-bearing for
admission, so it gains MinLength=1 and a matching check in the renderer
for clusters whose CRD is older than the binary.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 18:08:02 +03:00
Aleksei Sviridkin b167fd112a feat(gateway): let a tenant pick its Gateway class, and follow that class for TLS termination (#3861)
## What this PR does

Lets a tenant publish through a GatewayClass whose provider terminates
TLS upstream of the Gateway, and gives the platform the two knobs that
bound the choice. The first consumer is the Cloudflare Tunnel controller
from #3858, which ends TLS at the Cloudflare edge and reaches the
Gateway over the tunnel, but nothing here is tunnel-specific.

Until now every cert mode rendered HTTPS listeners with
`certificateRefs` and tied the TenantGateway's `Ready` to their
issuance. On such a class the platform issued origin certificates nobody
serves, and where ACME could not complete, the tenant's `gateway`
release never became Ready at all.

**Who picks the class.** The platform names a default in
`gateway.className` and the set a tenant may choose from in
`gateway.tenantSelectableClasses`; a tenant picks its own with
`tenant.spec.gatewayClass`. That set is an allowlist rather than free
choice because Tenant is a tenant-writable object while GatewayClass is
cluster-scoped: without it a tenant could attach its Gateway to a class
an admin keeps for internal-only traffic. Naming anything outside the
set fails that tenant's own gateway release with a message listing what
is allowed, and reaches no other tenant.

**Where TLS ends.** That is a property of the class, not of the tenant,
so it is declared once in `gateway.edgeTerminatedClasses`, and whichever
class a tenant lands on decides its `certMode`. The two therefore cannot
be configured out of step: a tenant on a tunnel class cannot keep
issuing origin certificates because a separate switch still says
otherwise.

In `edge` the controller renders the apex, its wildcard and every
inheriting child apex as plain HTTP listeners carrying the same
label-based `allowedRoutes` as the HTTPS listeners of the other modes,
so app HTTPRoutes attach by hostname unchanged. It mints no Issuer, no
Certificate and no http-to-https redirect route, and skips the
TLS-passthrough listeners an HTTP-only edge cannot serve, which leaves
the tenant Kubernetes API, VM export and CDI upload endpoints
unpublished through that Gateway. Those TLSRoutes each pinned a listener
that no longer exists, so the controller flips the acceptance it wrote
for them to `Accepted=False` with reason `NoMatchingParent`. Moving a
tenant into the mode removes the ACME objects it owned, and the
cluster-wide ACME solver drops its `sectionName: http` pin when the
publishing Gateway itself sits on such a class. Pointing
`gateway.className` at an edge class is therefore a supported topology
(the whole platform behind one terminator), and its cost is that those
three endpoints stop being published for the publishing tenant rather
than for one tenant among many. Nothing refuses it, because refusing it
would forbid the topology; `docs/gateway-classes.md` states the cost
where an operator will meet it.

Two consequences worth stating plainly. The wildcard-secret controller
replicated the operator's TLS Secret, private key included, into every
namespace owning a termination point, and a Gateway owner counted as
one; a Gateway on an edge class does not, so that tenant's replica is
pruned while its neighbours keep theirs. And listing a class as
edge-terminated is a claim nothing validates: name the bundled `cilium`
and its Gateways publish every app hostname over plain HTTP on a public
address. A hard `fail` there would forbid a legitimate topology, a
Cilium Gateway behind an external terminator, so it is stated in the
platform values, the chart README and the CRD field description instead.

**Documentation.** `docs/gateway-classes.md` is the operator-facing page
for the whole surface: the three platform values and what each bounds,
what a tenant writes to pick a class, how `certMode` is derived and in
what precedence, what moving a tenant between classes does to its
Issuer, its Certificates, the TLS Secrets those leave behind, the
redirect route and the wildcard replica, and what each way of getting
the names wrong actually looks like: a class name matching nothing
installed renders fine and never programs, a misspelling in
`edgeTerminatedClasses` silently leaves the tenant issuing certificates,
a default moved out from under a tenant that pinned the old name fails
that tenant's release. The chart README keeps the per-mode reference and
gains the one constraint it had not stated: the publishing tenant cannot
select a class that terminates TLS differently from the platform
default, because the cluster-wide ACME issuers derive their solver pin
from that default alone.

Verified on a v1.6.0-rc.2 dev cluster (Talos, k8s 1.36, cilium 1.19.5)
by building the controller from this branch: the CRD enum gained `edge`,
the Gateway came up with `edge` / `edge-apex` / `edge-child-*` HTTP
listeners and nothing else, no Issuer, no Certificate, no redirect
route, and the LoadBalancer Service exposed 80/TCP only. That run also
caught a defect no unit test could. The first implementation kept the
hostname-less `:80` listener at index 0, and on cilium 1.19.5
`CheckGatewayAllowedForNamespace` is first-listener-wins
([cilium#42159](https://github.com/cilium/cilium/issues/42159), fixed in
1.19.6), so an inheriting tenant's HTTPRoute came back
`NotAllowedByListeners` even though a later listener admitted it.
Dropping that listener, which in edge mode carries neither ACME nor the
redirect, flipped the route to `Accepted=True` and removed a catch-all
at the same time.

The live check predates the redesign to per-tenant classes, so it
exercised the mode rather than the selection path; the selection is
covered by unit tests across the four charts and both controllers.
Moving a live Gateway from one class to another is covered nowhere.
`Gateway.spec.gatewayClassName` has no immutability rule in the shipped
CRD, so the apiserver accepts the update; what the outgoing class
controller then does with the LoadBalancer Service it provisioned was
not exercised by the live run and no unit test reaches it.

### Screenshots

Not a UI change.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up: `networking/gateway-api.md` documents the platform
`gateway.*` rows and the cert modes, so `className`,
`tenantSelectableClasses`, `edgeTerminatedClasses` and
`tenant.spec.gatewayClass` need a doc PR there;
`docs/gateway-classes.md` from this PR is what that port starts from.
Not filed yet.
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: the provider hand-writes the Tenant schema, so
`spec.gatewayClass` needs an attribute there; not filed yet.
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

Two boxes stay unticked on purpose: the website and provider follow-ups
are real and not filed yet. I will open them once the shape here settles
in review.

### Release note

```release-note
feat(gateway): tenant Gateways can now run on a GatewayClass other than the bundled Cilium one. The platform sets the default in `gateway.className` and bounds tenant choice with `gateway.tenantSelectableClasses`; a tenant selects its own with `tenant.spec.gatewayClass`. Classes whose provider terminates TLS upstream are listed in `gateway.edgeTerminatedClasses`, and a tenant Gateway on such a class serves plain HTTP listeners, issues no certificates, and no longer receives a replica of the operator wildcard Secret. Such a Gateway also has no TLS-passthrough listeners, so the `api`, `vm-exportproxy` and `cdi-uploadproxy` entries of `publishing.exposedServices` are not published through it, and their TLSRoutes report `Accepted=False` with reason `NoMatchingParent`
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added edge certificate mode with HTTP-only listeners, no certificate
issuance, and no TLS passthrough.
* Added configurable GatewayClass selection for tenant-owned Gateways,
with platform defaults, allowlists, and validation.
* Added platform configuration for edge-terminating and
tenant-selectable GatewayClasses.
* **Bug Fixes**
* Improved wildcard certificate replication for edge-terminated
Gateways.
* Ensured safe cleanup and restoration when switching certificate modes.
* **Documentation**
* Updated Gateway, tenant, and certificate configuration guidance for
edge termination and GatewayClass selection.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 18:56:38 +04:00
Aleksei Sviridkin 8adf3a0863 feat(platform): forward kubevirt.disabledFeatureGates to the kubevirt CR
The kubevirt chart reads disabledFeatureGates from its own values, and
the iaas bundle is the only writer of the cozystack.kubevirt Package, so
nothing could set it: a Package patched by hand is reverted the next
time the platform re-renders it.

Platform values gain a documented kubevirt.disabledFeatureGates list,
forwarded into the emitted Package as
components.kubevirt.values.disabledFeatureGates alongside the entries
derived from resources.cpuAllocationRatio and the gpu block. The
parenthesized value walk is nil-safe, matching the gpu block, so a
cluster that nulls out the whole kubevirt key still renders.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 15:29:09 +03:00
Aleksei Sviridkin c91473fab1 docs(gpu): vGPU SR-IOV support ships with the bundled KubeVirt v1.9.0
Two places also said a hand-crafted Package CR takes precedence over the
bundle render. It does not: the platform chart is the only writer of
those Packages, helm.sh/resource-policy: keep blocks deletion and not
replacement, so the next render puts the bundle's spec back. Both now
say the name has to be in bundles.disabledPackages.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 15:29:09 +03:00
Aleksei Sviridkin 4c3692b0da feat(kubevirt): add disabledFeatureGates and pin the gates the host cannot carry
KubeVirt 1.9 changes what an unlisted Beta gate means. On 1.8
isFeatureGateEnabled ended in `return false`; on 1.9 IsEnabled ends in
`return fg.State == Beta`. Sixteen gates are Beta at 1.9.0 and none of
them is in this chart's featureGates list, so all sixteen arrive on with
the bump. The CR writes three of them back into disabledFeatureGates.

Template deploys the virt-template aggregated API server: two
Deployments, two APIServices, a webhook and a ValidatingAdmissionPolicy
that nothing on the platform uses, and one more aggregated API to break
discovery. Upstream also has spec.configuration.virtTemplateDeployment,
but it reads TemplateEnabled() first, so it stops the deployment and
leaves the gate and its API on; the gate stops both.

ExternalNetResourceInjection makes virt-controller skip
multus.NetworkToResource, so a NetworkAttachmentDefinition's
k8s.v1.cni.cncf.io/resourceName no longer becomes a resource request on
the launcher pod. That is right for a cluster running a
network-resources-injector, which this platform does not deploy: with
the gate on, a VM on an SR-IOV network schedules without its device.

ImageVolume is the one that depends on the host, so it is disabled by
cluster version rather than always. With it on, virt-controller drops
the containerDisk sidecars and mounts the image as a VolumeSource.Image
volume. Kubernetes has its own ImageVolume gate, off by default until
1.35, and an apiserver with it off strips every image volume and the
mounts referencing it — dropImageVolumes in pkg/api/pod/util.go — so the
launcher pod is admitted without the disk it needs and fails looking for
it. KubeVirt has no capability check to notice. The management cluster is
supported from 1.33, so below 1.35 the chart keeps the gate off and the
sidecar path in use, and from 1.35 leaves it alone.

The rest of the Beta set stays on, but a cluster may be unable to carry
a given gate. A disabledFeatureGates list in the chart values appends to
the static opt-outs for that case, and naming a static opt-out again
renders one entry, since the operator writing down what their cluster
declines has no reason to know the chart already disables it.

A gate named in both lists is not a configuration KubeVirt honours, and
it fails differently depending on when it lands. The update validator
refuses the write and runs with failurePolicy Fail, so on a cluster that
already has the CR the HelmRelease stops converging with every other
field in it. The create admitter validates nothing and virt-config reads
featureGates before disabledFeatureGates, so a fresh install is admitted
and the gate stays on while the values file says it is off. The template
stops the render and names the gate instead. The guard reads the same
version-conditional list as the render, so extraFeatureGates may ask for
ImageVolume exactly where the chart is not disabling it. Which list the
colliding gate came from decides what the operator can do about it, so
the message splits three ways: Template is pinned off and cannot be
asked back, a gate the chart enables cannot be switched off from values
at all, and anything else arrived through extraFeatureGates and can
leave from either side.

HotplugVolumes carries a note now. It is deprecated at 1.9 in favour of
DeclarativeHotplugVolumes, but virt-api accepts kubevirt-csi's VMI-level
removevolume on a VM-owned VMI only while the gate is on, so tenant
volume detach depends on the pin, and the pin in turn keeps the newly
default-on DeclarativeHotplugVolumes inert. The update target's header
stops describing the two lists as curated defaults and says to check
them against the pinned release's featuregate registrations at each
bump, since a gate that reached GA is enabled whether the CR names it or
not.

make update grows a marker per line of the new block. Every insert in
that target is guarded by one directive it also inserts, so a hand-merge
that keeps the directive and drops a sibling line leaves the guard
satisfied, the insert skipped and the render green. The three static
entries are the only lines these inserts add that are CR content in
their own right rather than a wrapper around a value: their absence
cannot surface as a value that stopped taking effect, the CR simply goes
back to deploying virt-template, to letting KubeVirt skip NAD resource
injection, and to image volumes an apiserver below 1.35 discards. The
target cannot rebuild a partial block, so it names the missing marker
and stops. The block it does rebuild is compared against the committed
one byte for byte, because the insert text and the template are two
copies of the same lines and only one of them renders. The tail also
greps for the guard header the block's skip reads, which no insert
restores: losing it makes the template unrenderable, and that is the one
hand-merge loss the tail would otherwise wave through.

The suite states the cluster version instead of inheriting the test
plugin's default, which is unstated and below 1.35, and pins both sides
of the boundary.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 15:29:08 +03:00
Aleksei Sviridkin 3e370e79a0 chore(kubevirt): update KubeVirt to v1.9.0
Refetches the vendored operator manifest from the v1.9.0 release. The
release turns every Beta feature gate on by default and adds
disabledFeatureGates for opting out; the KubeVirt CR side of that lands
separately.

The virt-launcher QEMU build moves with the bump — el9 qemu-kvm
10.1.0-10 to 10.1.0-20 — and that is the thing to check before moving
the release pin, rather than a QEMU major: kubevirt/kubevirt#16386 lost
live migration for every running VM on 1.6.2 to 1.7.0, one build apart
within the same QEMU version, on virtio feature bits the target
launcher would not accept. VMs running on the 1.8.4 launcher were
migrated onto the 1.9.0 one on a three-node test cluster before this
bump. The update target now carries the check as a note above the pin.

make update used to wipe templates/ before fetching, which also removed
the local PrometheusRule template; it now only overwrites the file it
fetches.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 15:29:04 +03:00
myasnikovdaniil 7b0c35b9b3 feat(monitoring): add KubeVirt VM dashboard (#2198)
## Summary

- Adds `dashboards/vm/virtual-machines.json` — KubeVirt VM resource
metrics (CPU, memory, network, storage) with namespace and VM template
variables
- Adds `dashboards/vm/console-logs.json` — VM serial console log viewer
via VictoriaLogs with namespace filtering
- Adds `vm/virtual-machines` and `vm/console-logs` entries to
`dashboards.list`

## Problem

`dashboards.list` references `vm/virtual-machines` and `vm/console-logs`
but the corresponding JSON files don't exist.

The console-logs dashboard includes `kubernetes_namespace_name`
filtering to prevent cross-tenant log exposure.

Relates to #2194

## Test plan

- [ ] Verify JSON is valid for both dashboards
- [ ] Import dashboards in Grafana and verify panels render with
KubeVirt metrics
- [ ] Verify console-logs queries include namespace filter

```release-note
Add KubeVirt VM dashboards (virtual-machines and console-logs)
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* **VM Console Logs dashboard**: View live console output with namespace
and VM filtering, fast refresh, and improved log presentation (details,
wrapping, and sorting).
* **Virtual Machines dashboard**: Rich VM monitoring covering CPU,
memory, network, storage I/O, latency, swap/page faults, and filesystem
usage with thresholds and formatted visuals.
* **Dashboard catalog updated**: Added multiple new monitoring
dashboards for broader coverage and easier discovery, including the new
VM dashboards.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 16:59:24 +05:00
Aleksei Sviridkin 773d8dabce fix(gateway): withdraw TLSRoute acceptance when the mode drops passthrough
Edge terminates TLS at the class provider, so its Gateway carries plain
HTTP listeners only and a route pinned to a passthrough section has
nothing left to attach to. The api, vm-exportproxy and cdi-uploadproxy
endpoints of publishing.exposedServices are the ones that pin one.
Hostname claims are empty for every whole-apex mode, so the status pass
ran over an empty ref set and left the Accepted=True written while the
tenant ran HTTP-01 standing on a route the switch had just unpublished.
Whoever went looking for the dead endpoint read that condition as proof
the route was fine.

The rule this settles on is that the controller owns a condition only
where it judges. Edge judges every attached route by the absence of the
listener it needs and writes Accepted=False with Gateway API's
NoMatchingParent, the reason defined for a parentRef whose port or
sectionName matches no listener. The other whole-apex modes render
those listeners and serve them but collect no claims, so they hold no
opinion: they drop the entry rather than leave a withdrawal standing on
a listener that is serving, naming a class provider that no longer
terminates anything. HTTP-01 collects claims and has already judged
every route this pass could speak about.

A route declaring no hostnames is judged by none of them, because
claims are gathered inside a loop over spec.hostnames and TLSRoute
v1alpha2 sets no minItems on that field. Withdrawing one would write a
condition with no path back, since HTTP-01 renders the listener again
and judges only the routes that produced claims, so both directions
skip it.

The tests read the controller name from the constant the controller
stamps rather than restating it, so a helper cannot go looking for a
name that is no longer written and report absence it never checked.

The switch already deletes the redirect route, the Issuer and the
wildcard Certificate it invalidates. The statuses it owns were the one
thing left behind.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 14:33:36 +03:00
Aleksei Sviridkin 5976ddf02f fix(gateway): normalise the GatewayClass name and refuse a blank one
The class lists arrive over the values channel comma-joined, so every
reader trims each entry as it splits them. The scalar those entries are
compared against was not trimmed anywhere. A class name carrying a
stray space therefore failed `has` against its own padded self, and the
two readers of that scalar disagreed about the same class: the gateway
chart put the tenant on an edge-terminated class, while the
cluster-wide ACME issuers still pinned their solver to a listener an
edge Gateway does not render. That stops every HTTP-01 certificate in
the cluster from issuing, with nothing in any status naming the cause.

Normalise once at the single writer so both readers see a clean value,
and keep a defensive trim at each reader. The tenant's own choice does
not travel that channel — it comes from tenant.spec.gatewayClass — so
it is trimmed where the gateway chart reads it.

Unset and whitespace-only are different events and no longer share an
outcome. Unset means the choice was not made, and falling back to the
bundled class stays its documented behaviour. Whitespace-only means a
choice was made and is garbage, in the field that decides where TLS
terminates: resolving it quietly would take a tenant the operator meant
to put behind an edge terminator and publish it directly, with a real
certificate on a real address. Both the platform default and the
tenant's own choice now refuse it by name.

The kind check beside that refusal is part of the same fix, not a tidy
up. Through Helm's values path an unquoted numeric scalar arrives as a
float64, and the whitespace check compares it against a string, so
`className: 123` dies at that comparison — one line before the trim —
with an error naming neither the key nor the file. Refusing whitespace
without checking the kind first would therefore have made the numeric
case worse than it was. `kindIs "string"` is the right test because it
never has to name the type that actually arrived.

The order of the guard's steps is part of the guard. The value is read
before anything normalises it, so a default that fires on any falsy
value cannot launder 0 or false into the empty string and resolve them
to the bundled class ahead of the kind check. Unset is the one case
that still falls back, and each of those four inputs is pinned, so a
normalisation drifting back ahead of a check fails rather than quietly
covering only the loudest bad input.

Also correct a cleanup log line: the wildcard Certificate is deleted
for every mode that is not DNS-01, so naming HTTP-01 in that message is
wrong for existingSecret and for edge alike. And state the listener
name invariant plainly: the cap is 253, not the 63 of a DNS label, and
the longest name the builders produce is 84.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-04 14:33:35 +03:00
Aleksei Sviridkin 7851969448 docs(gateway): document GatewayClass selection and edge TLS termination
An operator setting up a class whose provider terminates TLS upstream
had no page answering their questions: which platform values bound the
choice, what a tenant writes to make it, what a move between classes
does to certificates and to the replicated wildcard Secret, and how the
mechanism fails when a configured class name matches nothing installed.

Add docs/gateway-classes.md covering that surface. Describe the mode
and its consequences in the chart README alongside the other three,
including the one constraint stated nowhere else: the publishing tenant
cannot select a class whose TLS termination differs from the platform
default, because the cluster-wide ACME issuers derive their solver pin
from that default alone.

State the blast radius where it actually applies. The endpoints an edge
Gateway stops publishing exist only where those routes are rendered at
all, and both pages say what the transition does to the route
conditions the controller owns, so a reader is not left inferring it
from the code.

The security model names the selector each listener really carries. The
port-80 listener selects on the built-in kubernetes.io/metadata.name
label, which kube-apiserver writes and a namespace cannot forge; the
HTTPS and TLS-passthrough listeners select on
namespace.cozystack.io/gateway, written by the tenant chart and the
controller, so that guarantee does not extend to them. What closes the
exposure is that Namespace is cluster-scoped with no tenant verb on it,
and that the route-hostname policies bound what a labelled namespace
could publish.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-04 14:33:29 +03:00
myasnikovdaniil 1b2beac3f8 [cilium] Set resources requests/limits on cilium-operator (#2618)
## Summary

Companion to #2617 (cilium-agent). The upstream cilium chart ships with
`operator.resources: {}`, which triggers a `no CPU limit` conformance
warning on the `cilium-operator` Deployment.

Override `packages/system/cilium/values.yaml` to set the sizing
recommended by the upstream `values.yaml` comment:

- `limits` : `cpu: 1000m`, `memory: 1Gi`
- `requests` : `cpu: 100m`, `memory: 128Mi`

## Test plan

- [ ] `helm template . --namespace kube-system` from
`packages/system/cilium/` renders a `resources` block on the
`cilium-operator` container of the Deployment (verified locally)
- [ ] Re-run the conformance/lint tool — the `cilium-operator has no CPU
limit` warning should be gone
- [ ] CI passes

### Release note

\`\`\`release-note
Set CPU/memory requests and limits on the cilium-operator Deployment to
clear the "no CPU limit" conformance warning. Values follow the upstream
chart's recommended sizing (limits 1000m/1Gi, requests 100m/128Mi).
\`\`\`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated Cilium system component resource configuration with CPU and
memory requests and limits.

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2618)

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 16:30:49 +05:00
Aleksei Sviridkin 487cafac3e feat(platform): let a tenant pick its Gateway class from an allowlist
Tenant Gateways were pinned to the bundled Cilium class. Give the
platform a default in gateway.className, the set a tenant may choose
from in gateway.tenantSelectableClasses, and the set whose provider
terminates TLS upstream of the Gateway in
gateway.edgeTerminatedClasses. A tenant picks its own with
tenant.spec.gatewayClass, read only when it owns a Gateway; an
inheriting tenant runs on its ancestor's.

The selectable set is an allowlist rather than free choice because
Tenant is a tenant-writable object while a GatewayClass is
cluster-scoped: without it a tenant could attach its Gateway to a class
an admin keeps for internal-only traffic. Naming anything outside the
set fails that tenant's own gateway release and reaches no other
tenant.

Where TLS ends is a property of the class, so the gateway chart derives
certMode from whichever class the tenant landed on instead of taking it
as a separate input. The two therefore cannot be configured out of
step.

The publishing tenant cannot cross that line by itself. The
cluster-wide ACME issuers point their HTTP-01 solver at its Gateway and
are rendered from the platform default alone, so a mismatch would pin
the solver to a listener that Gateway does not have. Those issuers now
drop the sectionName pin when the default class is edge-terminated,
letting the challenge attach by hostname instead.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-04 14:26:54 +03:00
Aleksei Sviridkin 57c51f58ee feat(gateway): stop replicating the wildcard key into edge namespaces
The wildcard-secret controller replicates the operator's TLS Secret,
private key included, into every namespace that owns a termination
point, and a namespace owning a Gateway counted as one without further
question. A Gateway whose class ends TLS at its provider's edge
terminates nothing locally, so the replica has no consumer there.

Read certMode off the tenant's own TenantGateway rather than a
cluster-wide value, because the class is a per-tenant choice. Keep the
key whenever the answer is not a confident no: a namespace that also
owns an ingress controller still terminates locally, a namespace
holding more than one TenantGateway keeps it while any of them is not
at the edge, and an unreadable list or a namespace whose TenantGateway
has not appeared yet keeps it too, so a transient API error never
withdraws a key a Gateway may still be serving.

Watch TenantGateway, so the withdrawal follows the class change rather
than waiting for the periodic resync. The watched set is a value the
tests assert on, because the controller-runtime builder exposes nothing
about what it watched: a dropped watch would otherwise leave both
suites green while the withdrawal quietly degraded to that resync,
leaving the private key in a namespace that stopped terminating TLS.

Pruning the replica is not the same as leaving the namespace key-free.
The Certificate objects go, but the Secrets they issued stay, as they
do on any other mode transition.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-04 14:26:54 +03:00
Aleksei Sviridkin 2ef0e8d3a1 feat(gateway): add an edge cert mode for upstream-terminated classes
Some GatewayClasses have their provider terminate TLS in front of the
Gateway and reach it over plain HTTP. On such a class the platform
issued origin certificates nobody serves, and where ACME could not
complete, the Gateway's listeners never became ready.

Add certMode=edge to TenantGateway. The controller renders the apex,
its wildcard and every inheriting child apex as plain HTTP listeners
carrying the same label-based allowedRoutes as the HTTPS listeners of
the other modes, so application HTTPRoutes attach by hostname
unchanged. It mints no Issuer and no Certificate, and removes the
http-to-https redirect route, because the edge performs that redirect
itself. TLSPassthroughServices are not rendered: an HTTP-only edge
cannot carry a TLS listener, which leaves the endpoints relying on one
unpublished through such a Gateway.

The hostname-less port-80 listener is left out. It carries the ACME
challenge and the redirect in the other modes and has neither here, so
it would only be a catch-all admitting any hostname. Leaving it out
also keeps a hostname-pinned listener at index 0, which is what
Cilium's first-listener-wins namespace check reads before the fix in
cilium/cilium#45693.

Moving into the mode garbage-collects the Issuer and Certificate
objects the controller owned. The TLS Secrets they issued stay behind:
cert-manager runs with enableCertificateOwnerRef false, so a
Certificate does not own the Secret it issued into. Every mode
transition already behaves that way.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-04 14:26:54 +03:00
Aleksei Sviridkin a2cb47c961 feat(gateway): update the CRD bundle to Gateway API v1.6.1 and gate GRPCRoute hostnames (#3872)
## What this PR does

Bumps the vendored Gateway API CRD bundle in
`packages/system/gateway-api-crds` from v1.5.1 to v1.6.1. The package
Makefile pins the upstream ref, so the change is that pin plus the
re-rendered `templates/crds-experimental.yaml`.

The bundle version is not only bookkeeping: it is a compatibility signal
consumers read at runtime. Every CRD in the bundle carries a
`gateway.networking.k8s.io/bundle-version` annotation, and a Gateway API
controller can read it off the `gatewayclasses` CRD to decide whether
the installed bundle matches the one it was built against. Controllers
that do this typically accept any patch release but require an exact
major and minor match, and report `SupportedVersion=False` with reason
`UnsupportedVersion` on their GatewayClass when the minor differs, a
status condition that surfaces the skew rather than a functional break.
The Cloudflare Tunnel Gateway API controller proposed for the platform
in #3858 is built against Gateway API v1.6.1 and performs exactly that
comparison, so against the v1.5.1 bundle shipped today its GatewayClass
reports the bundle unsupported, and with this bump it reports supported.
Neither change depends on the other landing first: this PR only moves
the bundle, and the check lives entirely on the consumer side.

One consequence worth stating plainly, since it will outlive this PR: an
exact-minor check makes the shipped bundle version a coupled pin for any
such controller. A future bump to v1.7.x flips the same condition back
to `UnsupportedVersion` until that controller is rebuilt against the
newer bundle, so the two want to move together.

The bundle still emits the same twelve CustomResourceDefinitions and the
same two `safe-upgrades.gateway.networking.k8s.io` admission objects,
and every CRD still carries the `helm.sh/resource-policy: keep` stamp
the Makefile's post-processing step adds. One of the policy's two CEL
rules is rewritten, though, and it changes what a cluster accepts. That
has its own section below.

Two kinds change their version surface: TCPRoute and UDPRoute gain a
served `v1` that takes over as the storage version, and their `v1alpha2`
stays served with a deprecation marker. Every other kind keeps exactly
the versions it served at v1.5.1. That matters because the Cilium
operator resolves GatewayClass, Gateway, HTTPRoute and GRPCRoute in
`v1`, ReferenceGrant in `v1beta1` and TLSRoute in `v1alpha2` before it
enables its Gateway API controller; all six are still served by the
v1.6.1 bundle. TLSRoute in particular keeps `v1alpha2`, which the tenant
gateway controller, the platform TLSRoute templates and the tenant
hostname admission policy all address.

Schema changes are small and land on fields nothing in this repository
sets. `sessionPersistence.idleTimeout` is gone from HTTPRoute, GRPCRoute
and XBackendTrafficPolicy, removed upstream under GEP-1619 because no
implementation ever implemented it and HTTP cookies have no native
idle-timeout mechanism, so no working behaviour is lost even though the
field disappears from the schema; HTTPRoute gains a lower bound on
`retry.attempts` and now rejects duplicate `retry.codes`; ReferenceGrant
requires `spec` at the root. In the other direction, Gateway
infrastructure annotations and frontend CA certificate references go
from a limit of 8 to 16, and TLSRoute hostnames from 16 to 1024. No CRD
schema loses or changes a CEL validation rule, and the six added belong
exclusively to the new TCPRoute and UDPRoute `v1` schemas.

### The safe-upgrades admission policy accepts a different set of
bundles

The experimental kustomization ships a `ValidatingAdmissionPolicy` and
its binding, both named `safe-upgrades.gateway.networking.k8s.io`, next
to the twelve CRDs. The binding is `validationActions: [Deny]` with
`failurePolicy: Fail`, matching CREATE and UPDATE on
`apiextensions.k8s.io/v1 customresourcedefinitions` cluster-wide, and
Cozystack installs it in two places: on the host through
`cozystack.gateway-api-crds`, and inside every guest cluster that sets
`addons.gatewayAPI.enabled`.

Its second rule reads the `gateway.networking.k8s.io/bundle-version`
annotation off an incoming CRD. At v1.5.1 that rule was a blacklist:
reject anything matching `v1.[0-4].\d+` or containing `v0`, allow the
rest. At v1.6.1 it is a whitelist: allow `v0.0.0-dev` outright,
otherwise require the annotation to start with `v1.` and not match
`^v1\.[0-4](\.|$)`. Evaluating both expressions over a spread of
annotation values:

| bundle-version | v1.5.1 policy | v1.6.1 policy |
| --- | --- | --- |
| `v1.6.1`, `v1.5.1`, `v1.5.0-dev`, `v1.10.0` | allow | allow |
| `v1.4.0`, `v1.4.10`, `v1.0.0` | reject | reject |
| `v0.0.0-dev` | reject | **allow** |
| `v2.0.0` | allow | **reject** |
| `1.6.1` (no leading `v`) | allow | **reject** |
| `v1.4` (no patch) | allow | **reject** |
| `main`, empty string | allow | **reject** |

Nothing in this repository trips the new rejections: every CRD the
bundle ships is stamped `v1.6.1`. What is exposed is an operator
applying a Gateway API CRD set from somewhere else on a cluster that
carries the policy. A fork, a vendor build or a nightly whose annotation
reads `1.6.1` or `v2.0.0` used to go in and is now denied with `reason:
Invalid`, naming a policy the operator may not know is installed, and
whatever HelmRelease owns those CRDs then sits un-Ready. Uninstalling
the policy is the escape hatch, and the denial message says so.

The same hunk fixes an upstream inconsistency: at v1.5.1 the policy pair
was stamped `v1.5.0-dev` while its twelve CRDs said `v1.5.1`. At v1.6.1
all fourteen documents agree.

The Gateway and HTTPRoute objects the tenant gateway controller renders
were serialized across all three certificate modes and checked against
both the v1.5.1 and the v1.6.1 schemas; the two runs are identical, so
the bump does not change what the controller can create. `go.mod` still
pins `sigs.k8s.io/gateway-api` v1.4.1 and this PR deliberately leaves it
there. The CRD bundle and the Go client types have been on separate
versions since before this change, and nothing the controller emits
depends on the gap.

One thing to know before the next bump: from v1.6 on, the two things
upstream calls the experimental bundle disagree. This package follows
`config/crd/experimental`, the kustomization, which emits 12 CRDs at
v1.6.1; the published `experimental-install.yaml` for the same tag emits
13, the extra one being `xbackends.gateway.networking.x-k8s.io`, new in
v1.6 and not listed in the kustomization. At v1.5.1 the two agreed, so
there was nothing to choose between. Nothing in this repository
references XBackend and no vendored CRD depends on it, so following the
kustomization costs nothing today, but whoever bumps next inherits the
choice, and it should be made deliberately rather than by whichever
command gets typed.

The second commit corrects a comment this bump invalidated. It claimed
TLSRoute stays at `v1alpha2` through Gateway API v1.5 and that the
promotion to `v1` was still upstream-only; TLSRoute has been served at
`v1` since v1.5.0, so the claim was already wrong before this bump and
the version range it named is now wrong too.

The third commit closes a gap that correction exposed.
`cozystack-route-hostname-policy-tls` named `v1alpha2` alone while the
bundle serves `tlsroutes` at `v1`, `v1alpha2` and `v1alpha3`: three
served versions against one named. The gap is not new and this bump
neither creates nor widens it: the v1.5.1 bundle and the v1.6.1 one
serve the same three versions, with the same two marked deprecated.
Coverage of the two unnamed versions rested entirely on request
conversion, because `matchPolicy` is unset on both rules in that file
and so defaults to `Equivalent`, which rewrites a request submitted
under any served version into the one the rule names before the policy
sees it. That is why there is no bypass to demonstrate, and equally why
the arrangement is brittle: the conversion has nothing left to convert
into once no named version is served, and a rule that selects nothing is
not caught by `failurePolicy: Fail`, because nothing failed. The rule
now names all three, which is the shape the HTTPRoute rule above it
already used, and `hack/route-hostname-policy-version-coverage.bats`
reads the served versions out of the vendored bundle and fails when a
policy does not cover every one of them, so a future bundle move cannot
outrun the list silently, in either direction.

That guard carries negative fixtures rather than only watching a correct
tree pass. Three of them drive `assert_covers` itself, with the two
readers redefined in a subshell, against an omitted served version, an
empty served set and an empty named set. Each requires the specific
complaint text rather than merely a non-zero exit, so a helper renamed
out from under it fails for the right reason instead of reading as a
successful rejection. A fourth drives the comparison helper directly.

The fourth commit pins the vendored bundle version in the CRD package's
unit suite. Nothing outside the package Makefile named a Gateway API
version, so re-running `make update` against a different ref landed
green: twelve CRDs, every one stamped, document count unchanged, suite
passing. The new case reads the
`gateway.networking.k8s.io/bundle-version` annotation off every rendered
CRD and fails when it is not the version the Makefile pins.

The fifth commit extends the version-coverage guard to
`cozystack-gateway-hostname-policy`, which names the `gateways` versions
literally the same way the two route policies do, and whose version set
lives in the vendored bundle and moves on `make update` there. It
matched the served set already; the guard now says so on every run
instead of leaving one of the three hostname policies unpinned.

Two notes for whoever reads the release: the TCPRoute and UDPRoute
storage version moves from `v1alpha2` to `v1`, and while nothing in this
repository ships either kind and the CRDs declare no conversion block,
an upgraded cluster accumulates both entries in `status.storedVersions`
— inert today, load-bearing only if a later bundle stops serving
`v1alpha2`.

The sixth commit gives GRPCRoute the same hostname rule the other two
route kinds have. GRPCRoute carries `spec.hostnames` and it can reach a
tenant Gateway: the port-443 listeners pin `allowedRoutes.kinds` to
HTTPRoute and TLSRoute, but the port-80 listener leaves `kinds` unset,
and an unset list resolves to the kinds the listener protocol supports,
which for HTTP includes GRPCRoute. A GRPCRoute in a tenant namespace
could claim any hostname on the plaintext listener with nothing checking
it. Tenants hold no `gateway.networking.k8s.io` RBAC in Cozystack, so
this is the same defence-in-depth against a chart bug that the other two
rules are, not a tenant-user control. The rule is the HTTPRoute rule
with `grpcroutes` in place of `httproutes` and the one version the
bundle serves for it, and the coverage guard pins that list to the
bundle like the others.

Several places in the tree enumerate which route kinds the hostname
policies cover, and all of them named two: the port-443 kinds invariant
in the tenant gateway controller and the test that pins it,
`packages/extra/gateway/README.md`, and the threat model together with
the self-assessment. They name three now. The port-443 kinds set also
carries an accurate reason now. cilium#45559 governs only that every
port-443 listener carries the same set, not what is in it, so the
contents are a least-privilege choice: nothing the platform ships needs
gRPC, TCP or UDP routing on port 443. The README had also described
HTTPS and TLS-passthrough listeners as carrying different kinds, where
the controller gives both the same one.

One more commit makes the suite's existing assertions bite. A
ValidatingAdmissionPolicyBinding whose `policyName` points at another
policy renders without complaint and leaves its own policy inert, and
the suite pinned `validationActions` on the bindings while never pinning
`policyName` at all, so pointing the GRPCRoute binding at the HTTPRoute
policy kept every case green with the GRPCRoute control doing nothing.
All three bindings now pin the policy they name. In the same file,
`uncovered()` assigned to the variable names its caller reads back when
printing a mismatch, so a call site with the arguments swapped would
have printed a diagnostic naming the wrong side.

Two things are named rather than made. The version-coverage guard
collects `apiVersions` from every `resourceRules` entry mentioning the
resource without also matching on `apiGroups` or `operations`, so a rule
in another group, or one scoped to a single verb, would donate its
versions to the set coverage is measured against; the chart's own suite
pins the group and the resource on the rule it checks, which closes that
a layer below. And
`packages/system/cozystack-api/templates/api-tlsroute.yaml` renders
`gateway.networking.k8s.io/v1alpha2` while the bundle serves `v1` as
storage and marks `v1alpha2` deprecated, but the identical pin sits in
the kubevirt and kubevirt-cdi TLSRoute templates, and the pinned
`sigs.k8s.io/gateway-api` carries no `apis/v1` TLSRoute type for the
controller side to follow, so moving one of the three here would leave
the tree less consistent than leaving all three.

Closes #3789.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file: neither `packages/system/gateway-api-crds/Makefile`
nor its rendered CRD template matches any entry. Leaving every box empty
rather than ticking the "no downstream repository" line, because the
docs site carries a Gateway API version statement that is already
inaccurate and needs a decision that is not mine:
`content/en/docs/next/networking/gateway-api.md` and its v1.5 and v1.6
copies say Gateway API v1.5 ships TLSRoute at `v1alpha2` and that the
graduation to `v1` is still upstream-only, when v1.5.0 already shipped
TLSRoute `v1` as the storage version.

### Release note

```release-note
chore(gateway-api-crds): update the vendored Gateway API CRD bundle to v1.6.1. TCPRoute and UDPRoute gain a served v1 that becomes their storage version, with v1alpha2 still served and now deprecated; every other kind keeps the versions it served at v1.5.1. Upstream removed `sessionPersistence.idleTimeout` from HTTPRoute, GRPCRoute and XBackendTrafficPolicy; the field is not set by anything Cozystack renders, but a hand-written route that carries it loses it silently on its next write, with no admission error.

The bundle's `safe-upgrades.gateway.networking.k8s.io` ValidatingAdmissionPolicy changes which CRDs it accepts. Cozystack installs it cluster-wide on the host and in every guest cluster with `addons.gatewayAPI.enabled`. Its `gateway.networking.k8s.io/bundle-version` rule was a blacklist on two patterns and is now a whitelist on the `v1.` prefix with a `v0.0.0-dev` escape hatch. A bundle annotated `v0.0.0-dev` is now accepted where it was denied. A bundle annotated `v2.0.0`, `1.6.1` without the leading `v`, `main`, `v1.4` without a patch, or carrying an empty annotation is now denied where it was accepted. Nothing Cozystack ships is affected, since every CRD in the bundle is stamped `v1.6.1`, but applying a Gateway API CRD set from a fork, a vendor build or a nightly on such a cluster can now fail admission with `reason: Invalid`. Uninstalling the policy remains the documented escape hatch.

fix(cozystack-basics): the tenant TLSRoute hostname admission policy now names every served TLSRoute version (`v1`, `v1alpha2`, `v1alpha3`) instead of `v1alpha2` alone. Coverage of the other two rested on request conversion, which holds only while a named version stays served, and a rule left selecting nothing is not caught by `failurePolicy: Fail`.

feat(cozystack-basics): GRPCRoute hostnames in `tenant-*` namespaces are now gated by the same admission policy that gates HTTPRoute and TLSRoute. A tenant Gateway's port-80 listener leaves `allowedRoutes.kinds` unset, which admits GRPCRoute by protocol default, so a GRPCRoute could claim any hostname on the plaintext listener unchecked. Tenants hold no `gateway.networking.k8s.io` RBAC, so this is defence-in-depth against a chart bug rather than a tenant-user control.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
  - Added hostname admission validation for `GRPCRoute` resources.
- Expanded `TLSRoute` hostname validation to support all served Gateway
API versions.
- Ensured route hostname policies consistently cover HTTP, TLS, and gRPC
routes.

- **Security**
- Updated listener policies to restrict port-443 routes to supported
HTTP and TLS route types.
- Clarified fail-closed hostname validation behavior across Gateway,
Route, and Ingress resources.

- **Documentation**
- Updated security and threat-model documentation to include GRPCRoute
handling and validation coverage.

- **Maintenance**
  - Updated the bundled Gateway API CRDs to version 1.6.1.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 15:23:36 +04:00
myasnikovdaniil b7e5100e73 fix(monitoring): strip infrastructure metadata from tenant logs (#2196)
## Summary

- Adds a `modify` filter in fluent-bit that removes
`kubernetes_node_name` and `kubernetes_host` from all container logs
- Prevents infrastructure topology (node names, hosts) from leaking to
tenant users

## Problem

`kubernetes_node_name` is exposed in log metadata, leaking node topology
to end-users. Other sensitive fields like `kubernetes_host` are also
visible.

Relates to #2194

## Test plan

- [ ] `helm template` monitoring-agents and verify `Remove
kubernetes_node_name` filter is present
- [ ] Deploy and verify `kubernetes_node_name` is absent from logs in
tenant Grafana

```release-note
Strip infrastructure metadata (node name, host) from tenant logs
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Improved Kubernetes monitoring log processing by filtering out most
`kubernetes_*` fields for `kube.*` events, while preserving the key
Kubernetes identity fields (such as pod, namespace, and
container-related names) for clearer, more consistent log output.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 16:16:24 +05:00
Aleksei Sviridkin 2d6a18888d docs(rabbitmq): name caSecretName for what the operator does with it
The README and the TLS suite called spec.tls.caSecretName the inter-node
mTLS knob and said the chart leaves it unwired. The field has nothing to
do with Erlang distribution: under it the operator writes
ssl_options.cacertfile, ssl_options.verify = verify_peer, and a cacertfile
for the management and metrics listeners, every one of them client-facing.
The cluster-operator this platform ships carries no distribution TLS under
any field, so there was no inter-node path to leave unwired either.

Say what the field does at each place it is named, and give the
client-verification section the operator field it was missing, along with
the reason verify_peer on its own does not make a client certificate
mandatory.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 12:14:48 +03:00
Aleksei Sviridkin a1f512664a fix(rabbitmq): shorten the refusals that land in a release condition
Both messages ran 700-800 characters. They surface in a HelmRelease
condition read through kubectl describe, which truncates, so the part an
operator needs was the part most likely to be cut. Keep the objects that
block, the two-upgrade order, the command that confirms the namespace is
clear and a pointer to the README section; the mechanism is already there
in full.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:48:24 +03:00
Aleksei Sviridkin 2d2f2d1ecb docs(rabbitmq): argue the opt-in default from timing, not availability
The paragraph closed by saying TLS moves clients before there is a
key-free way to hand them the CA. This chart ships that way, and the
section two headings down documents it. The reason the default holds is
that the move lands the instant the flag is set, before any client has
had a chance to pick the trust anchor up.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:48:16 +03:00
Aleksei Sviridkin 824fbefeef docs(rabbitmq): drop the Role header the rule below already carries
The file header and the block at the secrets rule both said no TLS Secret
is granted in either TLS state. The one at the rule is where a reader
editing the grant will be looking.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:48:15 +03:00
Aleksei Sviridkin 13a85af3e4 test(rabbitmq): assert the tenant grants as whitelists
The checks grepped the ResourceDefinition source for forbidden names, a
shape the nats sibling documents as broken: quoting the scalar, spelling
the name through a helper, or reaching the Secret through a label all
leave the grep green. Adding a quoted rabbitmq-{{ .name }}-tls to the
include passed every test.

Compare whole values instead, the way the converged siblings do: the
include and exclude lists as parsed YAML, and the dashboard Role's rules
and kinds against rendered output, so the spelling used in the template
cannot smuggle a name past. The Role's secrets rule is the one part that
grows from values, so a declared user gets its own case.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:48:15 +03:00
Aleksei Sviridkin 9e6465d093 fix(rabbitmq): scope the topology-object refusal to this release
A tenant namespace holds every RabbitMQ application the tenant runs, and
the listing enumerated all of them. A second instance with users kept the
first from ever closing its listeners, and the refusal named the other
application's objects and told the operator to delete them. The mechanism
behind the refusal does not reach that far either: a sibling's topology
objects are reconciled through their own broker, whose listeners this flag
leaves alone.

Read spec.rabbitmqClusterReference.name and keep only the objects that
name this release. The hand-written fixture gained the reference it was
missing, which is what hid the gap, and a sibling release now has a case
of its own.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:45:04 +03:00
Aleksei Sviridkin e521c3b8c7 test(rabbitmq): isolate the external gate on the platform-host SAN
Every existing case with external: false also left _namespace.host unset,
so the platform host stayed out of the SAN list because no host was
supplied, not because the gate held. Replacing the gate with a constant
left the suite green. Supply a host with external: false so the gate is
the only thing deciding.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:28:37 +03:00
Aleksei Sviridkin fcab5dbb07 docs(rabbitmq): carry the TLS consequences to where the toggle is rendered
cozyvalues-gen propagates the @field line and nothing under it, so the
description reaching values.schema.json, the README table and the
dashboard form said only that TLS turns on, while the continuation lines
holding the breaking part of the change stayed in values.yaml. Fold the
listener, binding-port and metrics-port moves into the @field line, and
leave the continuation with the part that is genuinely about the struct.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:28:04 +03:00
Aleksei Sviridkin af003fd520 docs(rabbitmq-rd): describe the trust-anchor include by its real mechanism
Both the cozyrds comment and the bats header said the chart labels its CA
Secret for publication. Nothing labels that Secret: the chart names it in
a TenantProjection sentinel, and the CA-extraction controller stamps
internal.cozystack.io/tenant-ca on the key-free projection it writes. A
reader who went looking for the missing label on the CA Certificate would
have found nothing, and the obvious repair — a secretTemplate stamping the
label onto the CA Secret — would offer a tenant the CA private key.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:27:25 +03:00
Aleksei Sviridkin 0d8a5c7941 docs(rabbitmq): keep the Certificate rationale out of the applied object
Two comment blocks in certmanager.yaml used `#`, which the renderer does
not strip, so fourteen lines of prose about the sentinel wiring and the CA
rotation policy were copied verbatim into every Certificate the chart
applies. The rest of the file already explains itself with Go-template
comments; use those here too, and cut the parts the README carries.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:26:32 +03:00
Aleksei Sviridkin fd9b96b2c5 fix(rabbitmq): refuse to close the plaintext listeners in one upgrade
Clearing users and vhosts and setting tls.disableNonTLSListeners in the
same edit rendered cleanly, because the existing refusal reads values and
both were already empty by then. Helm applies every object in the target
release before pruning the ones the release dropped, so the broker closed
plaintext and lost the operator-connection-uri annotation while the User,
Vhost and Permission objects still held their deletion finalizers.
messaging-topology-operator then had to reach the management API over a
certificate it verifies against the system trust pool, which cannot hold
a per-release CA, so the delete never completed and the upgrade reported
success over objects left in Terminating.

Read the topology objects actually present in the namespace instead, and
only while the flag is being introduced, so a release that already closed
its listeners keeps rendering and an object created afterwards does not
block it. Both refusals and the README now carry the two-upgrade order,
the command that confirms the namespace is clear, and the way back out.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:24:51 +03:00
Aleksei Sviridkin e04c5c62a3 fix(rabbitmq): create the tenant-ca projection the Role relies on
The dashboard Role already refuses to grant the key-bearing TLS
Secrets, but nothing in the chart told the CA-extraction controller
to publish the key-free trust anchor those Secrets feed: no
TenantProjection sentinel referenced the CA Certificate's Secret as
a source. Without that sentinel the controller has nothing to act
on, so rabbitmq-<name>.tenant-ca is never created and a tenant has
no path to the CA at all, contrary to what the README and the Role's
own comments describe.

Add templates/tenant-projection.yaml, gated on the same TLS
condition as the certificate chain and mirroring the nats and qdrant
charts: it names the CA Certificate's Secret as the projection
source. Drop the internal.cozystack.io/publish-ca-cert labels from
certmanager.yaml, since the controller resolves its source by name
through the sentinel rather than by a label on the Secret, and
correct the comments and README passage that described that label as
the actual publication mechanism.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:48 +03:00
Aleksei Sviridkin ff56ebae72 docs(rabbitmq): cut the values.yaml TLS prose to the callsite facts
The annotation block restated the README's TLS section nearly verbatim.
Only the @param and @field lines reach the generated schema and README
table; the rest was a second copy that would drift against the first.

Keeps the three facts that change what a caller sets — neither field is
inferred from external, enabling TLS moves the binding port, and
disableNonTLSListeners is not external-only — and points at the README.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:48 +03:00
Aleksei Sviridkin 411a3d6c02 docs(rabbitmq): cut template comments duplicating the README
The TLS comments had grown to narrate rejected alternatives and operator
internals at length, roughly matching the volume of configuration they
annotate. The README already carries every long-form explanation, so the
duplicates are prose that will drift out of sync with it.

Each block keeps the mechanism a next editor needs to avoid breaking
something and points at the README section holding the rest.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:47 +03:00
Aleksei Sviridkin a0efa75dd2 fix(rabbitmq): add the missing bare pod-wildcard SAN
The certificate covered the client and headless Services in all four
resolvable forms but gave the pod wildcard only three: bare
*.<release>-nodes was absent. A client in the broker's own namespace
dialling <pod>.<release>-nodes resolves through the DNS search path,
connects, and fails only at hostname verification — which reads as a
certificate bug rather than a missing name, the exact failure the
four-form rule exists to prevent.

The suite asserted the headless forms with contains, so it stayed
green for the form nobody listed; it now pins the whole list.

Also drops key encipherment from the leaf usages: it authorises RSA
key transport and the key is ECDSA, so it requested a capability the
key type cannot supply.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:47 +03:00
Aleksei Sviridkin 52e60935e9 docs(rabbitmq): correct the disableNonTLSListeners escape hatch
The refusal message and the README told the reader to manage users and
vhosts outside the chart. That reads as an instruction to move them
into hand-written User/Vhost/Permission objects, which fails the same
way: the topology operator derives its connection from the
RabbitmqCluster, not from who created the CR, so those objects break
identically and without the chart's refusal to warn anyone.

Say what the flag actually costs — the cluster leaves that operator's
reach entirely — and point at rabbitmqctl or a definitions import.

Also narrows the claim that no option closes plaintext: a variant
through spec.rabbitmq.additionalConfig looks reachable but is not
wired or validated here, so it is named as an open question rather
than left implicitly denied.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:47 +03:00
Aleksei Sviridkin 636228f706 test(rabbitmq): pin the operator connection URI to the cluster domain
Every existing assertion on rabbitmq.com/operator-connection-uri ran on
the default domain, so a hardcoded cozy.local would have satisfied all
of them. On a cluster with its own cluster-domain that URI does not
resolve and messaging-topology-operator loses the broker outright, so
the case that distinguishes the two is worth having.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:46 +03:00
Aleksei Sviridkin 4e3d9cd784 docs(rabbitmq): drop the TLS claims that only held under inheritance
The TLS scope section described an upgrade that turns TLS on by itself
and told the reader to set tls.enabled: false to avoid it. Neither is
true now that the field defaults to false: the rolled pods, the
cert-manager dependency, the metrics port swap and the rewritten
binding port are all consequences of an explicit edit, so they are
described that way.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:46 +03:00
Aleksei Sviridkin 36ee8a2608 feat(rabbitmq): make TLS opt-in instead of following external
tls.enabled inherited the value of external when unset, so enabling
external access also enabled TLS. That is not free on an existing
instance: setting spec.tls.secretName makes the operator rewrite the
port key of the <release>-default-user Secret from 5672 to 5671, and
that Secret is the Service Binding handed to the tenant. Every client
resolving its endpoint from the binding — the documented way — would
move onto TLS on upgrade and have to verify a per-release CA it has no
key-free way to obtain.

Resolve unset, null and false to false instead, so TLS is only ever on
because someone asked for it. enabled becomes a plain bool now that
there is no third state to express.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:45 +03:00
Aleksei Sviridkin dd04377e71 docs(rabbitmq): trim template comments to what the callsite needs
The template had grown to roughly 60% comment prose, most of it restating the
README verbatim — the strategic-merge-patch reasoning, the topology-operator TLS
derivation and the disableNonTLSListeners scope each existed in two places that
can drift apart, and the README is the copy users read. Several also carried
review-thread voice ("both directions are deliberate", "refuse the contradiction
rather than drop it") rather than describing the artifact.

Keep at each callsite only the facts an editor needs before touching that line —
why the override exists in neither direction, why the connection URI is pinned to
plaintext, what the flag does not cover — and let the README own the rest.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:45 +03:00
Aleksei Sviridkin 8e48cfc8fd fix(rabbitmq): refuse contradictory TLS settings and correct the upgrade note
Two things this change got wrong about its own behaviour.

tls.disableNonTLSListeners was silently discarded whenever TLS resolved to off,
because the value was folded into the TLS condition before anything read it.
Asking to close the plaintext listeners with TLS off leaves nothing listening,
and the operator rejects the combination itself, returning "TLS must be enabled
if disableNonTLSListeners is set to true" and setting ReconcileSuccess=False.
Swallowing it in the chart was worse than what the operator does: the operator
at least records a TLSError event, while a dropped flag rendered a
healthy-looking cluster with every plaintext listener open and no trace of the
request. The likeliest way in is an operator hardening an internal broker who
sets only the flag, since tls.enabled follows external and so resolves to false
there. Fail with a message naming tls.enabled instead.

The README also claimed messaging clients are unaffected by enabling TLS. That
holds only for a client with a hardcoded port. <release>-default-user is the
Service Binding object the chart grants to the tenant, and the operator rewrites
its port key from 5672 to 5671 as soon as spec.tls.secretName is set — gated on
TLS alone, not on whether the plaintext listener stayed open — along with the
per-plugin port keys. A binding-driven client, which is the documented way to
consume it, follows that change and must speak TLS from then on, without being
able to verify the certificate until the trust anchor is obtainable. Say so.

Also record that the connection-uri annotation fans out: the operator's
annotation filter strips only kubernetes.io and k8s.io keys, so it lands on every
child object including the Secret the tenant reads.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:45 +03:00
Aleksei Sviridkin d668dfbd6a feat(rabbitmq): add opt-in switch to close the plaintext listeners
TLS so far only added encrypted endpoints; nothing closed the unencrypted ones,
and the chart had no way to express the intent. tls.disableNonTLSListeners gives
the operator that switch, defaulting to false so an upgrade never closes a
listener a client is already using — the choice to break plaintext has to be
made deliberately.

Its reach is worth stating plainly, because the natural reading is wrong in both
directions. It writes `listeners.tcp = none` into rabbitmq.conf, which is broker
level: the socket is not opened on any interface, so in-cluster clients lose
plaintext at the same instant external ones do, and there is no configuration
that closes it outside while keeping it inside. In the other direction it is not
"TLS everywhere" either — Erlang distribution on 25672 and epmd on 4369 ride the
headless Service, which the flag never touches, so inter-node traffic stays in
the clear regardless.

The flag also cannot coexist with chart-managed users. messaging-topology-operator
derives its connection as TLSEnabled() && (!connectUsingHTTP ||
DisableNonTLSListeners()), so the flag forces TLS on that side whatever the
operator's own configuration says, and it verifies the broker against
x509.SystemCertPool(), which cannot hold a per-release CA. The connection-uri
annotation is no escape, since it would point at the port the flag just closed —
so that annotation is now omitted when the flag is set, rather than advertising a
closed port. Every User, Vhost and Permission would silently stop reconciling.
Rendering fails instead, naming the conflict, and the flag stays usable for an
instance whose users and vhosts are managed outside the chart.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:44 +03:00
Aleksei Sviridkin 6f1ada4159 fix(rabbitmq): add the two-segment wildcard SAN for pod addresses
The SAN list covered every Service name in all resolvable forms but applied the
rule only to the non-wildcard entries, leaving the per-pod wildcard with .svc and
.svc.<domain> and no two-segment form. That is the form the operator itself uses:
RABBITMQ_NODENAME is rabbit@$(MY_POD_NAME).$(K8S_SERVICE_NAME).$(MY_POD_NAMESPACE)
and K8S_HOSTNAME_SUFFIX matches, so <pod>.<name>-nodes.<namespace> is the
canonical pod address rather than an incidental alias, and it is the form the
upstream API documentation shows to users.

A client dialling a pod by that name resolves it through the DNS search path,
connects, and fails only at hostname verification — which presents as a
certificate bug rather than a missing name. It would also break outright the
moment inter-node mTLS is wired through caSecretName.

Move the clusterDomain helper above the tri-state doc block while here, so each
comment sits next to the definition it describes.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:44 +03:00
Aleksei Sviridkin 2f42cd6baf fix(rabbitmq): keep the topology operator on the plaintext management port
Setting spec.tls.secretName makes RabbitmqCluster.TLSEnabled() true, and
messaging-topology-operator derives its own connection from that:

    useTLSForConnection = TLSEnabled() && (!connectUsingHTTP || DisableNonTLSListeners())

connectUsingHTTP comes from CONNECT_USING_PLAIN_HTTP, which the shipped
Deployment does not set, so the expression collapses to TLSEnabled(). The
operator would dial https://<release>:15671 and verify it against
x509.SystemCertPool() — a process-wide pool that cannot contain a per-release
self-signed CA. Every User, Vhost and Permission this chart renders from users
and vhosts would fail with "certificate signed by unknown authority".

The failure is quiet, which is what makes it worth guarding: users already
applied to the broker keep working, so nothing looks wrong, while password
rotation, new users, new vhosts and permission changes all stop taking effect.

Annotate the cluster with rabbitmq.com/operator-connection-uri on the plaintext
management port when TLS is on. extractURIandScheme reports tlsEnabled=false for
an http:// scheme, so the operator keeps using the listener the chart
deliberately leaves open — that listener is load-bearing, not merely tolerated,
and the README now says so. Only the operator's management connection is
affected; broker traffic is untouched.

Also add the two-segment <name>.<namespace> SANs for both the client and headless
Services. That form resolves through the DNS search path, so a client connecting
to amqps://<name>.<namespace>:5671 reached the broker and failed only at hostname
verification — which presents as a certificate bug rather than a missing name.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 11:18:44 +03:00