The controller discovers where to query SeaweedFS bucket size metrics
via the namespace.cozystack.io/monitoring label, which assumes the
monitoring stack that scrapes SeaweedFS runs in the same cluster. Some
deployments host SeaweedFS and its metrics in a separate cluster, so
add a --seaweedfs-metrics-endpoint flag to cozystack-controller (and a
matching cozystackController.seaweedfsMetricsEndpoint chart value): an
explicit base URL of a Prometheus-compatible query API, with scheme,
host, optional port and path prefix; /api/v1/query is appended. Basic
auth may be embedded as URL userinfo. The flag is validated and
normalized at startup, so a malformed URL fails the rollout rather
than the queries. Empty keeps label-based discovery unchanged.
Bucket sizes feed billing, so metrics failures no longer collapse to
"no size": previously any query failure yielded an empty result map
and the reconciler overwrote Workload resources with it, silently
wiping the recorded sizes. The reconciler now retains the last known
s3-storage-bytes and s3-physical-storage-bytes values, emits a
BucketMetricsUnavailable warning event on the WorkloadMonitor, and
returns the error from Reconcile so it lands in the reconcile error
metrics.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Every catalog engine issues a per-release CA, but the object holding
ca.crt usually holds a private key beside it (cert-manager's tls.key,
CloudNativePG's ca.key), so granting a tenant read on it hands over key
material too. Add a controller that publishes the trust anchor as one
canonical, key-free Secret per release, "<release>.tenant-ca", holding
only ca.crt.
A chart declares the anchor by rendering a namespaced TenantProjection
sentinel (internal.cozystack.io/v1alpha1) that names the source Secret
and key. The controller rebuilds ca.crt from the parsed certificate DER
rather than copying the source, refuses any value carrying private key
material, and owns the projection by owner reference so a foreign Secret
at the same name is never overwritten. No tenant RBAC grants any verb on
internal.cozystack.io; a ValidatingAdmissionPolicy pins writes to
helm-controller as defence in depth.
The controller does not grant tenant read access — the lineage webhook
does, from the ApplicationDefinition's spec.secrets. This side stamps the
engine-agnostic internal.cozystack.io/tenant-ca label and forces a
re-admission when those selectors change.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What
Removes the `network.cozystack.io` API group — `ExposureClass` +
`ServiceExposure` (#3081) — and its reconciler, in favor of native
`Service` `type: LoadBalancer` + `loadBalancerClass`.
## Why
Full rationale and discussion in #3164. For **managed applications** the
chart already owns the Service, so:
- `type: LoadBalancer` + `loadBalancerClass` + an admin-provisioned pool
covers vendor-neutral external exposure without a dedicated API group
and controller;
- the `externalIPs` (node-IP) default never went through
`ServiceExposure` (its backend is a no-op) and is left unchanged;
- the only current consumer, the host ingress, keeps rendering `type:
LoadBalancer` and just drops the `ServiceExposure`; the admin provisions
the address pool / announcer out of band, as the install docs already
assume.
`ServiceExposure`'s one unique feature — dynamic per-namespace scoping
of a shared VIP pool — is a convenience the managed-app path does not
need, and it does not justify a second tenant-networking API group next
to `sdn.cozystack.io`.
## Changes
- Delete `api/network/v1alpha1` and
`internal/controller/serviceexposure` (+ CRD definitions, e2e bats,
controller registration, RBAC).
- Host ingress: drop the `ServiceExposure` template; add an optional
`publishing.loadBalancerClass` that sets `spec.loadBalancerClass` on the
host-ingress LoadBalancer Service.
- The `externalIPs` path and default behavior are unchanged.
## Verification
- `go build ./...` — green.
- `helm unittest` — `core/platform` 72/72, `extra/ingress` 25/25 (incl.
host-vs-tenant `loadBalancerClass` scoping and the immutable-field
migration guard).
## Migration
None required. `network.cozystack.io` (#3081) has never shipped in a
release — it is on `main` only (not backported to `release-1.5`; absent
from v1.4.x / v1.5.x), so no upgrading cluster has these CRDs and this
removes it before it is ever released.
Clusters that track `main` and explicitly set `publishing.exposureClass`
(opt-in, only since 2026-06-30) should provision their MetalLB / Cilium
address pool themselves and set `publishing.loadBalancerClass`; any
orphaned `cozystack-<class>` pool can be deleted by hand. Default
clusters (externalIPs path) are unaffected.
## Status
**Draft / RFC for the #3164 discussion** — this removes an API group
that landed only days ago, so nothing here should merge before the
design call. cc @lexfrei (author of #3081) and @lllamnyp (#3164).
Deferred: when Kubernetes actually removes `Service.spec.externalIPs`,
the node-IP default migrates via a MetalLB `IPAddressPool` with no
`L2Advertisement` (allocate-but-don't-announce) — a separate change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for configuring the ingress host Service with a native
`loadBalancerClass` value.
* Expanded chart tests to verify class wiring and immutability behavior
across install and upgrade scenarios.
* **Bug Fixes**
* Removed the legacy exposure-class/service-exposure flow and related
controller behavior.
* Updated templates and CRD packaging so the platform no longer renders
or depends on those deprecated resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove the network.cozystack.io API group (ExposureClass + ServiceExposure)
and its reconciler. Managed-application charts own their Service, so native
type: LoadBalancer + loadBalancerClass + an admin-provisioned pool covers
external exposure without a dedicated API group and controller. The host
ingress -- the only consumer -- keeps rendering type: LoadBalancer and gains
an optional publishing.loadBalancerClass to pick the LB controller; the
externalIPs (node-IP) default path is unchanged.
Rationale in cozystack#3164.
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The operator-provided wildcard MVP serves the root tenant only: the root
ingress controller and Gateway read a TLS Secret from their own namespace.
Child tenants run their own ingress controller / Gateway in a separate,
default-deny namespace and cannot read that Secret cross-namespace.
Add a controller that mirrors the operator wildcard Secret into every
tenant namespace that owns a TLS termination point (its
namespace.cozystack.io/ingress or .../gateway label equals its own name).
The source is identified by name from the platform values channel
(_cluster.wildcard-secret-name) with no extra operator input; replicas
carry a managed-by label and refresh on rotation. An individual replica is
removed when its namespace stops terminating TLS; every replica is torn
down only on an explicit disable (clearing wildcard-secret-name). A missing
or mistyped source, or a transiently-absent values channel, keeps existing
replicas, so tenant TLS never drops on a transient gap. The controller's
Secret cache is scoped to managed replicas and the values channel, not
every cluster Secret. Only the platform controller's ServiceAccount gains
Secret writes — no tenant RBAC is widened, and every consumer reads only
its own-namespace copy.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add a tenant quota reconciler to cozystack-controller that realises the
hierarchical (sub-tree) quota model at runtime. Each tenant's declared
quota is the budget for its whole sub-tree: a child that declares its own
quota carves a fixed slice out of the parent budget, while a child without
a quota shares the parent's remaining pool.
The controller maintains a per-namespace 'tenant-quota-allocated'
ResourceQuota (Kubernetes enforces the most restrictive quota, so it binds
without fighting Flux over the chart-rendered 'tenant-quota'), aggregating
sub-tree usage so a pool's members collectively stay within the budget. A
--tenant-quota-buffer-percent flag temporarily inflates pool budgets so
workloads already over a freshly-introduced quota keep running during
rollout; overcommitted pools emit a QuotaOvercommitted event.
The model and the selector-spans-namespaces/aggregated-usage approach are
adapted from OpenShift's ClusterResourceQuota controllers (openshift/api,
library-go, cluster-policy-controller, apiserver-library-go), Apache-2.0,
Copyright Red Hat, Inc. and the OpenShift Authors.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Add the serviceexposure controller to the cozystack-controller binary. It
resolves a ServiceExposure's ExposureClass to a backend and renders the
backend-specific pool and announcer CRs, then reports the assigned IPs in
status.
Backends are pluggable behind a small interface, one per file:
- externalIPs: renders nothing (the node owns the address) — preserves the
historical default behaviour.
- metallb: IPAddressPool + L2Advertisement in cozy-metallb, scoped to the
target Service's namespace.
- cilium: CiliumLoadBalancerIPPool (cilium.io/v2) + CiliumL2AnnouncementPolicy
(cilium.io/v2alpha1), scoped to the Service via serviceSelector.
- robotlb: renders nothing (the cloud LB owns allocation and announcement).
The rendered CRs belong to foreign API groups not in the Go scheme, so they
are emitted as unstructured objects. They carry no OwnerReference (the
Cilium CRs are cluster-scoped and the MetalLB CRs live in a different
namespace than the owning ServiceExposure), so ownership is tracked via a
management label plus owner annotations, cleanup is driven by a finalizer,
and the controller refuses to take over an identically-named resource it
does not manage. The controller never mutates the target Service, leaving
its shape to the chart that renders it.
Grant the controller write access to metallb.io and cilium.io pool /
announcer resources.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The dashboard controller used to project ApplicationDefinitions
into a set of dashboard.cozystack.io CRDs (Sidebar, Factory,
Navigation, Breadcrumb, CustomFormsOverride, CustomFormsPrefill,
CustomColumnsOverride, MarketplacePanel, TableUriMapping,
CFOMapping, BreadcrumbInside) consumed by the legacy UI.
cozystack-ui now reads ApplicationDefinitions directly, so the
controller, its API types, the generated CRDs and the related
RBAC rules are no longer used.
Removes:
- internal/controller/dashboard
- api/dashboard/v1alpha1
- packages/system/cozystack-controller/definitions/dashboard.cozystack.io_*.yaml
- dashboard.cozystack.io wiring from cmd/cozystack-controller/main.go
- dashboard.cozystack.io RBAC rules from cozystack-controller and cozy-dashboard charts
- ./api/dashboard/... codegen path from hack/update-codegen.sh
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
This patch drops the custom caching of the Cozystack resource
definitions in favor of the informer cache and adds a flag to the
Cozystack controller to select, whether it restarts the cozystack-api
deployment or the cozystack-api daemonset.
As with the new default behavior of using a local endpoint for the k8s
API by the lineage webhook and the Cozystack API, the Cozystack
controller now also defaults to restarting a Cozystack API DaemonSet
instead of a Deployment. To revert to the old behavior, disable the
local k8s API endpoint on the webhook and cozystack API and set the
`cozystackController.cozystackAPIKind` value in the Cozystack controller
system Helm chart to "Deployment".
```release-note
[controller] Use informer cache instead of the older bespoke
implementation and add support for running the Cozystack API as a
DaemonSet.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
The lineage-controller-webhook makes a lot of outgoing API calls for
every event it handles, contributing to a high API server latency,
increasing the number of in-flight requests and generally degrading
performance. This patch remedies this by separating the lineage
component from the cozystack-controller and deploying it as a separate
component on all control-plane nodes. Additionally, a new internal label
is introduced to track if a resource has already been handled by the
webhook. This label is used to exclude such resources from
consideration. Addresses #1513.
```release-note
[lineage] Break webhook out into a separate daemonset. Reduce
unnecessary webhook calls by marking handled resources and excluding
them from consideration by the webhook's object selector.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Due to a deficiency of cozypkg (--with-source reconciles the HelmChart,
but not the HelmRepository), we have to use workarounds to bulletproof
the latest migration, by applying directly from the assets server.
```release-note
[installer] Run 20th migration using helm charts directly from the
assets server instead of relying on cozypkg to reconcile its resources
properly.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch implements name-based selectors for
`CozystackResourceDefinitions.spec.secrets`. Application developers may
now specify secrets that should or should not be visible to end users by
specifying a `resourceNames` field with a string slice of acceptable
names. This will, for instance, let developers exclude a secret like
`postgres-dbname-superuser` that has a predictable name even if it does
not have predictable labels. Simple templates are supported, so
`postgres-{{ .name }}-superuser` is also a valid entry under
`resourceNames`.
```release-note
[lineage, controller] Let application developers determine resource
visibility for end users by name, as well as by labels.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
A new dashboard based on https://github.com/PRO-Robotech/openapi-ui
project
<img width="1720" height="1373" alt="Screenshot 2025-08-01 at 09-01-00
OpenAPI UI"
src="https://github.com/user-attachments/assets/7ae04789-24ec-4e4b-830b-6f16e96513eb"
/>
<img width="1720" height="1373" alt="Screenshot 2025-08-01 at 09-01-14
OpenAPI UI"
src="https://github.com/user-attachments/assets/ca5aa85d-43f0-4b5b-b87a-3bc237834f10"
/>
<img width="1720" height="1373" alt="Screenshot 2025-08-01 at 09-02-05
OpenAPI UI"
src="https://github.com/user-attachments/assets/ebee7bfa-c3ac-4fe6-b5e1-43e9e7042c6a"
/>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-api] Implement TenantNamespace, TenantModules, TenantSecret and TenantSecretsTable resources
[cozystack-controller] Introduce new dashboard-controller
[dashboard] Introduce new dashboard based on openapi-ui
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
[cozystack-controller] Introduce new dashboard-controller
[dashboard] Introduce new dashboard based on openapi-ui
Co-authored-by: kklinch0 <kklinch0@gmail.com>
Signed-off-by: kklinch0 <kklinch0@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Many resources created as part of managed apps in cozystack (pods,
secrets, etc) do not carry predictable labels that unambiguously
indicate which app originally triggered their creation. Some resources
are managed by controllers and other custom resources and this
indirection can lead to loss of information. Other controllers sometimes
simply do not allow setting labels on controlled resources and the
latter do not inherit labels from the owner. This patch implements a
webhook that sidesteps this problem with a universal solution. On
creation of a pod/secret/PVC etc it walks through the owner references
until a HelmRelease is found that can be matched with a managed app
dynamically registered in the Cozystack API server. The pod is mutated
with labels identifying the managed app.
```release-note
[cozystack-controller] Add a mutating webhook to identify the Cozystack
managed app that ultimately owns low-level resources created in the
cluster and label these resources with a reference to said app.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch expands the CozystackResourceDefinitions with new label
selector fields to include and exclude secrets by their labelsets.
This will enable application developers to selectively show or hide
application secrets to and from end-users.
```release-note
[platform] Add selectors for application secrets, offering developers
an API to control secret visibility for end users.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
In an earlier patch the Cozystack controller now reads arbitrary objects
in the cluster to establish the lineage of any created pod, service,
pvc, or secret. These objects may be created by various other
controllers, so in general, the controller now requires read permissions
on arbitrary objects in the cluster.
```release-note
[cozystack-controler] Fix an RBAC error that prevented the workload
labelling feature from working.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Many resources created as part of managed apps in cozystack (pods,
secrets, etc) do not carry predictable labels that unambiguously
indicate which app originally triggered their creation. Some resources
are managed by controllers and other custom resources and this
indirection can lead to loss of information. Other controllers sometimes
simply do not allow setting labels on controlled resources and the
latter do not inherit labels from the owner. This patch implements a
webhook that sidesteps this problem with a universal solution. On
creation of a pod/secret/PVC etc it walks through the owner references
until a HelmRelease is found that can be matched with a managed app
dynamically registered in the Cozystack API server. The pod is mutated
with labels identifying the managed app.
```release-note
[cozystack-controller] Add a mutating webhook to identify the Cozystack
managed app that ultimately owns low-level resources created in the
cluster and label these resources with a reference to said app.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
Based on the comprehensive summary of changes, here are the release
notes:
- **New Features**
- Added a new Kubernetes controller for managing workload monitoring
- Introduced telemetry collection capabilities with configurable options
- Added new Custom Resource Definitions (CRDs) for Workload and
WorkloadMonitor
- **Improvements**
- Enhanced API infrastructure with new API group and version
- Improved deployment configurations for various system components
- Added development container and workflow configurations
- **Bug Fixes**
- Updated import paths to correct domain naming
- **Chores**
- Updated copyright years
- Refined module dependencies
- Standardized code linting and testing configurations
- **Infrastructure**
- Increased `cozystack-api` deployment replicas from 1 to 2 for improved
availability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Timofei Larkin