## What this PR does The VM export routes point at a Service that does not exist, and the tenant network policy would block the Ingress even if it did. KubeVirt creates the export proxy Service under the name `virt-exportproxy` (`VirtExportProxyName` in `pkg/virt-operator/resource/generate/components`, v1.9.0). The Ingress and the TLSRoute that publish `vm-exportproxy.<host>` name a `vm-exportproxy` Service, and nothing creates one. The backend was `virt-exportproxy` when the Ingress was added in0ddaff938. Then3780925a6renamed the hostname to `vm-exportproxy` and the backend along with it. The hostname stays, only the backend goes back. Renaming the backend is not enough for the Ingress, because it is served by the ingress controller in tenant-root and tenant egress is default-deny. Each allowed destination has its own `allow-to-*` policy. The CDI upload proxy has `allow-to-cdi-upload-proxy`, the export proxy had nothing. The new `allow-to-vm-exportproxy` policy renders for the same tenants and the same source pods as the CDI one. Its destination is narrower: only the `kubevirt.io=virt-exportproxy` pods in `cozy-kubevirt`, on TCP 8443, the container port virt-operator gives them. The unit tests pin the backend name for the Ingress and the TLSRoute, and the placement, selector and port of the new policy. I checked this on a test cluster with KubeVirt v1.9.0 and ingress-nginx. Without the fix the controller logs `Error obtaining Endpoints for Service "cozy-kubevirt/vm-exportproxy"`. With the backend renamed and the tenant allowed egress to virt-exportproxy, a passthrough request to `vm-exportproxy.<host>` reaches the export proxy, and it answers with its own serving certificate. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kubevirt): requests to vm-exportproxy.<host> now reach the KubeVirt export proxy: the Ingress and TLSRoute route to the virt-exportproxy Service KubeVirt creates, and tenants are allowed egress to the export proxy. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tenant workloads can now connect to the KubeVirt export proxy over TCP port 8443. * **Bug Fixes** * Updated export-proxy Ingress and TLSRoute routing to use the `virt-exportproxy` backend, keeping export-proxy traffic directed to the correct service. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Tenant
A tenant is the main unit of security on the platform. The closest analogy would be Linux kernel namespaces.
Tenants can be created recursively and are subject to the following rules:
Tenant naming
Tenant names must be alphanumeric:
- Lowercase letters (
a-z) and digits (0-9) only - Must start with a lowercase letter
- Dashes (
-) are not allowed, unlike with other services - Maximum length depends on the cluster configuration (Helm release prefix and root domain)
This restriction exists to keep consistent naming in tenants, nested tenants, and services deployed in them.
A tenant cannot be named foo-bar because parsing internal resource names like tenant-foo-bar would be ambiguous.
For example:
- The root tenant is named
root, but internally it's referenced astenant-root. - A nested tenant could be named
foo, which would result intenant-fooin service names and URLs.
Unique domains
Each tenant has its own domain.
By default, (unless otherwise specified), it inherits the domain of its parent with a prefix of its name.
For example, if the parent had the domain example.org, then tenant-foo would get the domain foo.example.org by default.
Kubernetes clusters created in this tenant namespace would get domains like: kubernetes-cluster.foo.example.org
Example:
tenant-root (example.org)
└── tenant-foo (foo.example.org)
└── kubernetes-cluster1 (kubernetes-cluster1.foo.example.org)
Nesting tenants and reusing parent services
Tenants can be nested. A tenant administrator can create nested tenants using the "Tenant" application from the catalogue.
Higher-level tenants can view and manage the applications of all their children tenants. If a tenant does not run their own cluster services, it can access ones of its parent.
For example, you create:
- Tenant
tenant-u1with a set of services likeetcd,ingress,monitoring. - Tenant
tenant-u2nested intenant-u1.
Let's see what will happen when you run Kubernetes and Postgres under tenant-u2 namespace.
Since tenant-u2 does not have its own cluster services like etcd, ingress, and monitoring,
the applications running in tenant-u2 will use the cluster services of the parent tenant.
This in turn means:
- The Kubernetes cluster data will be stored in
etcdfortenant-u1. - Access to the cluster will be through the common
ingressoftenant-u1. - Essentially, all metrics will be collected in the
monitoringfromtenant-u1, and only that tenant will have access to them.
Example:
tenant-u1
├── etcd
├── ingress
├── monitoring
└── tenant-u2
├── kubernetes-cluster1
└── postgres-db1
Network isolation
Tenant network isolation is enforced with Cilium network policies. On the isp-hosted and isp-hosted-slim platform variants the CNI belongs to the host, so these policies are created only if the host serves the Cilium policy API when the tenant is installed or updated. On a host without Cilium, tenants get no network isolation from Cozystack, and isolating them is up to the host. If Cilium appears on the host later, the policies are created on the tenant's next chart or values change, not automatically.
Parameters
Common parameters
| Name | Description | Type | Value |
|---|---|---|---|
host |
The hostname used to access tenant services (defaults to using the tenant name as a subdomain for its parent tenant host). | string |
"" |
etcd |
Deploy own Etcd cluster. | bool |
false |
monitoring |
Deploy own Monitoring Stack. | bool |
false |
tracingCentral |
Send this tenant's traces to the shared VictoriaTraces store in tenant-root instead of a per-tenant backend; its Grafana reads them through a vmauth there that pins the tenant's own account, so tenants sharing the store cannot read each other's traces. It needs the tenant's own Monitoring (monitoring: true) and tenant-root hosting the store (tracingCentralHost: true on its Monitoring, with a cluster-mode tracingStorages entry named generic); without that the tenant's monitoring render fails rather than dropping spans. tenant-root, and whoever it admits to its Grafana, are inside the store's trust boundary. Applications should export over OTLP/HTTP, in requests below 3 MiB: spans above the platform's per-tenant write rate are retried over HTTP but dropped over gRPC, and a larger request is refused and not retried. |
bool |
false |
ingress |
Deploy own Ingress Controller. | bool |
false |
gateway |
Deploy own Gateway API Gateway (backed by Cilium Gateway API controller). When unset (the default) or set to false, the tenant does not own a Gateway and instead publishes through the nearest ancestor tenant that owns one, regardless of whether host is empty or custom (false currently behaves the same as unset and does not opt out of that inheritance). Set to true to give the tenant its own Gateway with its own LoadBalancer IP and Certificate. A tenant with a custom (non-derived) host that needs its apex publicly reachable must set true explicitly, because an ancestor's TLS certificate does not cover a custom apex. Note: leave the key absent (do not write gateway: null): the chart distinguishes "unset" via missing-key, not via null value, to satisfy the JSON schema generated from this comment. |
bool |
false |
gatewayClass |
GatewayClass for this tenant's own Gateway. Empty uses the platform default (gateway.className). Only the classes the operator lists in the platform's gateway.tenantSelectableClasses, plus that default, may be named here — anything else fails this tenant's own gateway release and reaches no other tenant. Read only when this tenant owns a Gateway; an inheriting tenant uses its ancestor's. Whether a class terminates TLS itself or at its provider's edge is declared once per class on the platform, not here. |
string |
"" |
seaweedfs |
Deploy own SeaweedFS. | bool |
false |
computeplane |
Deploy own ComputePlane — a single-tenant, Cozystack-managed cluster for untrusted-code applications. The tenant receives no admin kubeconfig for it. Automatic routing of catalog applications onto it (placement: ComputePlane) is a planned follow-up and is not available yet; until it lands, external catalogs target the cluster via its computeplane-cluster-admin-kubeconfig Secret. See design-proposals/compute-plane in cozystack/community. | bool |
false |
schedulingClass |
The name of a SchedulingClass CR to apply scheduling constraints for this tenant's workloads. | string |
"" |
resourceQuotas |
Define resource quotas for the tenant. | map[string]quantity |
{} |
Configuration
Resource Quotas
The resourceQuotas parameter allows you to limit resources available to the tenant. Supported keys include:
Compute resources (converted to requests.X and limits.X):
cpu- Total CPU cores (e.g.,"4"or"500m")memory- Total memory (e.g.,"4Gi"or"512Mi")ephemeral-storage- Ephemeral storage limit (e.g.,"10Gi")storage- Total persistent storage (e.g.,"100Gi")
Object count quotas (passed as-is):
pods- Maximum number of podsservices- Maximum number of servicesservices.loadbalancers- Maximum number of LoadBalancer servicesservices.nodeports- Maximum number of NodePort servicesconfigmaps- Maximum number of ConfigMapssecrets- Maximum number of Secretspersistentvolumeclaims- Maximum number of PVCs
Example:
resourceQuotas:
cpu: 4
memory: 4Gi
storage: 10Gi
services.loadbalancers: "3"
pods: "50"