Commit Graph
3839 Commits
Author SHA1 Message Date
Arsolitt 80a01fd330 chore(kafka): regenerate codegen deepcopy artifacts
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-05-27 11:16:41 +03:00
Arsolitt f37149c70d fix(kafka): decouple external listener from tls.enabled (tri-state)
The external LoadBalancer listener (port 9094) was rendered whenever
tls.enabled was explicitly true, regardless of the external flag. This caused
Strimzi to stall waiting for a LoadBalancer IP when tls.enabled=true but
external=false.

External is now the sole gate for the listener. tls.enabled only controls
TLS on the listener when it is rendered (auto-on when external=true and
tls.enabled is unset, explicit value otherwise). Update tests to match.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-05-27 11:16:41 +03:00
Arsolitt 6f15714846 test(kafka): add TLS test coverage
- Add helm-unittest suite covering TLS listener config, tri-state nil/true/false cases
- Cover external=false + tls.enabled=false negative combinations
- Add dashboard resourcemap tests for CA secret inclusion
- Add bats tests for cozyrds schema nullable field and Strimzi CA secret naming

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-05-27 11:16:40 +03:00
Arsolitt 9407ccabd9 feat(kafka): implement TLS via Strimzi listener configuration
- Configure Kafka CR with TLS listener spec using Strimzi-managed PKI
- Gate external bootstrap RBAC resources on $showExternal predicate
- Use resolved $tlsEnabled variable for consistent tri-state evaluation
- Add Strimzi clients CA secret reference to dashboard resourcemap

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-05-27 11:16:40 +03:00
Arsolitt 2eba34618c feat(kafka): add TLS schema and generated artifacts
- Add tri-state tls.enabled (*bool) to Kafka API types for strict nil/true/false semantics
- Extend values.yaml and JSON schema with TLS configuration block
- Update Makefile with sed+jq patching for nullable bool schema and test target
- Update cozyrds schema to reflect nullable tls.enabled type
- Update README with Strimzi-managed PKI documentation and authentication notes

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-05-27 11:16:39 +03:00
Aleksei Sviridkin b630706df6 feat(apps): mark stateful-app storageClass fields as immutable (#2639)
## Summary

Annotate `storageClass` as immutable across every stateful chart that
exposes one: clickhouse, foundationdb, harbor, http-cache, kafka (Kafka
+ ZooKeeper), kubernetes (top-level + nodeGroups[]), mariadb, mongodb,
nats, openbao, opensearch, postgres, qdrant, rabbitmq, redis, vm-disk.
The PVC-pinning argument is identical for every chart — Kubernetes fixes
`storageClassName` on each PVC at creation, so editing this field on an
existing resource never migrated data; the schema annotation makes that
contract explicit and lets schema-aware consumers refuse the edit
up-front.

Schema-aware consumers today: the Cozystack UI
(cozystack/cozystack-ui#6) reads the rule from the chart's openAPISchema
and renders the field read-only on edit forms.

This PR is **schema-side only**. The cozystack aggregated apiserver does
not yet evaluate CEL rules in `openAPISchema` (`MariaDB` / `Postgres` /
etc. are application-backed, not CRDs); tracking issue
cozystack/cozystack#2657 covers wiring CEL into
`pkg/registry/apps/application/rest.go::Update`. Until that lands, an
apiserver-level `kubectl patch` that changes `storageClass` is still
accepted — but the underlying StatefulSet PVCs do not move, so there's
no data-corruption risk, only a confusing UX. The UI catches that case
today; the apiserver gate closes it permanently.

## What changes

For each app in the list above:

- `packages/apps/<app>/values.yaml` — gain a `## @immutable` directive
directly above (or below, for fields inside a `@typedef`) the
`storageClass` line.
- Regenerated via `make generate` in each package:
- `api/apps/v1alpha1/<app>/types.go` — kubebuilder `XValidation` marker
on the relevant field(s).
- `packages/apps/<app>/values.schema.json` — `x-kubernetes-validations:
[{rule: "self == oldSelf", message: …}]` on `storageClass`.
  - `packages/apps/<app>/README.md` — regenerated parameter table.
- `packages/system/<app>-rd/cozyrds/<app>.yaml` — the embedded
`openAPISchema` on the ApplicationDefinition reflects the change.

Refactor: `pkg/registry/apps/application/rest.go` factors the inline
schema-build path into a new `buildSpecSchema` helper so it can be
unit-tested without standing up a fake apiserver. Three regression tests
in `pkg/registry/apps/application/rest_spec_schema_test.go` pin the
contract that `x-kubernetes-validations` survives the
v1→internal→structural conversion (test loads the actual MariaDB
ApplicationDefinition fixture on disk, asserts the rule reaches the
structural schema). Empty-input and malformed-JSON paths are also
pinned.

No e2e admission test is added because the apiserver does not yet
enforce the rule — see cozystack/cozystack#2657.

No changelog entry: schema-side annotation is invisible at the apiserver
layer today (UI-only consumer) and the release-cut PR will add the entry
when admission enforcement lands.

## Dependencies

Requires cozystack/cozyvalues-gen#24 to be merged and released, and the
`cozyvalues-gen` binary pin in `.github/workflows/pre-commit.yml` line
50 bumped to the release that includes that PR. Pre-commit CI will fail
until both happen — the stock `v1.3.0` binary silently ignores `##
@immutable` and regenerates artefacts without the rule, producing a
drift against what's committed.

Once that ordering is satisfied, rebase this branch and the pre-commit
job will be green.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* storageClass is now treated as immutable after creation for many
stateful apps (enforced by chart/CRD schema validation).

* **Documentation**
* Added a storage immutability guide explaining the contract and
enforcement.
* Updated chart READMEs and values docs to mark storageClass as
immutable and clarified nodeGroup storage remains intentionally mutable.

* **Tests**
* Added tests validating schema parsing and preservation of immutability
rules.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2639?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-27 10:57:38 +03:00
myasnikovdaniil d3f282cee1 test(e2e): pre-pull platform images via staged-busybox DaemonSet (#2724)
## What this PR does

Pre-pulls every image referenced by `packages/system/kubeovn`,
`packages/system/linstor`, and `packages/system/cert-manager` onto every
node before those HelmReleases install. Cluster-member workloads (OVN
raft, LINSTOR, cert-manager webhook) fail when replicas start at
different times due to per-node image-pull stagger; pre-pulling means
all replicas start with images already cached.

## Mechanics

Renders each chart with `helm template`, then runs a `yq` filter that
walks every PodSpec-shaped object and emits the images of each container
— scoping the output to images the kubelet actually pulls (skips
configmap fields and CRD examples that happen to contain an `image:`
key). Each render is staged through a tmp file so a helm-template
failure trips `set -e` cleanly (cozytest.sh runs `@test` bodies under
`/bin/sh`, which is dash on Ubuntu CI — no `pipefail`).

The pre-pull DaemonSet itself is the hard part: distroless images
(cert-manager and friends) ship no shell and no `/bin/sleep`, so the
obvious `command: ["sleep", "infinity"]` fails with `exec: ENOENT`. The
script stages a statically-linked `busybox:musl` into a shared
`emptyDir` from an initContainer; every prepull container then execs
`/shared/sleep` regardless of what the image itself ships. The glibc
`busybox` tag is unusable — it needs `/lib64/ld-linux-x86-64.so.2` which
distroless lacks.

## Origin

Lifted from #2619 — squash of `test(e2e): pre-pull cert-manager images`
→ POSIX rewrite → cert-manager removal (failed on distroless) →
reintroduction with the busybox helper that solves the distroless
problem. Final state only — the four iterative commits aren't
review-friendly individually.

## Verification

- `sh -n hack/e2e-prepull-images.sh` clean
- All chart-referenced images held Ready under the DaemonSet on local
kind v1.33.1 single-node

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Improved end-to-end test reliability for platform image pre-pulling by
rendering charts separately and consolidating extracted images.
* Simplified and consolidated readiness waits during tenant
configuration for more robust application readiness checks.

* **Chores**
* Enhanced image pre-pulling to reliably handle distroless images using
a staged helper binary and shared volume approach.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2724?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-27 02:55:56 +05:00
Aleksei Sviridkin 5af5962edf feat(apps): mark stateful-app storageClass fields as immutable
Annotate storageClass as immutable across every stateful app whose
chart exposes one — clickhouse, foundationdb, harbor, http-cache,
kafka (Kafka + ZooKeeper), kubernetes (top-level only), mariadb,
mongodb, nats, openbao, opensearch, postgres, qdrant, rabbitmq,
redis, vm-disk. Without coverage across the catalogue the UI would
render storageClass read-only on some apps and editable on others —
a worse mental model than uniform behaviour.

The annotation lands at the schema layer (CEL XValidation rule
'self == oldSelf' on values.schema.json / types.go / embedded
openAPISchema in cozyrds/<app>.yaml). The cozystack aggregated
apiserver does not yet evaluate CEL on application-backed resources;
that is tracked at cozystack/cozystack#2657. The UI
(cozystack/cozystack-ui#6) consumes the marker today.

The PVC-pinning argument is identical for every chart: Kubernetes
fixes a PVC's storageClassName at creation time, so editing this
field on an existing resource never migrated data. The schema
annotation makes that contract explicit.

kubernetes.nodeGroups[name].storageClass is intentionally NOT
annotated: the field is optional and undefaulted, so a strict
'self == oldSelf' rule would block any future attempt to set it on
an existing node group. A richer ratcheting form is needed there —
see docs/storage-immutability.md.

A refactor in pkg/registry/apps/application/rest.go extracts the
inline schema-build path into buildSpecSchema so it can be
unit-tested. Three regression tests in rest_spec_schema_test.go pin
the contract that x-kubernetes-validations survives the
v1->internal->structural conversion (loaded from the actual MariaDB
ApplicationDefinition fixture on disk, not a hand-edited mock).

Each per-app README gains a one-line blockquote pointing to a new
docs/storage-immutability.md that explains the contract and which
consumers enforce it today.

Requires cozystack/cozyvalues-gen with @immutable support — bump the
pre-commit pin in .github/workflows/pre-commit.yml once the release
exists.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-26 23:51:11 +03:00
Myasnikov DaniilandClaude Opus 4.7 12ac2adec9 chore: retrigger CI
Empty commit to fire the Pull Request workflow, which never ran for
this commit because GitHub Actions was down during the original push.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-27 01:24:19 +05:00
Andrei Kvapil 7e92265823 chore(img): refresh Cozystack logo and add icon/stacked variants (#2746)
## Summary

- Replaces `img/cozystack-logo-black.svg` and
`img/cozystack-logo-white.svg` with the updated Cozystack brand mark
(icon + "COZYSTACK" wordmark).
- Adds new layouts so the logo can be used on more surfaces without
re-cropping:
- `cozystack-logo-stacked-{black,white}.svg` — icon stacked over the
wordmark
  - `cozystack-icon-{black,white}.svg` — icon-only mark
- All SVGs are exported on a transparent background, so the same file
works on light and dark surfaces.

## Files

| Path | State | Notes |
|---|---|---|
| `img/cozystack-logo-black.svg` | replaced | horizontal wordmark, black
fill, transparent bg |
| `img/cozystack-logo-white.svg` | replaced | horizontal wordmark, white
fill, transparent bg |
| `img/cozystack-logo-stacked-black.svg` | new | icon over wordmark,
black fill, transparent bg |
| `img/cozystack-logo-stacked-white.svg` | new | icon over wordmark,
white fill, transparent bg |
| `img/cozystack-icon-black.svg` | new | icon only, black fill,
transparent bg |
| `img/cozystack-icon-white.svg` | new | icon only, white fill,
transparent bg |

## Test plan

- [ ] Verify rendering of `cozystack-logo-{black,white}.svg` in the
README on both light and dark GitHub themes.
- [ ] Confirm no other in-repo references to `img/cozystack-logo-*.svg`
need width/height adjustments (existing aspect ratio ~2.8:1 → new ~3:1,
very similar).
2026-05-26 21:22:36 +02:00
Viktoriia Kvapil 6451373251 chore(img): refresh Cozystack logo and add icon/stacked variants
Replace the old "COZYΞTACK" wordmark with the updated brand mark
(icon + "COZYSTACK" wordmark) and add additional layouts:

  - cozystack-logo-{black,white}.svg          horizontal wordmark (replaced)
  - cozystack-logo-stacked-{black,white}.svg  icon over wordmark (new)
  - cozystack-icon-{black,white}.svg          icon only (new)

All variants use a transparent background so they work on both light and
dark surfaces. SVGs are optimized with https://autocrop.cncf.io while
keeping the original padding.

Signed-off-by: Viktoriia Kvapil <bisunviktoria@gmail.com>
2026-05-26 20:54:57 +02:00
Myasnikov Daniil 2ace581117 test(e2e): stage prepull image list to avoid yq pipefail trap
Without pipefail (dash-CI), `yq | hack/e2e-prepull-images.sh` would let a
yq failure produce empty input; prepull-images.sh exits 0 on no-input
and the test passes with pre-pull silently skipped. Stage the yq output
to a tmp file and feed it via stdin so set -e trips on a yq failure.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-26 16:00:30 +05:00
myasnikovdaniil 2fa15ce044 fix: startupProbes for kubevirt-cdi-operator and grafana-operator (#2725)
## What this PR does

Adds `startupProbe` to two operator deployments whose readiness probes
fail repeatedly during a cold cluster install, generating noise and (for
CDI) blocking dependants on slow leader-election:

- **`kubevirt-cdi-operator`** — leader-election against a slow
`kube-apiserver` (Kamaji control plane bringup) consistently takes >30s
on a cold install. The default readinessProbe with 1s `periodSeconds` /
3 `failureThreshold` fires the first time election times out, the
operator pod gets marked NotReady, and dependants block on it.
- **`grafana-operator`** — bootstrap CRD discovery + Helm-managed
Dashboard reconcile loop spikes startup duration past readinessProbe's
default budget under load.

`startupProbe` gives each operator a longer one-time window to reach
steady state; the regular `readinessProbe` then takes over with its
tight per-second cadence to detect genuine ongoing failures.

## Origin

Both commits lifted unchanged from #2619.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Configuration Updates**
  * Added startup health checks to Grafana operator container
  * Added startup health checks to KubeVirt CDI operator container

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2725?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-26 15:47:56 +05:00
Aleksei Sviridkin edb3eeb011 fix(info): use root-host for Keycloak OIDC issuer URL in tenant kubeconfig (#2704)
## What this PR does

The kubeconfig generated by the `info` package for non-root tenants
pointed `--oidc-issuer-url` at `keycloak.<TENANT>.<ROOT-DOMAIN>`, but
Keycloak's ingress and cert live at `keycloak.<ROOT-DOMAIN>` (built from
`_cluster.root-host`). `kubectl oidc-login` then failed TLS verification
against the nginx-ingress default fake cert, so the auto-generated
kubeconfig was unusable for any non-root tenant.

`$host` was being sourced from `_namespace.host` (tenant-specific
subdomain) with a lookup-based override that tried to read
`tenant-root`'s `spec.values.host`. That override has been dead since
#1787 migrated `tenant-root` to `valuesFrom` (Secret-backed) —
`spec.values` is now always empty, so the override never fires.

Source `$host` from `_cluster.root-host` directly, matching what
`packages/system/dashboard/templates/gatekeeper.yaml` already does for
the same Keycloak realm.

Verified with `helm template tenant1 packages/extra/info -n
tenant-tenant1 --set-string _namespace.host=tenant1.example.org
--set-string _cluster.root-host=example.org ...` — issuer URL is now
`https://keycloak.example.org/realms/cozy` regardless of the
per-namespace host.

### Release note

```release-note
fix(info): use root-host for the Keycloak OIDC issuer URL in tenant kubeconfigs so `kubectl oidc-login` succeeds against the real Keycloak cert on non-root tenants
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated kubeconfig generation to consolidate host configuration
handling, now using a single authoritative source instead of multiple
fallback mechanisms.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2704?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-26 13:41:05 +03:00
Aleksei Sviridkin 92000f1618 fix(kubernetes): add config_path patch for containerd 2.x (#2723)
What this PR does
This PR adds support for containerd 2.x CRI config_path configuration,
as the CRI plugin section was renamed from io.containerd.grpc.v1.cri to
io.containerd.cri.v1.images.

It also makes the sed patching regexes more resilient by matching any
quotes (single or double) around the plugin names, which fixes the issue
on Ubuntu 24.04 (Noble) where containerd 2.x uses single quotes:
[plugins.'io.containerd.cri.v1.images'.registry].

Release note
fix(kubernetes): add config_path patch for containerd 2.x

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Broadened Kubernetes cluster bootstrap handling to better support
different containerd plugin layouts, improving compatibility when
configuring container registry paths during cluster initialization.
* **Tests**
* Added regression tests and fixtures to validate the containerd
configuration patching logic and integrated the check into the
Kubernetes test target.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2723?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-26 11:04:39 +03:00
Aleksei Sviridkin 3756f5c459 feat(networking): Gateway API support via Cilium (supersedes #2213) (#2470)
## What this PR does

Replaces ingress-nginx as the default ingress layer for cozystack-native
services with **Cilium Gateway API**, materialised per-tenant via a new
`TenantGateway` CRD reconciled by `cozystack-controller`. Supersedes
#2213 / #2208. Stacked on top of #2468.

## Architecture

The chart renders one `gateway.cozystack.io/v1alpha1 TenantGateway` CR
per tenant that owns its own Gateway. The cozystack-controller
reconciles the actual `Gateway`, per-tenant `Issuer`, `Certificate`
(wildcard or per-listener), and HTTP→HTTPS redirect `HTTPRoute` from
there. Helm does not render `Gateway` or `Certificate` directly — that
prevents the Helm-vs-controller race on `Gateway.spec.listeners` that
route-driven listener additions would otherwise cause.

```
┌── TenantGateway CR (rendered by extra/gateway chart for owning tenants)
│
└──▶ cozystack-controller reconciles:
        ├─ Gateway (with dynamic listeners, label-based allowedRoutes)
        ├─ Issuer (per-tenant ACME account)
        ├─ Certificate(s) — wildcard + per-child SANs in DNS-01, per-listener in HTTP-01
        ├─ HTTPRoute (controller-owned http→https redirect)
        ├─ Namespace label patching (cozy-* system namespaces from spec.AttachedNamespaces)
        └─ Watches HTTPRoute / TLSRoute attachments to materialise listeners
```

## Inheritance: tenants attach by default, opt into their own Gateway

A tenant gets a dedicated Gateway (own LB Service, own LB IP, own
Certificate) only when it explicitly asks via
`tenant.spec.gateway=true`. Every other tenant in the tree publishes
through the Gateway of the nearest ancestor that owns one — same shape
as the existing `_namespace.ingress` inheritance.

The `apps/tenant` chart writes a `namespace.cozystack.io/gateway` label
on each tenant namespace, carrying either the tenant's own name (when
owning a Gateway) or the inherited ancestor name (when inheriting). The
Gateway's listener `allowedRoutes.namespaces.selector` is keyed on that
label, so the same selector admits routes from every namespace pointing
at the owner — descendants and cozy-* system namespaces alike.
cozystack-controller patches the same label onto every namespace in
`tgw.Spec.AttachedNamespaces` and garbage-collects labels it wrote when
an entry is removed from the attach list (annotation-gated GC so labels
written by the apps/tenant chart are never stripped).

In DNS-01 mode, the controller extends the parent's wildcard Certificate
with `<child-apex>` + `*.<child-apex>` SANs per inheriting child, and
adds a `*.<child-apex>` HTTPS listener per child apex referencing the
same cert. Without this expansion the parent's single-level wildcard
cannot match a child route's hostname (`harbor.alice.example.org` is two
labels past the parent's `*.example.org`).

Opt-out into own Gateway is the right choice when: a tenant needs its
own LB IP (DNS pinned, firewall rule), the apex is not derived from the
parent (custom `host` value the ancestor's cert can't cover), or the
tenant wants its own ACME account / cert authority. Otherwise leave
`gateway` unset and inherit.

## End-to-end flow

1. Platform: `gateway.enabled=true` toggles cert-manager solver,
switches per-system-app templates from Ingress to HTTPRoute / TLSRoute.
2. Per tenant: explicit `tenant.spec.gateway=true` renders a
`TenantGateway` CR and triggers the controller to materialise the owner
Gateway. Tenants with the field unset inherit through
`_namespace.gateway` propagation in the tenant chart.
3. Adding a published app under any tenant (owner or inheriting) is just
deploying its HTTPRoute. The controller picks up the hostname, adds a
per-listener Certificate (HTTP-01) or extends the wildcard cert's SANs
(DNS-01), and updates the owner Gateway's listeners.

Defaults stay legacy: `gateway.enabled=false`,
`tenant.spec.gateway=false`. Existing clusters upgrade unchanged.

## External IP allocation — mechanism-agnostic

The per-tenant Gateway's auto-created `LoadBalancer` Service draws its
IP from whatever LB allocator the cluster admin has configured at the
platform layer — same shape as ingress-nginx today. The tenant API stays
mechanism-agnostic — no `gatewayIP` field, no allocator-specific
manifest in the tenant chart. Cozystack itself ships MetalLB installed
but does not render any `IPAddressPool` / `L2Advertisement` /
`BGPAdvertisement` from this chart; admins set up the allocator that
suits their environment (MetalLB pool with L2 / BGP, Cilium LB-IPAM with
announcer, robotlb against a cloud provider, or
`Service.spec.externalIPs` pinning).

If a tenant needs a specific address (DNS already pinned, firewall rule,
etc.), the operator pre-allocates it on the admin side: either
pre-create the Service with `loadBalancerIP` set, or hand the tenant a
reference to a named admin-managed pool. Per-Service IP uniqueness is
the allocator's responsibility — same as for any other LoadBalancer
Service.

## Cert mode: HTTP-01 (default) vs DNS-01 (opt-in)

`publishing.certificates.solver` selects how the controller sources TLS
certs.

**HTTP-01 (default)** — per-app HTTPS listener + per-app Certificate.
New apps require zero platform-side config: deploy the HTTPRoute,
controller does the rest. Inheriting children's hostnames pick up
per-listener certs the same way the owner's own apps do.

**DNS-01 (opt-in)** — single wildcard Certificate (`<apex>` +
`*.<apex>`) covering every published app under the apex, plus per-child
SANs (`<child-apex>` + `*.<child-apex>`) for every inheriting tenant.
Pick this for clusters where many apps share the apex and Let's Encrypt
rate limits matter. The DNS provider account must be able to write TXT
records under every apex level the parent serves — for deeply-nested
inherited children that requires zone delegation or a wide-scope
provider credential.

| `publishing.certificates.dns01.provider` | Required
`publishing.certificates.dns01.<provider>` keys |
| --- | --- |
| `cloudflare` (default) | `cloudflare.secretName`,
`cloudflare.secretKey` |
| `route53` | `route53.region`, `route53.secretName` (and `accessKeyID`
if not on IRSA) |
| `digitalocean` | `digitalocean.secretName` |
| `rfc2136` | `rfc2136.nameserver`, `rfc2136.tsigKeyName`,
`rfc2136.secretName` |

The platform chart wires every provider's keys into `_cluster.dns01-*`;
the per-tenant gateway chart and the cluster-wide `cluster-issuers.yaml`
both read them.

## Strengthens admission coverage on apps.cozystack.io/*

The `cozystack-tenant-host-policy` probe in `gateway.bats` initially
looked like a flake — the apply that was supposed to be VAP-rejected
just succeeded. Investigation traced it to the custom REST handler at
`pkg/registry/apps/application/rest.go`. genericapiserver hands every
storage method a callback into the validating admission chain:
`createValidation` for Create, `updateValidation` for Update,
`deleteValidation` for Delete. `Update` was correctly invoking its
callback since 23e399bd, but `Create` and `Delete` accepted the
parameter and never called it — admission was silently bypassed for both
verbs on every `apps.cozystack.io` resource (Tenant, MariaDB, Postgres,
Kubernetes, …).

This PR wires the missing `createValidation` and `deleteValidation`
calls into Create and Delete. Every ValidatingAdmissionPolicy and
ValidatingWebhook against `apps.cozystack.io/*` now fires on all three
verbs as the API contract requires. Per KEP-3488 aggregated APIServers
are responsible for enforcing admission on their own kinds;
cozystack-api is configured to do so via
`genericoptions.NewRecommendedOptions`, the only thing missing was the
explicit `*Validation` callbacks in the custom REST.

## Security — five guards, grouped by what they defend

Tenants in Cozystack interact with the platform exclusively through
`apps.cozystack.io/*` resources (Tenant, Bucket, Kubernetes, …) served
by `cozystack-api`. Tenant RBAC (`cozy:tenant:*` aggregated to a
RoleBinding in the tenant's own namespace) does not grant write access
to `gateway.networking.k8s.io/*`, core `Namespaces`, or
`cozystack.io/Package`. The protections below split into three groups by
who they defend against:

**Tenant-user-input gates.** Layer 4 (`cozystack-tenant-host-policy`).
`Tenant.spec.host` is the user-supplied field that surfaces as a
security boundary at the hostname layer; gated on every Create / Update
via `cozystack-api`'s admission chain.

**Defense-in-depth.** Layers 1, 2, 5, 7. These do not protect against
tenant-user input (tenants don't hold the relevant RBAC). They guard
against bugs in cozystack-controller / Flux, supply-chain compromise of
an app chart that emits Gateway API resources, and confused-deputy
mistakes by a cluster admin. Fail-closed via `failurePolicy: Fail` +
`validationActions: [Deny]`.

The five layers themselves:

1. **Listener `allowedRoutes` label selector** —
`namespace.cozystack.io/gateway = <owner-tenant-name>`
(controller-written for cozy-* via `AttachedNamespaces`, Helm-written
for tenant namespaces via the tenant chart, both feeding into the same
Gateway selector).
2. **`cozystack-gateway-hostname-policy`** — VAP on `Gateway`
CREATE/UPDATE. Restricts listener hostnames to the namespace's
`namespace.cozystack.io/host` apex.
4. **`cozystack-tenant-host-policy`** — VAP on `Tenant` CREATE/UPDATE.
5. **`cozystack-namespace-host-label-policy`** — VAP on core `Namespace`
CREATE/UPDATE. Immutability gate on the host label.
7. **`cozystack-route-hostname-policy`** — VAP on `HTTPRoute` and
`TLSRoute` CREATE/UPDATE. Closes the hostname-hijack vector across
namespaces under inheritance — a child cannot claim a hostname outside
its own apex.

The previous Layer 3 (`cozystack-gateway-attached-namespaces-policy`)
and Layer 6 (render-time `fail` in cozystack-basics) banned `tenant-*`
entries in `publishing.gateway.attachedNamespaces` on the assumption
that the attach list was the hijack vector. Under inheritance the attach
surface is the label selector, not the static list; the hijack vector is
closed by Layers 4/5/7 independently. Both Layer 3 and Layer 6 are
dropped — kept in numbering above for continuity with the earlier review
threads.

## Foreign-takeover guards

Five reconcile paths refuse to silently rewrite a pre-existing object
that shares the controller-derived name but carries no `OwnerReference`
back to the TenantGateway: `Gateway`, redirect `HTTPRoute`, per-tenant
`Issuer`, wildcard `Certificate`, per-listener `Certificate`. An
operator who hand-pinned a Certificate or Issuer at the controller's
derived name (private CA, manual cert pinning) gets an explicit
`Ready=False/ReconcileError` condition instead of having their config
silently destroyed and the resource re-issued from a different ACME
account.

The namespace-label patching path applies the same discipline:
cozystack-controller only writes / strips
`namespace.cozystack.io/gateway` on namespaces it annotates with
`cozystack.io/gateway-attached-by`. Labels written by the apps/tenant
chart (no annotation) are never touched, so inheritance for tenant
namespaces survives every reconcile.

## Tests

- **Go controller tests** — 50+ cases in
`internal/controller/tenantgateway/`: idempotency, OwnerReference
cascade, multi-parentRef status, HTTP-01↔DNS-01 mode-transition cert
lifecycle, foreign-takeover refusals (Gateway / HTTPRoute / Issuer /
Certificate / per-listener Certificate), DNS-01 provider matrix
(cloudflare / route53 / digitalocean / rfc2136), HTTPS Kinds=[HTTPRoute]
restriction, fail-closed CEL on missing host label, status condition
wiring. Plus, for inheritance: label-based allowedRoutes selector shape,
namespace label patch + garbage-collect via owner annotation, Helm-owned
label safety, DNS-01 wildcard SAN expansion per inheriting child,
per-child wildcard listener materialisation.
- **Admission-chain tests** —
`pkg/registry/apps/application/rest_admission_test.go` pins
`createValidation`, `updateValidation`, `deleteValidation` paths via
sentinel callbacks.
- **helm-unittest** across `extra/gateway` (TenantGateway CR rendering,
both modes), `core/platform` (DNS-01 wiring), `apps/tenant`
(gateway-effective decision matrix including inheritance),
`system/cozystack-basics` (three VAPs + Layer 7 VAP + tenant-root
namespace label including the conditional
`namespace.cozystack.io/gateway: tenant-root` + cozystack-values-secret
`_namespace.gateway` propagation), `system/cert-manager-issuers` (full
DNS-01 provider matrix), `system/cozystack-api` and `system/dashboard`
(Ingress↔TLSRoute/HTTPRoute toggle), `system/monitoring` (Grafana /
Alerta Ingress↔HTTPRoute toggle).
- **e2e bats** — `hack/e2e-apps/gateway.bats` (17 cases) covering
GatewayClass Accepted, Gateway Programmed + LB Service, HTTPRoute
Accepted with status, VAP rejects cross-tenant hostname hijack, VAP
rejects `tenant.spec.host` writes from non-trusted SA
(regression-detector for the admission-chain fix), VAP rejects
`namespace.cozystack.io/host` label changes from non-trusted SA,
route-hostname VAP rejects HTTPRoute hostnames outside the apex, and
three new cases for inheritance — Package admission accepts `tenant-*`
in `attachedNamespaces` (regression guard for the dropped ban), child
tenant without explicit gateway inherits `_namespace.gateway` from a
Gateway-owning parent in lockstep across the namespace label and the
cozystack-values Secret, and the cross-namespace attach end-to-end where
a child tenant's HTTPRoute reaches Accepted on the parent's Gateway via
the inheritance label.
- **CI wiring** — `make test-controllers` target invoked from
`pull-requests.yaml` runs `go test ./internal/...`.

## Backward compatibility

The only always-on change is the flip of `cilium.envoy.enabled` +
`cilium.gatewayAPI.enabled` to `true` (extra `cilium-envoy` DaemonSet,
~100 MB RAM/node idle). Everything else is gated behind
`gateway.enabled=false` / `tenant.spec.gateway=false`. Existing clusters
see no behavioural change until an operator explicitly opts in.

The cozystack-api admission-chain fix is unconditional. Existing
operators who do not define any VAP / admission webhook on
`apps.cozystack.io` resources see no observable change. Operators who DO
have their own webhooks/VAPs against these kinds will start seeing them
fire on Create and Delete (as intended) — review your policies before
merging if you maintain custom admission for `apps.cozystack.io/*`.

Operators who set `publishing.gateway.attachedNamespaces` to include
`tenant-*` entries used to hit a render-time fail or an admission
rejection — now the apply succeeds and the listed tenant namespaces
simply pick up the gateway-attach label alongside the cozy-* system
namespaces. This matches the inheritance model and was kvaps' explicit
ask in the May 25 review.

## Release note

```release-note
- Adds Gateway API support via Cilium, materialised per-tenant via the new `gateway.cozystack.io/v1alpha1 TenantGateway` CRD reconciled by `cozystack-controller`. Opt-in via `publishing.gateway.enabled=true` at the platform level; per-tenant opt-in via `tenant.spec.gateway=true` for dedicated Gateway / LB IP, otherwise the tenant inherits the nearest ancestor's Gateway via a label-based `allowedRoutes` selector (same shape as the existing `_namespace.ingress` inheritance). Includes HTTP-01 (default) and DNS-01 (opt-in, with cloudflare / route53 / digitalocean / rfc2136 providers) cert solvers; DNS-01 extends the parent's wildcard Certificate with per-child apex SANs and adds a `*.<child-apex>` listener per inheriting tenant. Tenant API is mechanism-agnostic for IP allocation — the LoadBalancer Service draws its IP from the cluster's admin-configured allocator (MetalLB / Cilium LB-IPAM / robotlb / externalIPs).
- Strengthens admission coverage on `apps.cozystack.io/*` resources: `cozystack-api` now invokes `createValidation` and `deleteValidation` callbacks on Create and Delete (Update was already wired). Operators with custom ValidatingAdmissionPolicies or webhooks targeting `apps.cozystack.io/*` will see them fire on all three verbs as the API contract requires; review your policies before merging if you maintain such admission.
```
2026-05-26 10:29:30 +03:00
Aleksei Sviridkin c85e473523 test(e2e): inheritance attach asserts controller writes, not Cilium Accepted
The previous e2e check waited for
`HTTPRoute.status.parents[0].conditions[Accepted]=True` after
applying a child tenant's HTTPRoute against the parent's Gateway.
That condition is set by Cilium after the listener's TLS Secret
becomes Ready — and in the e2e cluster every fresh certificate fails
to issue (LE prod refuses every `.example.org` hostname by policy,
visible across alerta/grafana/dashboard/seaweedfs-s3 cluster certs
in cozyreport). So Cilium never set Accepted, the route timed out
3/3 retries, and the failure was environmental — not a regression
in the inheritance code path.

The contract the test exists to pin is cozystack-controller's
behaviour, which has two observable writes:

1. The parent Gateway grows a per-listener HTTPS entry for the
   child route's hostname (proves collectHostnameClaims saw the
   route through the inheritance label and reconcileGateway
   appended the listener).
2. A per-listener Certificate object appears in the parent
   namespace with the route's hostname in spec.dnsNames (proves
   reconcilePerListenerCertificates ran for the same hostname).

Both writes are deterministic and depend only on the inheritance
label + the HTTPRoute existing — not on ACME, not on Cilium TLS
readiness. The test now polls those two objects directly within
shorter, scenario-appropriate timeouts (120s for the listener, 60s
for the Certificate). The route Accepted check is dropped with an
explanation in the test comment so a future cluster with a working
ACME doesn't get a misleading pre-condition added back. The
namespace inheritance label is also verified up front as the
precondition collectHostnameClaims's filter reads.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 23:51:00 +03:00
Aleksei Sviridkin 320172137e fix(tenantgateway): include inheriting child namespaces in hostname claim collection
The HTTP-01 inheritance flow deadlocked at the route-Accepted step:
the apps/tenant chart correctly labelled the child namespace with
`namespace.cozystack.io/gateway=<owner>`, the parent Gateway's
allowedRoutes selector matched by label, but `collectHostnameClaims`
filtered routes by the static set `{tgw.Namespace} ∪ Spec.AttachedNamespaces`
— so a child route in a namespace reached purely via the inheritance
label was silently dropped. No per-listener Certificate, no per-app
HTTPS listener, no hostname match → `Accepted=False,
Reason=NoMatchingListenerHostname` indefinitely.

Caught by the new `child tenant's HTTPRoute attaches to parent's
Gateway via inheritance label` e2e bats — failed 3/3 retries on the
CI run for `afc7765e6`. The Go controller-level
TestReconcile_HTTP01CollectsHostnamesFromInheritingChildNamespaces
test pins the same contract at a faster feedback loop: an HTTPRoute
in a namespace labelled `namespace.cozystack.io/gateway=<owner>` but
NOT listed in Spec.AttachedNamespaces must still cause the
controller to render a per-listener HTTPS listener for its hostname.

The fix extends the `allowed` set in `collectHostnameClaims` to
also include namespaces matched by the gateway label selector, so the
hostname-claim collector and the Gateway's allowedRoutes selector
agree on which namespaces can attach. DNS-01 mode is unaffected — it
returns nil from `collectHostnameClaims` immediately and uses the
separate `collectInheritingChildApexes` path for SAN expansion.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 21:43:51 +03:00
myasnikovdaniil 2cb16d5ae6 refactor(build): standardize image tagging to fix concurrent PR push conflicts (#2711)
## What this PR does

Concurrent PR builds were racing in the OCIR registry. Every package
Makefile tagged its images via `settag`, which on any non-release commit
collapsed to `:latest` — so every image across every PR was pushed to
the same tag, and parallel builds got 409s on the manifest PUT. Two
outliers (`metallb`, `redis-operator`) used hardcoded version tags and
collided across PRs unconditionally.

This PR replaces `settag` and the `TAG=latest` fallback in
`hack/common-envs.mk` with a single `image-tags` macro driven by three
env vars:

| Env | Meaning |
|---|---|
| `IMAGE_TAG` | Build-unique tag, always pushed |
| `PUBLISH_VERSIONED` | Also push `:<component-version>` (when provided)
|
| `PUBLISH_FLOATING` | Also push `:latest` |

Wired into the two workflows:

- **`pull-requests.yaml`** — `IMAGE_TAG=pr-<N>-<sha>`, both `PUBLISH_*`
off. Every PR build pushes a unique tag set; no collisions, no `:latest`
move.
- **`tags.yaml`** — `IMAGE_TAG=<ref_name>`, both `PUBLISH_*` on. Release
output preserved (component versions + `:latest`) plus a uniform
`:<git-tag>` handle on every image for traceability.
- **Local** — defaults to `IMAGE_TAG=dev`, no flags, `PUSH=0`.

All 30 package Makefiles converted to `$(call
image-tags,<repo>,<versioned-tag>)`. Image consumers reference digests
pinned into `values.yaml` by each Makefile, so the tag is decorative for
cluster-side pulls — the build-unique tag is sufficient for E2E.

Also drops the redundant `:<component-version>-<git-tag>` form (cilium,
linstor, matchbox, ubuntu-container-disk, kubevirt-*,
cluster-autoscaler). Nothing in the repo or downstream values.yaml
consumed it; the same information is preserved via the new `:<git-tag>`
plus the existing `:<component-version>` tags on release.

`ubuntu-container-disk` builds a separate image per Kubernetes version
in a loop, so its primary tag became `:<k8s-ver>-<IMAGE_TAG>` to prevent
loop iterations within a single build from pushing to the same tag.

Verified by dry-running `make -n image` across representative packages
(simple, dual-tag, skopeo, flux-push, loop) under all four contexts (PR
/ release / main / dev) — tag sets are unique and behave as expected.

### Release note

```release-note
refactor(build): PR CI now pushes images under a unique `pr-<N>-<sha>` tag per build instead of `:latest`. Concurrent PR builds no longer race in the registry. Release builds keep the existing `:<component-version>` and `:latest` tags and additionally publish a uniform `:<git-tag>` handle across every image. The redundant `:<component-version>-<git-tag>` tag (cilium, linstor, matchbox, ubuntu-container-disk, kubevirt-*, cluster-autoscaler) is no longer published — image references in values.yaml are digest-pinned and were the only legitimate consumers.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Centralized and simplified container image tagging across build
workflows.
* PR builds now use unique image tags derived from PR metadata for
better isolation.
* Added configurable flags to control publishing of versioned and
floating tags, making release tagging behavior more predictable.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2711?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-25 22:16:57 +05:00
Aleksei Sviridkin afc7765e69 ci(external-dns): regenerate cozyrds schema for gatewayAPI description
`cozyvalues-gen` bakes the values.schema.json's
`gatewayAPI.description` into the JSON openAPISchema field of the
external-dns release-definition CRD at
`packages/system/external-dns-rd/cozyrds/external-dns.yaml`. The Q7
commit (18fd155d2) updated the description from 'HTTPRoute only' to
'HTTPRoute and TLSRoute' but missed running `make generate` in
`packages/extra/external-dns/`, so the cozyrds JSON drifted from
the source values.yaml. Pre-commit drift check caught it on CI.

This commit ships the regenerated cozyrds schema only — no
behavioural change. The 'make generate' rule in extra/external-dns
runs cozyvalues-gen and `hack/update-crd.sh`, both already in
CI's drift verification path.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 19:11:52 +03:00
Aleksei Sviridkin 4120f8795e test(e2e): cover child-tenant inheritance attach flow
Replaces the now-obsolete "Package admission rejects tenant-* in
gateway.attachedNamespaces" e2e (the VAP it pinned was dropped in
the inheritance refactor) and adds two new e2e cases that cover
the end-to-end inheritance contract:

* "Package admission accepts gateway.attachedNamespaces with
  tenant-* entries" — regression guard against a future refactor
  re-introducing the gate; the hijack surface is now handled by
  Layers 4/5/7 (Tenant.spec.host VAP, namespace label immutability
  VAP, HTTPRoute hostname VAP), not by entries in attachedNamespaces.

* "child tenant without explicit gateway inherits _namespace.gateway
  from a Gateway-owning parent" — chart-level lockstep: a parent
  tenant with gateway=true produces both the namespace label and the
  cozystack-values Secret pointing at itself, and a child Tenant
  under that parent with the gateway field unset receives the
  parent's name in both places. Plus a negative assertion that the
  child does NOT get its own gateway HelmRelease.

* "child tenant's HTTPRoute attaches to parent's Gateway via
  inheritance label" — full cross-namespace attach: parent owns
  the Gateway, child inherits, an HTTPRoute in the child namespace
  pointing at parent's Gateway reaches Accepted=True once Cilium
  evaluates the label-based allowedRoutes selector. Route hostname
  uses the child's derived apex so Layer 7 admits it.

The Gateway-Programmed wait is intentionally omitted from the
last test — Programmed depends on the LB allocator that ships with
the e2e cluster, while Accepted is a pure spec evaluation on the
parent Gateway's allowedRoutes selector and is the property the
inheritance refactor is meant to deliver.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 19:07:40 +03:00
Aleksei Sviridkin f36240a374 ci: bump cozyvalues-gen pin to v1.5.0 (#2730)
## Summary

Bump the `cozyvalues-gen` binary pinned in
`.github/workflows/pre-commit.yml` from `v1.4.0` to `v1.5.0`.

`v1.5.0` (cozystack/cozyvalues-gen#24) adds support for the `@immutable`
annotation. Without this bump, pre-commit re-runs `make generate` with
the old `v1.4.0` binary on every PR; the old binary silently ignores `##
@immutable` directives and regenerates `values.schema.json` / embedded
`openAPISchema` without the corresponding `x-kubernetes-validations`
entries — which then shows up as committed-vs-regenerated drift on any
PR that tries to use the new annotation.

## No-op for main today

No chart on main currently uses `@immutable` (zero hits for `##
@immutable` under `packages/apps/*/values.yaml`), so this bump
regenerates byte-identical output. The change is purely lifting the
floor for downstream PRs that will start using the annotation.

## Unblocks

- cozystack/cozystack#2639 — apply `@immutable` to `storageClass` across
every stateful Cozystack app (currently fails pre-commit on the v1.4.0
pin).

## Release info

- v1.5.0 release notes:
https://github.com/cozystack/cozyvalues-gen/releases/tag/v1.5.0
- Tag signed by F20B7CCBF00A5CAE8F9F8B8C7988329FDF395282 (Aleksey
Sviridkin)
- Goreleaser uploaded `cozyvalues-gen-linux-amd64.tar.gz` (the artefact
this workflow downloads) along with the rest of the OS/arch matrix.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated development tooling dependencies for build process
improvements.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2730?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-25 19:06:03 +03:00
Aleksei Sviridkin 6e0c2d609d docs(gateway): describe inheritance model and DNS-01 SAN expansion
Replace the 'Inheritance from parent Gateway' known-limitation entry
(which described inheritance as deferred) with:

* A top-of-README 'Inheritance: when to opt in for a separate
  Gateway' section explaining when a tenant owns its Gateway vs
  inherits, and how the namespace.cozystack.io/gateway label is
  written by apps/tenant chart and patched onto cozy-* by the
  controller.
* DNS-01 cert-mode docs spell out SAN expansion for child apexes:
  the wildcard Certificate now carries <child-apex> + *.<child-apex>
  per inheriting tenant, with a per-child *.<child-apex> listener
  on the parent Gateway. Documents the DNS-provider-permission
  requirement for deeply-nested children (ACME challenge writes
  TXT under each apex zone, so zone delegation or wide-scope
  provider creds are needed for grandchild apexes).
* New known-limitation entries: 64-listener cap on the parent
  Gateway under inheritance (with the high-fanout escape via
  tenant.spec.gateway=true), and Cilium sharing-key port-collision
  (multiple per-tenant Gateways still each take their own LB IP
  until ListenerSet ships in Cilium).

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 19:02:22 +03:00
Aleksei Sviridkin 32241afe03 feat(gateway): drop tenant-* ban on gateway-attached-namespaces
Removes two gates that prevented child tenants from attaching to a
parent's publishing Gateway:

1. Render-time fail() in templates/gateway-hostname-policy.yaml that
   refused to render the chart when _cluster.gateway-attached-
   namespaces contained any tenant-* entry.
2. cozystack-gateway-attached-namespaces-policy VAP that enforced
   the same rule at Package CR admission time.

The original rationale — 'a tenant in attachedNamespaces can hijack
the publishing tenant's hostnames' — assumed a static-name
allowedRoutes whitelist where adding tenant-* literally widened the
attach surface for arbitrary hostnames. Under the inheritance
refactor (Stage 1/3/5) the attach surface is governed by the
namespace.cozystack.io/gateway label selector, not by entries in
AttachedNamespaces; and hostname hijack is independently blocked by
three other layers:

* Layer 4 — cozystack-tenant-host-policy VAP on Tenant.spec.host
* Layer 5 — cozystack-namespace-host-label-policy VAP on the
  namespace label
* Layer 7 — cozystack-route-hostname-policy VAP on HTTPRoute and
  TLSRoute hostnames

Those three defend against the hijack vector regardless of what
sits in attachedNamespaces, and they apply uniformly to every
tenant (inheriting or owning a Gateway). The dropped gates added
no extra defence, while their presence broke the inheritance flow
kvaps reasonably asks for in the May 25 review.

Doc count for gateway-hostname-policy.yaml drops from 8 to 6 (one
VAP + one Binding removed); helm-unittest fixtures updated to
match, with a regression-guard test asserting the removed VAP name
no longer appears in any rendered document.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 19:00:31 +03:00
Aleksei Sviridkin 99fe080def feat(tenantgateway): DNS-01 SAN expansion + listeners for child apexes
In DNS-01 mode, the parent's wildcard Certificate now covers every
inheriting child tenant's apex (`<child-apex>` + `*.<child-apex>`
as additional SANs) and the parent Gateway grows one wildcard HTTPS
listener per child apex referencing the same Certificate. Without
the expansion, a child route hostname like `harbor.alice.example.org`
fails to match the parent's listeners — `*.example.org` is a single-
label wildcard and `alice.example.org` is two labels deep — and the
TLS handshake fails the SNI check because the parent cert never
listed the child apex.

The controller discovers inheriting children by listing namespaces
labelled `namespace.cozystack.io/gateway = <tgw.Namespace>` (the
existing inheritance contract from Stage 1/3) and reading their
`namespace.cozystack.io/host` label, which apps/tenant chart writes
to the child apex. Children with no host label are skipped
(defensive); a child host equal to the parent apex is deduplicated
(cert-manager rejects duplicate dnsNames). HTTP-01 mode is
unchanged — its per-listener / per-cert flow already handles child
hostnames without any wildcard.

Listener naming: `https-child-<first-label>-<8-hex>`, where the hex
suffix is sha256(child-apex)[:8] so two children whose first DNS
label collides produce distinct listener names.

Cap: Gateway API hard-caps spec.listeners at 64 entries. The parent
uses 3 slots (http + https + https-apex) leaving ~61 wildcard
children per parent. Operators whose subtree fans out beyond that
must run the high-fanout branch on its own Gateway via
tenant.spec.gateway=true.

Tests (TDD red→green):
* TestReconcile_DNS01WildcardCertCoversInheritingChildApexes pins
  the SAN list contains both `<child-apex>` and `*.<child-apex>`.
* TestReconcile_DNS01WildcardCertDeduplicatesChildApexEqualToParent
  pins the dedupe path (child host == parent apex).
* TestReconcile_HTTP01WildcardCertNeverRendered pins the inverse:
  HTTP-01 mode never renders a wildcard Certificate.
* TestReconcile_DNS01GatewayHasListenerPerChildApex pins the
  listener-side expansion (one `*.<child-apex>` listener per
  inheriting tenant, all referencing the parent's wildcard cert).

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 18:57:32 +03:00
Aleksei Sviridkin 30302eabe4 feat(tenantgateway): switch Gateway allowedRoutes to label selector
The Gateway's HTTPS / TLS-passthrough listeners now match attached
namespaces by the namespace.cozystack.io/gateway label keyed on the
TenantGateway's own namespace, instead of a static
kubernetes.io/metadata.name In [list] whitelist. This restores
inheritance parity with the legacy ingress flow: child tenants of a
Gateway-owning ancestor publish through the ancestor's Gateway by
default — without their own LB IP / Certificate, and without
operator intervention to add each child by name to platform values.

Two writers populate the label:

* apps/tenant chart namespace.yaml — each tenant namespace gets a
  label pointing at the nearest ancestor that owns a Gateway (self
  if owning, inherited otherwise). The tenant Helm chart is the
  source of truth for tenant-tree labels; the controller never
  touches them (annotation-gated GC).
* cozystack-controller (ensureNamespaceLabels) — patches the label
  onto every namespace in tgw.Spec.AttachedNamespaces, so cozy-*
  system namespaces (cert-manager, monitoring, harbor, …) reach the
  publishing Gateway alongside the tenant tree. Tracks its own
  writes via the cozystack.io/gateway-attached-by annotation; on a
  reconcile where an entry is removed from AttachedNamespaces, the
  label is garbage-collected (annotation present + namespace no
  longer expected → strip both label and annotation).

Also adds the gateway-attach label to tenant-root itself via
cozystack-basics/templates/tenant-root.yaml when
_cluster.gateway-enabled is true, so the Gateway in tenant-root's
own namespace passes its own selector for the controller-owned
http→https redirect HTTPRoute.

Tests (TDD, red→green):
* TestReconcile_HTTPSListenerUsesGatewayLabelSelector pins the new
  MatchLabels shape on HTTPS listeners.
* TestReconcile_LabelsAttachedNamespaces pins the patch path: every
  namespace in spec.AttachedNamespaces (plus tgw.Namespace) carries
  namespace.cozystack.io/gateway=<owner> after reconcile.
* TestReconcile_LabelGCRemovesDroppedAttachedNamespaces pins the GC
  path: removing an entry between reconciles strips the label.
* TestReconcile_LabelGCDoesNotStripHelmOwnedLabels pins the safety
  contract: labels lacking the controller annotation (Helm-owned
  tenant labels) are left alone.
* tenant-root_test.yaml adds the positive case for the conditional
  label render under _cluster.gateway-enabled=true.

The HTTP listener (port 80) keeps the narrower static-name selector
— it allows only the tenant namespace plus cozy-cert-manager (for
HTTP-01 ACME challenges) and must NOT accept routes from app
namespaces or child tenants, since unattended routes on the HTTP
listener serve plaintext and leak credentials.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 18:32:20 +03:00
Aleksei Sviridkin 97dff75b5f feat(tenant): inherit _namespace.gateway from parent
Mirror the existing _namespace.ingress inheritance for Gateway API
attachment. A child tenant under a Gateway-owning ancestor publishes
its routes through that ancestor's Gateway by default, without burning
a dedicated LB IP / Certificate per child.

Resolution rule (tenant.gatewayEffective):
  * gateway=true explicit  → own Gateway (LB IP + cert burned)
  * gateway=false explicit → parked (no inherit, no own)
  * gateway unset          → inherit $parentNamespace.gateway

Removes the auto-default for derived-apex tenants — the previous shape
gave every derived-apex child its own Gateway / IP / cert, which scales
linearly with tenant depth and produces ~N×K LB IPs on a tree of N
top-level × K derived-apex tenants. Inheritance collapses that to one
LB IP per Gateway-owning ancestor.

Custom-apex tenants (host explicitly set to a non-derived value) now
fall through to inheritance instead of auto-firing a separate Gateway.
The previous auto-fire conflated 'custom apex' with 'want public
exposure' and would have silently failed TLS termination anyway (the
ancestor's cert does not cover an unrelated apex). The new rule:
explicit gateway=true is the only way to ask for a separate Gateway.

The namespace.cozystack.io/gateway label and _namespace.gateway in the
rendered cozystack-values Secret now both carry the owner-tenant name
— could be self (when this tenant owns one) or any ancestor up the
chain. Apps in the tenant render HTTPRoutes whose parentRefs.namespace
points at that owner.

helm-unittest pinned for the HelmRelease render side. The
cross-namespace attach end of the flow (ReferenceGrants, listener /
SAN expansion on the owner Gateway) lands separately in the
TenantGateway controller.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 18:18:54 +03:00
Aleksei Sviridkin 943441b0c7 ci: bump cozyvalues-gen pin to v1.5.0
v1.5.0 adds the @immutable annotation support
(cozystack/cozyvalues-gen#24). Without the bump, pre-commit re-runs
make generate with the v1.4.0 binary, silently strips any
x-kubernetes-validations entries that downstream PRs add to
values.schema.json / embedded openAPISchema, and reports drift against
the committed artefacts.

This is a no-op for the current main: no chart currently uses
@immutable, so regeneration with v1.5.0 produces identical output to
v1.4.0. Unblocks cozystack/cozystack#2639 (apply @immutable to
storageClass across stateful apps).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:45:27 +03:00
Aleksei Sviridkin 0994821bda feat(monitoring): toggle ingress↔HTTPRoute on _cluster.gateway-enabled
Adds Gateway API equivalents for the two ingress endpoints in the
monitoring chart — Grafana (web UI) and Alerta (alert console) — gated
on the same _cluster.gateway-enabled flag the keycloak / dashboard /
cozystack-api templates already use.

* Grafana's ingress is configured inline via Grafana CRD's spec.ingress
  block; that block is now wrapped with {{ if ne gateway-enabled true }}
  and a standalone HTTPRoute attaching to the tenant Gateway is rendered
  in the alternate branch.
* Alerta uses a standalone Ingress object; the same Ingress↔HTTPRoute
  swap is wired through an if/else.

Both routes attach to Gateway cozystack in the publishing namespace
(default tenant-root), matching the convention established by the rest
of the system charts.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:31:03 +03:00
Aleksei Sviridkin 18fd155d2b feat(external-dns): enable gateway-tlsroute alongside gateway-httproute
The Gateway API source-flag toggle (gatewayAPI=true) already enabled
gateway-httproute discovery. cozystack ships TLSRoute attachments for
SSL-passthrough services (kubernetes control plane API, kubevirt CDI
uploadproxy, kubevirt VM exportproxy) — without gateway-tlsroute in the
external-dns sources list those hostnames are never picked up for DNS
record management.

Adds gateway-tlsroute under the same conditional and updates the value
description / schema / README to reflect that the toggle now covers both
route kinds.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:27:14 +03:00
Aleksei Sviridkin 7c893bc653 docs(gateway): security model as three groups, mechanism-agnostic IP allocation
Security model section split into three groups by who they defend
against, anchoring the framing in the apps.cozystack.io/* tenant API
surface (tenants do not write Gateway API resources directly, so most
of the seven layers are not protecting against tenant-user input):

- tenant-user-input gates: Layer 4 (cozystack-tenant-host-policy)
  via cozystack-api admission. Tenant.spec.host is the user-supplied
  field that surfaces as a security boundary.
- defense-in-depth: Layers 1, 2, 5, 6, 7. Cover bugs in
  cozystack-controller / Flux, supply-chain compromise of an app
  chart, confused-deputy admin mistakes. Fail-closed.
- admin-against-themselves: Layer 3 (cozystack-gateway-attached-
  namespaces-policy). Catches kubectl edit on the platform Package.

Layer 7 wording fix: drop the implication that a tenant user with
HTTPRoute RBAC could exploit the cross-apex hostname surface — that
RBAC is not granted in Cozystack. Reframed as defense-in-depth
against an app chart bug or supply-chain compromise.

External IP allocation section reframed as admin-side concern:
allocator (MetalLB / Cilium LB-IPAM / robotlb / externalIPs) is
configured at the platform layer, tenant API stays
mechanism-agnostic. Per-Service IP uniqueness is the allocator's
responsibility — same shape as for any LoadBalancer Service.

Inheritance Known limitations entry: spell out both upstream gaps
(Gateway API 64-listener cap, Cilium sharing-key inactive on
443/TCP via cilium#42756) so readers understand why per-tenant
Gateway is the only scalable shape today.

publishing.gateway.attachedNamespaces -> gateway.attachedNamespaces
fix: the values block is at top-level, not under publishing.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:07:02 +03:00
Aleksei Sviridkin 039924f59d feat(gateway): align _namespace.gateway propagation with tenant.gatewayEffective
Two related fixes for the gap where the tenant chart's gateway HelmRelease materialises but child apps (harbor, bucket) silently fall back to ingress-nginx:

- packages/apps/tenant/templates/namespace.yaml: switch the
  _namespace.gateway resolution from raw .Values.gateway to
  tenant.gatewayEffective. This is the same helper consulted by
  gateway.yaml (HelmRelease render), so per-tenant Gateway
  materialisation and child-app HTTPRoute / Ingress selection now
  stay in lockstep — including the auto-default for derived-apex
  tenants.
- packages/system/cozystack-basics/templates/cozystack-values-secret.yaml:
  add a conditional gateway field to the tenant-root cozystack-values
  Secret. cozystack-basics is the only writer of _namespace.* for
  tenant-root (apps/tenant chart skips tenant-root via the existing
  ne 'tenant-root' gate). Without this line, tenant-root apps read
  _namespace.gateway as empty and route to ingress even when the
  platform's gateway.enabled flag is on. Conditional on
  _cluster.gateway-enabled='true' so non-Gateway clusters keep the
  empty value.

Test fixture in cozystack-basics/tests/cozystack-values-secret_test.yaml
pins the conditional and the regression-guard that the other four
_namespace keys (etcd, ingress, monitoring, seaweedfs) stay
unconditional.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:07:02 +03:00
Aleksei Sviridkin d2e851f63b feat: Gateway API support via Cilium
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.

The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.

publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.

Security model -- seven independent layers, all admission-time gates
fail-closed:

  1. Listener allowedRoutes namespace whitelist via the
     kube-apiserver-written kubernetes.io/metadata.name label;
     plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
     passthrough listeners restrict allowedRoutes.kinds to
     HTTPRoute / TLSRoute respectively.
  2. cozystack-gateway-hostname-policy VAP on Gateway.
  3. cozystack-gateway-attached-namespaces-policy VAP on Package --
     blocks tenant-* in gateway.attachedNamespaces.
  4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
     writes from non-trusted callers.
  5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
     writes to namespace.cozystack.io/host from non-trusted callers.
  6. Render-time fail in cozystack-basics for tenant-* in
     gateway.attachedNamespaces.
  7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
     scoped to tenant-* namespaces.

Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).

Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.

Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.

Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.

Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.

Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.

Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:07:02 +03:00
Aleksei Sviridkin cb2f606c76 fix(csi): verify VMI Ready after kubevirt-csi Publish to surface stuck-PVC failures (#2659)
## What this PR does

Stops `ControllerPublishVolume` from silently reporting success when the
volume is in VM spec but the hotplug attach never actually completed.

### Root cause

Upstream's fast path (`EnsureVolumeAvailableVM`) checks only
`vm.Spec.Template.Spec.Volumes`. If a prior Publish wrote that entry but
timed out waiting for VMI Ready (e.g. the infra PVC is stuck
`ClaimPending` because LINSTOR cannot autoplace), the next Publish
short-circuits to success. external-attacher sets `Attached=true`, but
QEMU was never sent `device_add`. The tenant kubelet's NodeStageVolume
then fails with:

```
rpc error: code = Unknown desc = couldn't find device by serial id
```

Observed in production. Sequence from kubevirt-csi-driver controller
logs:

```
16:53:13  /ControllerPublishVolume called
16:55:13  volume failed to be ready in time (2m), context deadline exceeded
16:55:14  /ControllerPublishVolume called  (retry)
16:55:14  Volume X already attached to VM Y - skipping hot-plug
16:55:14  /ControllerPublishVolume returned with response: {}
```

VMI volumeStatus during that window: `phase=Pending message="PVC is in
phase ClaimPending"`. VolumeAttachment then goes `Attached=true`; tenant
kubelet surfaces "couldn't find device by serial id".

### Fix

After upstream Publish returns success for a hotplug volume, the wrapper
re-reads VMI status and checks `Phase == VolumeReady` for this
DataVolume. If it is not Ready (or not present), the RPC returns
`codes.Unavailable` with the upstream reason, so external-attacher keeps
retrying instead of declaring the volume attached. The underlying
provisioning failure surfaces at the CSI layer instead of becoming a
confusing kubelet-side "device not found by serial id".

This is a wrapper-side workaround for a defect in upstream
`kubevirt.io/csi-driver`; the structural fix belongs upstream (tighten
the fast path to require VMI Ready) and will be filed separately. The
wrapper guard is safe to keep as defense-in-depth even after the
upstream fix lands.

Relates to #2634.

### Release note

```release-note
fix(csi): kubevirt-csi-driver Publish now verifies VMI Ready before reporting attached, surfacing stuck-PVC failures at the CSI layer instead of as "couldn't find device by serial id" on the tenant kubelet
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved hotplug volume publishing: non‑NFS volumes are now verified
on the VM before being marked published; if the volume is missing or not
ready, publishing returns unavailable, improving reliability for
hotplugged storage.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2659?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-25 17:04:43 +03:00
Aleksei Sviridkin 60357eb4a6 fix(csi): route RWX Block volumes to upstream hotplug detach (#2658)
## What this PR does

Fixes routing in `WrappedControllerService` so RWX **Block** volumes
(KubeVirt live-migration disks) are no longer treated as NFS during
unpublish and expand.

### Root cause

`ControllerUnpublishVolume` selected the NFS cleanup branch based on
`hasRWXAccessMode(pvc)` alone. RWX Block matched and went down the NFS
path — which just deletes a `CiliumNetworkPolicy`. For a block volume
that CNP does not exist, so `removeCNPOwnerReference` returns
successfully without doing anything, and the function reports success
without ever calling upstream's hotplug detach. The DataVolume entry
stays in `VM.spec.template.spec.volumes` permanently. When the same PVC
later attaches to a different VM, linstor-csi's anti-split-brain check
sees pods from two different VMs referencing the same RWX block PVC and
blocks the attach with:

```
RWX block volume <ns>/<pvc> is being used by pods from different VMs (<vm-a> and <vm-b>);
this is not supported - RWX block volumes with allow-two-primaries are only for live migration of a single VM
```

`ControllerPublishVolume` was unaffected because it routes on the
explicit `nfsVolumeKey` from `VolumeContext`.
`ControllerUnpublishVolume` does not receive `VolumeContext` and has to
infer from the PVC, and the previous inference was incomplete.

### Fix

`isNFSVolume(pvc)` now requires `ReadWriteMany` access mode **and**
`VolumeMode == Filesystem` (always set by our NFS `CreateVolume` path).
RWX Block correctly delegates to upstream `ControllerUnpublishVolume`,
which removes the entry from VM spec via `RemoveVolumeFromVM`. The same
predicate replaces the previous access-mode check in
`ControllerExpandVolume`, which was also clearing
`NodeExpansionRequired` for RWX Block — block volumes still need in-VM
disk expansion.

Fixes #2634.

### Release note

```release-note
fix(csi): route RWX Block volumes in kubevirt-csi-driver to upstream hotplug detach so VM spec entries are cleaned up on unpublish
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* More accurate detection of network-attached (NFS) vs block volumes,
preventing incorrect cleanup of block volumes.
* NFS volumes continue to receive special cleanup; non-NFS volumes now
follow upstream cleanup paths.

* **Refactor**
* Node-side expansion logic refined to base decisions on improved NFS
classification (RWX + filesystem), reducing expansion errors.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2658?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-25 16:58:55 +03:00
Myasnikov DaniilandClaude Opus 4.7 e42a5828fe refactor(build): skip duplicate versioned tag when it matches IMAGE_TAG
On release builds IMAGE_TAG = github.ref_name = vX.Y.Z, and Makefiles
that pass v$(COZYSTACK_VERSION) as the versioned arg evaluate to the
same vX.Y.Z on the tag commit. The macro then emitted
`--tag :vX.Y.Z --tag :vX.Y.Z --tag :latest`. docker buildx dedupes
internally so there is no functional impact, but the build logs carried
the redundant flag for every affected package.

Replace the empty-string guard with filter-out so the versioned tag is
also skipped when it would duplicate IMAGE_TAG.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-25 17:49:16 +05:00
Myasnikov DaniilandClaude Opus 4.7 1bcaa3de54 fix(flux-plunger): correct common-envs.mk include path
The include pointed at ../../../scripts/common-envs.mk, but that
directory does not exist — only hack/common-envs.mk does. flux-plunger
is not part of the root `make build`, so the broken path went unnoticed
until the new image-tags macro made the recipe depend on it.

Without the include, `make -C packages/system/flux-plunger image` fails
immediately with "No such file or directory".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-25 17:45:44 +05:00
Myasnikov Daniil 6c8a67db3d fix(talos): fail fast on skopeo copy errors in image-talos
The image-talos recipe chains skopeo copy with `;` in a single shell.
Without `set -e`, the recipe's exit status is the last command's, so a
failed copy in the middle of the chain can be masked by a later
successful command — or worse, by a false-branch `if`/`fi` (which exits
0 and would silently hide a push failure on PR builds where both
PUBLISH_* flags are 0).

Prepend `set -e;` to abort the recipe on the first failure. Also quote
"$SRC" defensively.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-25 12:58:08 +05:00
Myasnikov DaniilandClaude Opus 4.7 000d5165ca test(e2e): pre-pull platform images via staged-busybox DaemonSet
Cluster-member workloads (OVN raft, LINSTOR, cert-manager) fail when
replicas start at different times due to per-node image-pull stagger.
This test pre-pulls the images that those charts reference onto every
node before the platform HelmReleases install, so all replicas can
start with images already cached.

Mechanics:

- Source images directly from the rendered charts (kubeovn, linstor,
  cert-manager) via `helm template` + a `yq` filter that walks every
  PodSpec-shaped object and emits each container's image. Scopes the
  output to images the kubelet actually pulls — skips configmap fields
  and CRD examples that happen to contain an `image:` key. Each render
  is staged through a tmp file so a helm-template failure trips `set -e`
  cleanly without needing `pipefail` (cozytest.sh runs @test bodies
  under /bin/sh, which is dash on Ubuntu CI), and without capturing
  rendered YAML into a shell variable that `set -x` would expand and
  balloon the CI trace.

- The pre-pull DaemonSet is the hard part: distroless images
  (cert-manager and friends) ship no shell and no /bin/sleep, so the
  obvious `command: ["sleep", "infinity"]` fails with exec ENOENT.
  Stage a statically-linked busybox:musl into a shared emptyDir from
  an initContainer; every prepull container then execs /shared/sleep
  regardless of what the image itself ships. The glibc busybox tag is
  unusable here — it needs /lib64/ld-linux-x86-64.so.2 which distroless
  lacks. Loop is written in POSIX shell (no bash arrays) for the same
  /bin/sh-is-dash reason.

Verified locally on kind v1.33.1 single-node: every chart-referenced
image is held Ready under the DaemonSet before the platform HelmRelease
applies.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-25 11:56:30 +05:00
Myasnikov Daniil 9118cb3cc5 fix(grafana-operator): add startupProbe to survive slow startup under load
Install Cozystack failed because grafana-operator pod looped in
CrashLoopBackOff under concurrent helm-install load (Flux v2.8
helm-controller dispatching every system HR in parallel + kstatus
walking all applied resources). The chain that fell over:

  grafana-operator pod  liveness probe :8081/healthz fails 3x in 30s
    -> monitoring-rd     dependency 'cozy-grafana-operator/grafana-operator'
    -> tenant-rd         dependency 'cozy-system/monitoring-rd'
    -> cozystack-basics  dependency 'cozy-system/tenant-rd'
                         (cozystack-basics creates the cozy-public ns)
    -> kubevirt-cdi      RoleBinding cdi-clone-dv: namespace
                         "cozy-public" not found
    -> vm-disk-rd, dashboard, flux-plunger   all wait, install times out

Pod log shows the manager starts at T+0, controllers spin up workers,
folder/dashboard/datasource sync completes by T+10s — then SIGTERM at
T+27s. Liveness probe is the upstream chart default (delay=0s,
timeout=1s, period=10s, failureThreshold=3) which gives ~30s grace,
not enough on a fresh cluster where controller-runtime's HTTP probe
server is still binding while kubelet starts polling immediately.

Same shape as the kubevirt-cdi-operator commit already in this branch
(37b06e21). Mirror the fix:

- Edit deployment.yaml on the vendored chart to add a startupProbe
  (30 x 5s = 150s grace, /healthz on :8081). Once it passes once,
  liveness/readiness keep the existing strict config.
- Stash the change as patches/startupProbe.diff and re-apply it from
  the Makefile update target after the curl|tar upstream fetch, so
  the next chart bump does not silently regress.

The patch uses package-relative paths (a/charts/...) so `patch -p1`
from the package dir resolves correctly — repo-relative paths would
need patch's fuzzy fallback which is unreliable across distros.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 40949c80b1)
2026-05-25 11:54:45 +05:00
Myasnikov Daniil 43548d98eb fix(kubevirt-cdi-operator): add startupProbe to survive slow leader election
cdi-operator's controller-runtime starts the health probe HTTP server
(:8444) only after the leader election lease is acquired. On a fresh
cluster the lease can take 15–20 s, exceeding the liveness probe's
5s + 3×10s grace under load. Kubelet kills the container before the
probe can succeed; the pod loops in CrashLoopBackOff, CDI never
reconciles its CR, datavolumes.cdi.kubevirt.io stays unavailable, and
every downstream test that creates a DataVolume (tenant kubernetes
worker VMs, vm-instance) fails.

Add a startupProbe that hits /healthz with 30×5s grace (150 s total).
Once it succeeds once, liveness/readiness take over with their existing
strict config. No effect on healthy clusters; only relaxes the
ramp-up window.

Implemented as a patches/*.diff applied from the Makefile update
target, mirroring the fluxcd-operator pattern. Survives the next
upstream regen; if upstream restructures the area around the
readinessProbe/resources boundary the patch will fail loudly so the
maintainer notices.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 8f5d45e0eb)
2026-05-25 11:54:45 +05:00
Aleksei Sviridkin f874ba93b8 test(kubernetes): functional regression test for config_path seds
The helm-unittest assertions added in the previous commit cover the
structural shape of the sed lines (they must exist in
preKubeadmCommands). They do not catch the failure mode this PR exists
to fix: a future containerd release renames the registry plugin
section, the sed silently stops matching, and per-registry mirroring
under /etc/containerd/certs.d quietly breaks.

Add a bash test that:

1. Drift-checks the script's copy of the two sed lines against the
   verbatim strings embedded in templates/cluster.yaml, so the test
   cannot diverge from the template without failing.

2. Runs both seds against minimal fixtures of containerd 1.x and 2.x
   default configs (tests/fixtures/containerd-{1,2}x.toml) and asserts
   that config_path is rewritten exactly once in each. If containerd
   3.x renames the section again, the fixture stops matching, the test
   fails, and CI surfaces the regression instead of letting it ship
   silently.

Wired into the package's `make test` target, which is already invoked
in CI alongside helm-unittest.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-24 13:58:34 +03:00
Aleksei Sviridkin 47953c48b8 test(kubernetes): cover containerd config_path patching for both plugin schemas
Add helm-unittest assertions for both sed lines in preKubeadmCommands so
that a future regression to a single-regex form, which silently fails
against containerd 2.x where the plugin section was renamed and quoted
with single quotes by default, is caught at template render.

Add a Helm comment above the two seds explaining the rename from
io.containerd.grpc.v1.cri (containerd 1.x) to io.containerd.cri.v1.images
(containerd 2.x), so future readers understand why both lines exist and
that the wrong-version one is intentionally a no-op on each image
version.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-24 13:37:21 +03:00
Elaugaste dbcd09a0fd fix(kubernetes): add config_path patch for containerd 2.x
This adds support for containerd 2.x CRI config_path configuration, as the CRI plugin section was renamed from io.containerd.grpc.v1.cri to io.containerd.cri.v1.images. It also supports both single and double quotes around plugin names.

Assisted-By: Gemini <noreply@google.com>
Signed-off-by: Elaugaste <elaugaste@gmail.com>
2026-05-24 13:57:11 +06:00
Timur Tukaev 1810263f9d ci: pin Scorecard workflow actions by SHA (#2721)
## What this PR does

Fixes the OpenSSF Scorecard workflow added in #2720.

The initial workflow used floating major-version tags (`@v2`, `@v4`,
`@v3`). `ossf/scorecard-action` does not publish a floating `v2` tag —
only specific patch releases — so the first scheduled run failed:

> Unable to resolve action `ossf/scorecard-action@v2`, unable to find
version `v2`

This PR pins every action by its full commit SHA with a trailing comment
showing the human-readable version. This also satisfies the
**Pinned-Dependencies** Scorecard check itself.

- `actions/checkout@de0fac2` (v6.0.2)
- `ossf/scorecard-action@4eaacf0` (v2.4.3)
- `actions/upload-artifact@043fb46` (v7.0.1)
- `github/codeql-action/upload-sarif@03e4368` (v3)

### Release note

```release-note
NONE
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated CI/CD infrastructure dependency versions to maintain security
and stability.

**Note:** This release contains no user-facing changes.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2721?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-23 09:26:33 +05:00
Timur Tukaev 688f73f8f0 ci: pin Scorecard workflow actions by SHA
The initial Scorecard workflow used floating major-version tags
(@v2, @v4, @v3). ossf/scorecard-action does not publish a floating
v2 tag — only specific patch releases — so the first scheduled run
failed with `Unable to resolve action ossf/scorecard-action@v2`.

Pin every action by its full commit SHA with a trailing comment
showing the human-readable version. This also satisfies the
`Pinned-Dependencies` Scorecard check itself.

- actions/checkout@de0fac2 (v6.0.2)
- ossf/scorecard-action@4eaacf0 (v2.4.3)
- actions/upload-artifact@043fb46 (v7.0.1)
- github/codeql-action/upload-sarif@03e4368 (v3)

Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io>
2026-05-23 09:25:31 +05:00
Timur Tukaev 3a8d1c5090 ci: add OpenSSF Scorecard workflow (#2720)
## What this PR does

Adds a GitHub Actions workflow that runs the [OpenSSF
Scorecard](https://github.com/ossf/scorecard) on this repository.

### Schedule

- Weekly on Mondays at 06:00 UTC
- On branch protection rule changes
- On pushes to `main`
- Manual dispatch

### Where results go

- **Public**
[`scorecard.dev`](https://scorecard.dev/viewer/?uri=github.com/cozystack/cozystack)
— transparency for adopters and reviewers (CNCF DD).
- **Code-scanning dashboard** (Security tab) — for in-team triage of
individual findings.
- SARIF artifact retained for 5 days for local inspection.

### Token permissions

Default `read-all`, with the minimum required scopes elevated only
inside the analysis job:
- `security-events: write` — upload to code scanning
- `id-token: write` — publish to `scorecard.dev` via OIDC, no long-lived
secret
- `contents: read`, `actions: read`

### Why no README badge yet

This PR enables the workflow but **does not add a badge to the README**.
Once the first scans have completed and quick-win improvements (token
permissions across other workflows, dependency pinning in workflows,
etc.) have landed, a follow-up PR will add the badge.

### Context

- Tracked under the project's CNCF Incubation readiness work and the
in-progress security self-assessment.
- Companion to the OpenSSF Best Practices passing badge
(https://www.bestpractices.dev/en/projects/10177) — Scorecard adds
machine-verifiable, continuously-updated evidence; Best Practices
captures self-attested controls.

### Release note

\`\`\`release-note
NONE
\`\`\`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Added automated security scanning to the development process to
continuously monitor codebase health and security posture.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2720?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-23 09:22:32 +05:00
Timur Tukaev 2396d08ee5 ci: add OpenSSF Scorecard workflow
Runs the OpenSSF Scorecard action on:
- weekly schedule (Mon 06:00 UTC)
- branch protection rule changes
- pushes to main
- manual dispatch

Publishes results to:
- the public OpenSSF Scorecard API (scorecard.dev) for transparency
- the GitHub code-scanning dashboard (Security tab) for triage

No README badge is added yet — the badge will be introduced in a
follow-up PR once the first scans have run and any quick-win
improvements (token permissions, dependency pinning) have landed.

Aligns with the project's CNCF Incubation readiness work and the
security self-assessment in progress.

Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io>
2026-05-23 09:12:06 +05:00
Timur Tukaev b689613151 chore(maintainers): mark @lllamnyp as Core Maintainer (#2719)
## What this PR does

Updates @lllamnyp's responsibility in `MAINTAINERS.md` from
`Etcd-operator Lead` to `Core Maintainer`.

The etcd-operator is hosted in the separate
[aenix-io/etcd-operator](https://github.com/aenix-io/etcd-operator)
repository — its leadership scope is tracked there. In
`cozystack/cozystack`, @lllamnyp's role is a project-level Core
Maintainer, which this change reflects.

### Release note

​```release-note
NONE
​```
2026-05-23 07:06:26 +05:00
Timur Tukaev b7c6f2793a docs(contributor-ladder): document Emeritus technical offboarding and reactivation (#2718)
## What this PR does

Expands the **Stepping Down / Emeritus Process** section in
`CONTRIBUTOR_LADDER.md`:

- Adds an explicit PR-based mechanism for moving a maintainer's row from
`Active` to `Emeritus` in `MAINTAINERS.md`.
- Adds a five-business-day technical offboarding checklist: GitHub team
membership, org role, direct collaborator entries, `CODEOWNERS`, Actions
secret rotation, internal channels.
- Defines a path for an Emeritus maintainer to return to active status.

This codifies the offboarding flow actually exercised in #2717 and
removes ambiguity for future maintainer lifecycle events. Useful
evidence for the CNCF Incubation Due Diligence review (Governance
section, maintainer-lifecycle criteria).

### Release note

​```release-note
NONE
​```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated contributor documentation with details on the Emeritus
transition process, including offboarding procedures and reactivation
guidelines.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2718?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-23 07:06:02 +05:00