mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
91aae36da313474e4e795104f40a9cf995271c6a
3832
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Myasnikov Daniil
|
91aae36da3 |
feat(platform): NOTES.txt warns when no-delete VAP is silently skipped
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: on a pre-1.30 cluster the capability gate (`.Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ ValidatingAdmissionPolicy"`) evaluates to false and the policy + Binding render to nothing. There was no signal in `helm install`/`helm upgrade` output that the guardrail had been silently skipped, so operators on older clusters lost the protection without any way to discover that from the install log. Compounded the integration-test gap @lllamnyp called out in blocker 3. Add packages/core/platform/templates/NOTES.txt that renders a multi-line WARNING whenever the capability is absent: states the guardrail is NOT active, explains why (1.30+ required), notes that the label is still stamped on every protected object so the guardrail becomes active automatically on the next upgrade after a cluster bump, no migration step required. Add packages/core/platform/tests/notes_capability_gate_test.yaml: positive case asserts the warning text + the 1.30 hint + the label string render on a chart-level `capabilities.apiVersions: []`. Negative case asserts the warning is absent when admissionregistration.k8s.io/v1/ ValidatingAdmissionPolicy is in the capability list. Verified the positive case fails when the warning text is removed from NOTES.txt. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
919a8a9d05 |
docs(platform): clarify sequential self-protection on no-delete VAP
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: both the ValidatingAdmissionPolicy and the ValidatingAdmissionPolicyBinding carry the no-delete label so each individually requires the bypass dance, but the labeling is symmetric, not truly bilateral. Removing the label from the Binding and deleting it first leaves the VAP labeled but unenforced, so the VAP itself is then deletable without the bypass — and vice versa. That is by design: the guardrail is operational guidance, not adversarial defense against an operator with cluster-admin who is determined to remove it. Add a paragraph in the template comment header so a future reader does not waste time wondering why the symmetric self-protection isn't truly bilateral. Forward-references the whole-chart teardown recipe (make unprotect) so the comment links to the documented escape hatch. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
155c011dd5 |
test(platform): e2e check that no-delete VAP denies + bypass works
Address review feedback from @lllamnyp on packages/core/platform/templates/
deletion-protection.yaml: the VAP contract had no integration coverage.
A unit test that greps the rendered manifest is the wrong shape — the
contract is "the api-server denies a labeled DELETE with the documented
message" and that needs a real apiserver in the loop.
Add a post-install bats test to hack/e2e-install-cozystack.bats that runs
once cozystack is installed on the e2e cluster:
1. Skips on clusters that pre-date Kubernetes 1.30 (no VAP API).
2. Asserts the cozystack-version ConfigMap actually carries
platform.cozystack.io/no-delete=true — precondition, so the deny
assertion below would not misreport on a regressed binding.
3. Asserts `kubectl delete configmap cozystack-version -n cozy-system`
exits non-zero AND its error message contains the documented
"Deletion blocked: ... platform.cozystack.io/no-delete=true"
substring AND the `--namespace` bypass hint.
4. Confirms the ConfigMap survives the rejected delete.
5. Removes the label, deletes (must succeed), recreates the ConfigMap
with the original data.version and the label so the cluster ends
the test in the same state it started.
This single test catches every regression the PR is meant to prevent:
capability gate inverted, binding objectSelector mistyped,
validationActions flipped Deny→Warn, expression flipped false→true,
label-key drift between the binding and the manifests, and the bypass
docs missing --namespace.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
e34eeb0ee7 |
fix(platform): document --namespace bypass and add VAP unit tests
Address review feedback from @lllamnyp on packages/core/platform/templates/
deletion-protection.yaml:
The bypass instructions in both the template-author-facing comment and the
user-facing CEL message did `kubectl label <kind> <name>` with no
--namespace flag. Several protected objects are namespaced (cozystack-version
ConfigMap, cozystack-packages Repository, tenant-root HelmRelease), so an
operator who copies the documented command verbatim against their default
context gets a "not found" error and concludes that the bypass mechanism is
broken. Updated both call sites to show --namespace <ns> with an explicit
"(omit for cluster-scoped resources)" caveat.
The reviewer also flagged that the VAP contract is otherwise unasserted by
unit tests. Add packages/core/platform/tests/deletion_protection_test.yaml
covering:
- both the VAP and the Binding carry platform.cozystack.io/no-delete=true
on themselves (the self-protection contract)
- failurePolicy is intentionally absent on the VAP
- the deny message contains the bypass command WITH --namespace and the
cluster-scoped caveat (regression catch for this fix; verified failing
when --namespace is dropped from the template)
- the Binding's objectSelector + validationActions: [Deny] still match
the no-delete label
Wire packages/core/platform/Makefile's test target to helm unittest so
hack/helm-unit-tests.sh actually runs the new suite in CI (the existing
tests/ files were orphaned without it).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
ecdf3f3a15 |
fix(platform): drop redundant failurePolicy on no-delete VAP
Address review feedback from @lllamnyp on packages/core/platform/templates/ deletion-protection.yaml: the VAP's validation expression is the literal `false` — no parameters, no variables, no references — so CEL evaluation cannot fail and the api-server never has occasion to apply failurePolicy. The previous `failurePolicy: Fail` line was a no-op masquerading as operational config. Drop the field (the default is Fail anyway in admissionregistration.k8s.io/v1) and document in the template comment why it is intentionally absent, so the next reader does not re-add it on the assumption that it was forgotten. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
052f070b63 |
fix(platform): migration 42 uses a single declarative apply
Address review feedback from @lllamnyp on packages/core/platform/images/ migrations/migrations/42: the previous version did `kubectl label --overwrite` then `kubectl apply` on a manifest with no labels, relying on `kubectl apply`'s 3-way strategic merge to leave the imperatively-set label intact. That works in the steady state but is fragile — the contract is "the label is preserved because it is never in last-applied-configuration" — and it is inconsistent with the helm template at templates/cozystack-version.yaml, which bakes the label inline. Replace the two-step dance with one declarative apply that includes the deletion-protection label, matching the chart template. One source of truth, no last-applied-merge subtlety. The script remains idempotent (apply is a no-op once the ConfigMap matches). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
5d263a1e25 |
test(crdinstall): assert no-delete label survives Install + merge
Address review feedback from @lllamnyp on internal/crdinstall/install.go: the deletion-protection label loop was untested — TestInstall_appliesAllCRDs only checked that Install returned no error, so a refactor dropping the loop would have shipped green and silently disabled VAP coverage on every Cozystack CRD. - Extend TestInstall_appliesAllCRDs to read each applied CRD back from the fake client and assert platform.cozystack.io/no-delete=true is present. - Add TestInstall_preservesExistingLabels: feeds in a CRD whose manifest already carries app.kubernetes.io/managed-by=foo, asserts both that label and the deletion-protection label end up on the applied object. Locks down the merge contract in install.go:92-101 (the GetLabels()==nil branch is still covered by the first test). Verified the new assertions fail when the label-stamping loop is removed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
c7227bc666 |
feat(platform): migration 42 backfills no-delete label on cozystack-version
Address review feedback from coderabbitai on
packages/core/platform/templates/cozystack-version.yaml:10:
existing clusters skipped the label because the template's
`lookup`+`if not $configMap` guard never re-renders the ConfigMap on
upgrade (the skip is intentional — it prevents helm from clobbering the
data.version that the migration controller writes from its pre-upgrade
hook).
A one-shot migration is the correct mechanism here:
- Migration 42 in packages/core/platform/images/migrations/migrations/42
runs `kubectl label --overwrite configmap cozystack-version
platform.cozystack.io/no-delete=true` and stamps version=43. Slot 42
rather than 40 because main already occupies 40 (dashboard CRD
cleanup) and 41 (kubernetes ephemeralStorage→diskSize rename).
- Bump migrations.targetVersion 42 -> 43 so the hook fires on the
next platform upgrade.
Idempotent: --overwrite is a no-op when the label is already at the
target value, MIGRATION_DRY_RUN=1 prints without applying, and the
script aborts before stamping if the kubectl label call fails.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
e5dd5681c3 |
feat(platform): self-protect the deletion-protection VAP and Binding
Address review feedback from gemini-code-assist on packages/core/platform/templates/deletion-protection.yaml:15: label the ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding themselves with platform.cozystack.io/no-delete=true so the guardrail guards itself. A direct kubectl delete on either resource now returns the same denial as deletion of any other protected object; the same two-step label-then-delete bypass applies when intentional removal is needed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
43ef456d84 |
feat(platform): gate deletion-protection VAP on Kubernetes 1.30+ capability
Address review feedback from gemini-code-assist on packages/core/platform/templates/deletion-protection.yaml:42: wrap the ValidatingAdmissionPolicy and Binding in a Helm capability check so the template stays renderable on clusters that pre-date 1.30 (when ValidatingAdmissionPolicy graduated to GA). The check targets the specific Kind, not the API group/version — admissionregistration.k8s.io/v1 itself has existed since 1.16 for ValidatingWebhookConfiguration, so the group/version alone is not a meaningful capability gate. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
0c17d1870c |
feat(platform): add deletion-protection guardrail via ValidatingAdmissionPolicy
Block accidental DELETE on critical platform objects (tenant-root, cozy-system, LinstorCluster, CRDs, ClusterIssuers, cozystack-version ConfigMap, cozystack-packages OCIRepository, tenant-root HelmRelease) via a label-scoped ValidatingAdmissionPolicy. No webhook DaemonSet, Service, TLS, or image — the API server evaluates the CEL expression in-process. Requires Kubernetes 1.30+. Bypass for legitimate deletes is two steps: kubectl label <kind> <name> platform.cozystack.io/no-delete- kubectl delete <kind> <name> CRDs installed dynamically by the operator get the label stamped in internal/crdinstall/install.go before the apply call. Release note: Added deletion-protection guardrail (ValidatingAdmissionPolicy) blocking DELETE on cozystack platform objects labeled platform.cozystack.io/no-delete=true. Bypass by removing the label first. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
92000f1618 |
fix(kubernetes): add config_path patch for containerd 2.x (#2723)
What this PR does This PR adds support for containerd 2.x CRI config_path configuration, as the CRI plugin section was renamed from io.containerd.grpc.v1.cri to io.containerd.cri.v1.images. It also makes the sed patching regexes more resilient by matching any quotes (single or double) around the plugin names, which fixes the issue on Ubuntu 24.04 (Noble) where containerd 2.x uses single quotes: [plugins.'io.containerd.cri.v1.images'.registry]. Release note fix(kubernetes): add config_path patch for containerd 2.x <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Broadened Kubernetes cluster bootstrap handling to better support different containerd plugin layouts, improving compatibility when configuring container registry paths during cluster initialization. * **Tests** * Added regression tests and fixtures to validate the containerd configuration patching logic and integrated the check into the Kubernetes test target. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2723?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
3756f5c459 |
feat(networking): Gateway API support via Cilium (supersedes #2213) (#2470)
## What this PR does
Replaces ingress-nginx as the default ingress layer for cozystack-native
services with **Cilium Gateway API**, materialised per-tenant via a new
`TenantGateway` CRD reconciled by `cozystack-controller`. Supersedes
#2213 / #2208. Stacked on top of #2468.
## Architecture
The chart renders one `gateway.cozystack.io/v1alpha1 TenantGateway` CR
per tenant that owns its own Gateway. The cozystack-controller
reconciles the actual `Gateway`, per-tenant `Issuer`, `Certificate`
(wildcard or per-listener), and HTTP→HTTPS redirect `HTTPRoute` from
there. Helm does not render `Gateway` or `Certificate` directly — that
prevents the Helm-vs-controller race on `Gateway.spec.listeners` that
route-driven listener additions would otherwise cause.
```
┌── TenantGateway CR (rendered by extra/gateway chart for owning tenants)
│
└──▶ cozystack-controller reconciles:
├─ Gateway (with dynamic listeners, label-based allowedRoutes)
├─ Issuer (per-tenant ACME account)
├─ Certificate(s) — wildcard + per-child SANs in DNS-01, per-listener in HTTP-01
├─ HTTPRoute (controller-owned http→https redirect)
├─ Namespace label patching (cozy-* system namespaces from spec.AttachedNamespaces)
└─ Watches HTTPRoute / TLSRoute attachments to materialise listeners
```
## Inheritance: tenants attach by default, opt into their own Gateway
A tenant gets a dedicated Gateway (own LB Service, own LB IP, own
Certificate) only when it explicitly asks via
`tenant.spec.gateway=true`. Every other tenant in the tree publishes
through the Gateway of the nearest ancestor that owns one — same shape
as the existing `_namespace.ingress` inheritance.
The `apps/tenant` chart writes a `namespace.cozystack.io/gateway` label
on each tenant namespace, carrying either the tenant's own name (when
owning a Gateway) or the inherited ancestor name (when inheriting). The
Gateway's listener `allowedRoutes.namespaces.selector` is keyed on that
label, so the same selector admits routes from every namespace pointing
at the owner — descendants and cozy-* system namespaces alike.
cozystack-controller patches the same label onto every namespace in
`tgw.Spec.AttachedNamespaces` and garbage-collects labels it wrote when
an entry is removed from the attach list (annotation-gated GC so labels
written by the apps/tenant chart are never stripped).
In DNS-01 mode, the controller extends the parent's wildcard Certificate
with `<child-apex>` + `*.<child-apex>` SANs per inheriting child, and
adds a `*.<child-apex>` HTTPS listener per child apex referencing the
same cert. Without this expansion the parent's single-level wildcard
cannot match a child route's hostname (`harbor.alice.example.org` is two
labels past the parent's `*.example.org`).
Opt-out into own Gateway is the right choice when: a tenant needs its
own LB IP (DNS pinned, firewall rule), the apex is not derived from the
parent (custom `host` value the ancestor's cert can't cover), or the
tenant wants its own ACME account / cert authority. Otherwise leave
`gateway` unset and inherit.
## End-to-end flow
1. Platform: `gateway.enabled=true` toggles cert-manager solver,
switches per-system-app templates from Ingress to HTTPRoute / TLSRoute.
2. Per tenant: explicit `tenant.spec.gateway=true` renders a
`TenantGateway` CR and triggers the controller to materialise the owner
Gateway. Tenants with the field unset inherit through
`_namespace.gateway` propagation in the tenant chart.
3. Adding a published app under any tenant (owner or inheriting) is just
deploying its HTTPRoute. The controller picks up the hostname, adds a
per-listener Certificate (HTTP-01) or extends the wildcard cert's SANs
(DNS-01), and updates the owner Gateway's listeners.
Defaults stay legacy: `gateway.enabled=false`,
`tenant.spec.gateway=false`. Existing clusters upgrade unchanged.
## External IP allocation — mechanism-agnostic
The per-tenant Gateway's auto-created `LoadBalancer` Service draws its
IP from whatever LB allocator the cluster admin has configured at the
platform layer — same shape as ingress-nginx today. The tenant API stays
mechanism-agnostic — no `gatewayIP` field, no allocator-specific
manifest in the tenant chart. Cozystack itself ships MetalLB installed
but does not render any `IPAddressPool` / `L2Advertisement` /
`BGPAdvertisement` from this chart; admins set up the allocator that
suits their environment (MetalLB pool with L2 / BGP, Cilium LB-IPAM with
announcer, robotlb against a cloud provider, or
`Service.spec.externalIPs` pinning).
If a tenant needs a specific address (DNS already pinned, firewall rule,
etc.), the operator pre-allocates it on the admin side: either
pre-create the Service with `loadBalancerIP` set, or hand the tenant a
reference to a named admin-managed pool. Per-Service IP uniqueness is
the allocator's responsibility — same as for any other LoadBalancer
Service.
## Cert mode: HTTP-01 (default) vs DNS-01 (opt-in)
`publishing.certificates.solver` selects how the controller sources TLS
certs.
**HTTP-01 (default)** — per-app HTTPS listener + per-app Certificate.
New apps require zero platform-side config: deploy the HTTPRoute,
controller does the rest. Inheriting children's hostnames pick up
per-listener certs the same way the owner's own apps do.
**DNS-01 (opt-in)** — single wildcard Certificate (`<apex>` +
`*.<apex>`) covering every published app under the apex, plus per-child
SANs (`<child-apex>` + `*.<child-apex>`) for every inheriting tenant.
Pick this for clusters where many apps share the apex and Let's Encrypt
rate limits matter. The DNS provider account must be able to write TXT
records under every apex level the parent serves — for deeply-nested
inherited children that requires zone delegation or a wide-scope
provider credential.
| `publishing.certificates.dns01.provider` | Required
`publishing.certificates.dns01.<provider>` keys |
| --- | --- |
| `cloudflare` (default) | `cloudflare.secretName`,
`cloudflare.secretKey` |
| `route53` | `route53.region`, `route53.secretName` (and `accessKeyID`
if not on IRSA) |
| `digitalocean` | `digitalocean.secretName` |
| `rfc2136` | `rfc2136.nameserver`, `rfc2136.tsigKeyName`,
`rfc2136.secretName` |
The platform chart wires every provider's keys into `_cluster.dns01-*`;
the per-tenant gateway chart and the cluster-wide `cluster-issuers.yaml`
both read them.
## Strengthens admission coverage on apps.cozystack.io/*
The `cozystack-tenant-host-policy` probe in `gateway.bats` initially
looked like a flake — the apply that was supposed to be VAP-rejected
just succeeded. Investigation traced it to the custom REST handler at
`pkg/registry/apps/application/rest.go`. genericapiserver hands every
storage method a callback into the validating admission chain:
`createValidation` for Create, `updateValidation` for Update,
`deleteValidation` for Delete. `Update` was correctly invoking its
callback since
|
||
|
Aleksei Sviridkin
|
c85e473523 |
test(e2e): inheritance attach asserts controller writes, not Cilium Accepted
The previous e2e check waited for `HTTPRoute.status.parents[0].conditions[Accepted]=True` after applying a child tenant's HTTPRoute against the parent's Gateway. That condition is set by Cilium after the listener's TLS Secret becomes Ready — and in the e2e cluster every fresh certificate fails to issue (LE prod refuses every `.example.org` hostname by policy, visible across alerta/grafana/dashboard/seaweedfs-s3 cluster certs in cozyreport). So Cilium never set Accepted, the route timed out 3/3 retries, and the failure was environmental — not a regression in the inheritance code path. The contract the test exists to pin is cozystack-controller's behaviour, which has two observable writes: 1. The parent Gateway grows a per-listener HTTPS entry for the child route's hostname (proves collectHostnameClaims saw the route through the inheritance label and reconcileGateway appended the listener). 2. A per-listener Certificate object appears in the parent namespace with the route's hostname in spec.dnsNames (proves reconcilePerListenerCertificates ran for the same hostname). Both writes are deterministic and depend only on the inheritance label + the HTTPRoute existing — not on ACME, not on Cilium TLS readiness. The test now polls those two objects directly within shorter, scenario-appropriate timeouts (120s for the listener, 60s for the Certificate). The route Accepted check is dropped with an explanation in the test comment so a future cluster with a working ACME doesn't get a misleading pre-condition added back. The namespace inheritance label is also verified up front as the precondition collectHostnameClaims's filter reads. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
320172137e |
fix(tenantgateway): include inheriting child namespaces in hostname claim collection
The HTTP-01 inheritance flow deadlocked at the route-Accepted step:
the apps/tenant chart correctly labelled the child namespace with
`namespace.cozystack.io/gateway=<owner>`, the parent Gateway's
allowedRoutes selector matched by label, but `collectHostnameClaims`
filtered routes by the static set `{tgw.Namespace} ∪ Spec.AttachedNamespaces`
— so a child route in a namespace reached purely via the inheritance
label was silently dropped. No per-listener Certificate, no per-app
HTTPS listener, no hostname match → `Accepted=False,
Reason=NoMatchingListenerHostname` indefinitely.
Caught by the new `child tenant's HTTPRoute attaches to parent's
Gateway via inheritance label` e2e bats — failed 3/3 retries on the
CI run for `afc7765e6`. The Go controller-level
TestReconcile_HTTP01CollectsHostnamesFromInheritingChildNamespaces
test pins the same contract at a faster feedback loop: an HTTPRoute
in a namespace labelled `namespace.cozystack.io/gateway=<owner>` but
NOT listed in Spec.AttachedNamespaces must still cause the
controller to render a per-listener HTTPS listener for its hostname.
The fix extends the `allowed` set in `collectHostnameClaims` to
also include namespaces matched by the gateway label selector, so the
hostname-claim collector and the Gateway's allowedRoutes selector
agree on which namespaces can attach. DNS-01 mode is unaffected — it
returns nil from `collectHostnameClaims` immediately and uses the
separate `collectInheritingChildApexes` path for SAN expansion.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
myasnikovdaniil
|
2cb16d5ae6 |
refactor(build): standardize image tagging to fix concurrent PR push conflicts (#2711)
## What this PR does Concurrent PR builds were racing in the OCIR registry. Every package Makefile tagged its images via `settag`, which on any non-release commit collapsed to `:latest` — so every image across every PR was pushed to the same tag, and parallel builds got 409s on the manifest PUT. Two outliers (`metallb`, `redis-operator`) used hardcoded version tags and collided across PRs unconditionally. This PR replaces `settag` and the `TAG=latest` fallback in `hack/common-envs.mk` with a single `image-tags` macro driven by three env vars: | Env | Meaning | |---|---| | `IMAGE_TAG` | Build-unique tag, always pushed | | `PUBLISH_VERSIONED` | Also push `:<component-version>` (when provided) | | `PUBLISH_FLOATING` | Also push `:latest` | Wired into the two workflows: - **`pull-requests.yaml`** — `IMAGE_TAG=pr-<N>-<sha>`, both `PUBLISH_*` off. Every PR build pushes a unique tag set; no collisions, no `:latest` move. - **`tags.yaml`** — `IMAGE_TAG=<ref_name>`, both `PUBLISH_*` on. Release output preserved (component versions + `:latest`) plus a uniform `:<git-tag>` handle on every image for traceability. - **Local** — defaults to `IMAGE_TAG=dev`, no flags, `PUSH=0`. All 30 package Makefiles converted to `$(call image-tags,<repo>,<versioned-tag>)`. Image consumers reference digests pinned into `values.yaml` by each Makefile, so the tag is decorative for cluster-side pulls — the build-unique tag is sufficient for E2E. Also drops the redundant `:<component-version>-<git-tag>` form (cilium, linstor, matchbox, ubuntu-container-disk, kubevirt-*, cluster-autoscaler). Nothing in the repo or downstream values.yaml consumed it; the same information is preserved via the new `:<git-tag>` plus the existing `:<component-version>` tags on release. `ubuntu-container-disk` builds a separate image per Kubernetes version in a loop, so its primary tag became `:<k8s-ver>-<IMAGE_TAG>` to prevent loop iterations within a single build from pushing to the same tag. Verified by dry-running `make -n image` across representative packages (simple, dual-tag, skopeo, flux-push, loop) under all four contexts (PR / release / main / dev) — tag sets are unique and behave as expected. ### Release note ```release-note refactor(build): PR CI now pushes images under a unique `pr-<N>-<sha>` tag per build instead of `:latest`. Concurrent PR builds no longer race in the registry. Release builds keep the existing `:<component-version>` and `:latest` tags and additionally publish a uniform `:<git-tag>` handle across every image. The redundant `:<component-version>-<git-tag>` tag (cilium, linstor, matchbox, ubuntu-container-disk, kubevirt-*, cluster-autoscaler) is no longer published — image references in values.yaml are digest-pinned and were the only legitimate consumers. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Centralized and simplified container image tagging across build workflows. * PR builds now use unique image tags derived from PR metadata for better isolation. * Added configurable flags to control publishing of versioned and floating tags, making release tagging behavior more predictable. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2711?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
afc7765e69 |
ci(external-dns): regenerate cozyrds schema for gatewayAPI description
`cozyvalues-gen` bakes the values.schema.json's
`gatewayAPI.description` into the JSON openAPISchema field of the
external-dns release-definition CRD at
`packages/system/external-dns-rd/cozyrds/external-dns.yaml`. The Q7
commit (
|
||
|
Aleksei Sviridkin
|
4120f8795e |
test(e2e): cover child-tenant inheritance attach flow
Replaces the now-obsolete "Package admission rejects tenant-* in gateway.attachedNamespaces" e2e (the VAP it pinned was dropped in the inheritance refactor) and adds two new e2e cases that cover the end-to-end inheritance contract: * "Package admission accepts gateway.attachedNamespaces with tenant-* entries" — regression guard against a future refactor re-introducing the gate; the hijack surface is now handled by Layers 4/5/7 (Tenant.spec.host VAP, namespace label immutability VAP, HTTPRoute hostname VAP), not by entries in attachedNamespaces. * "child tenant without explicit gateway inherits _namespace.gateway from a Gateway-owning parent" — chart-level lockstep: a parent tenant with gateway=true produces both the namespace label and the cozystack-values Secret pointing at itself, and a child Tenant under that parent with the gateway field unset receives the parent's name in both places. Plus a negative assertion that the child does NOT get its own gateway HelmRelease. * "child tenant's HTTPRoute attaches to parent's Gateway via inheritance label" — full cross-namespace attach: parent owns the Gateway, child inherits, an HTTPRoute in the child namespace pointing at parent's Gateway reaches Accepted=True once Cilium evaluates the label-based allowedRoutes selector. Route hostname uses the child's derived apex so Layer 7 admits it. The Gateway-Programmed wait is intentionally omitted from the last test — Programmed depends on the LB allocator that ships with the e2e cluster, while Accepted is a pure spec evaluation on the parent Gateway's allowedRoutes selector and is the property the inheritance refactor is meant to deliver. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f36240a374 |
ci: bump cozyvalues-gen pin to v1.5.0 (#2730)
## Summary Bump the `cozyvalues-gen` binary pinned in `.github/workflows/pre-commit.yml` from `v1.4.0` to `v1.5.0`. `v1.5.0` (cozystack/cozyvalues-gen#24) adds support for the `@immutable` annotation. Without this bump, pre-commit re-runs `make generate` with the old `v1.4.0` binary on every PR; the old binary silently ignores `## @immutable` directives and regenerates `values.schema.json` / embedded `openAPISchema` without the corresponding `x-kubernetes-validations` entries — which then shows up as committed-vs-regenerated drift on any PR that tries to use the new annotation. ## No-op for main today No chart on main currently uses `@immutable` (zero hits for `## @immutable` under `packages/apps/*/values.yaml`), so this bump regenerates byte-identical output. The change is purely lifting the floor for downstream PRs that will start using the annotation. ## Unblocks - cozystack/cozystack#2639 — apply `@immutable` to `storageClass` across every stateful Cozystack app (currently fails pre-commit on the v1.4.0 pin). ## Release info - v1.5.0 release notes: https://github.com/cozystack/cozyvalues-gen/releases/tag/v1.5.0 - Tag signed by F20B7CCBF00A5CAE8F9F8B8C7988329FDF395282 (Aleksey Sviridkin) - Goreleaser uploaded `cozyvalues-gen-linux-amd64.tar.gz` (the artefact this workflow downloads) along with the rest of the OS/arch matrix. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated development tooling dependencies for build process improvements. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2730?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
6e0c2d609d |
docs(gateway): describe inheritance model and DNS-01 SAN expansion
Replace the 'Inheritance from parent Gateway' known-limitation entry (which described inheritance as deferred) with: * A top-of-README 'Inheritance: when to opt in for a separate Gateway' section explaining when a tenant owns its Gateway vs inherits, and how the namespace.cozystack.io/gateway label is written by apps/tenant chart and patched onto cozy-* by the controller. * DNS-01 cert-mode docs spell out SAN expansion for child apexes: the wildcard Certificate now carries <child-apex> + *.<child-apex> per inheriting tenant, with a per-child *.<child-apex> listener on the parent Gateway. Documents the DNS-provider-permission requirement for deeply-nested children (ACME challenge writes TXT under each apex zone, so zone delegation or wide-scope provider creds are needed for grandchild apexes). * New known-limitation entries: 64-listener cap on the parent Gateway under inheritance (with the high-fanout escape via tenant.spec.gateway=true), and Cilium sharing-key port-collision (multiple per-tenant Gateways still each take their own LB IP until ListenerSet ships in Cilium). Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
32241afe03 |
feat(gateway): drop tenant-* ban on gateway-attached-namespaces
Removes two gates that prevented child tenants from attaching to a parent's publishing Gateway: 1. Render-time fail() in templates/gateway-hostname-policy.yaml that refused to render the chart when _cluster.gateway-attached- namespaces contained any tenant-* entry. 2. cozystack-gateway-attached-namespaces-policy VAP that enforced the same rule at Package CR admission time. The original rationale — 'a tenant in attachedNamespaces can hijack the publishing tenant's hostnames' — assumed a static-name allowedRoutes whitelist where adding tenant-* literally widened the attach surface for arbitrary hostnames. Under the inheritance refactor (Stage 1/3/5) the attach surface is governed by the namespace.cozystack.io/gateway label selector, not by entries in AttachedNamespaces; and hostname hijack is independently blocked by three other layers: * Layer 4 — cozystack-tenant-host-policy VAP on Tenant.spec.host * Layer 5 — cozystack-namespace-host-label-policy VAP on the namespace label * Layer 7 — cozystack-route-hostname-policy VAP on HTTPRoute and TLSRoute hostnames Those three defend against the hijack vector regardless of what sits in attachedNamespaces, and they apply uniformly to every tenant (inheriting or owning a Gateway). The dropped gates added no extra defence, while their presence broke the inheritance flow kvaps reasonably asks for in the May 25 review. Doc count for gateway-hostname-policy.yaml drops from 8 to 6 (one VAP + one Binding removed); helm-unittest fixtures updated to match, with a regression-guard test asserting the removed VAP name no longer appears in any rendered document. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
99fe080def |
feat(tenantgateway): DNS-01 SAN expansion + listeners for child apexes
In DNS-01 mode, the parent's wildcard Certificate now covers every inheriting child tenant's apex (`<child-apex>` + `*.<child-apex>` as additional SANs) and the parent Gateway grows one wildcard HTTPS listener per child apex referencing the same Certificate. Without the expansion, a child route hostname like `harbor.alice.example.org` fails to match the parent's listeners — `*.example.org` is a single- label wildcard and `alice.example.org` is two labels deep — and the TLS handshake fails the SNI check because the parent cert never listed the child apex. The controller discovers inheriting children by listing namespaces labelled `namespace.cozystack.io/gateway = <tgw.Namespace>` (the existing inheritance contract from Stage 1/3) and reading their `namespace.cozystack.io/host` label, which apps/tenant chart writes to the child apex. Children with no host label are skipped (defensive); a child host equal to the parent apex is deduplicated (cert-manager rejects duplicate dnsNames). HTTP-01 mode is unchanged — its per-listener / per-cert flow already handles child hostnames without any wildcard. Listener naming: `https-child-<first-label>-<8-hex>`, where the hex suffix is sha256(child-apex)[:8] so two children whose first DNS label collides produce distinct listener names. Cap: Gateway API hard-caps spec.listeners at 64 entries. The parent uses 3 slots (http + https + https-apex) leaving ~61 wildcard children per parent. Operators whose subtree fans out beyond that must run the high-fanout branch on its own Gateway via tenant.spec.gateway=true. Tests (TDD red→green): * TestReconcile_DNS01WildcardCertCoversInheritingChildApexes pins the SAN list contains both `<child-apex>` and `*.<child-apex>`. * TestReconcile_DNS01WildcardCertDeduplicatesChildApexEqualToParent pins the dedupe path (child host == parent apex). * TestReconcile_HTTP01WildcardCertNeverRendered pins the inverse: HTTP-01 mode never renders a wildcard Certificate. * TestReconcile_DNS01GatewayHasListenerPerChildApex pins the listener-side expansion (one `*.<child-apex>` listener per inheriting tenant, all referencing the parent's wildcard cert). Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
30302eabe4 |
feat(tenantgateway): switch Gateway allowedRoutes to label selector
The Gateway's HTTPS / TLS-passthrough listeners now match attached namespaces by the namespace.cozystack.io/gateway label keyed on the TenantGateway's own namespace, instead of a static kubernetes.io/metadata.name In [list] whitelist. This restores inheritance parity with the legacy ingress flow: child tenants of a Gateway-owning ancestor publish through the ancestor's Gateway by default — without their own LB IP / Certificate, and without operator intervention to add each child by name to platform values. Two writers populate the label: * apps/tenant chart namespace.yaml — each tenant namespace gets a label pointing at the nearest ancestor that owns a Gateway (self if owning, inherited otherwise). The tenant Helm chart is the source of truth for tenant-tree labels; the controller never touches them (annotation-gated GC). * cozystack-controller (ensureNamespaceLabels) — patches the label onto every namespace in tgw.Spec.AttachedNamespaces, so cozy-* system namespaces (cert-manager, monitoring, harbor, …) reach the publishing Gateway alongside the tenant tree. Tracks its own writes via the cozystack.io/gateway-attached-by annotation; on a reconcile where an entry is removed from AttachedNamespaces, the label is garbage-collected (annotation present + namespace no longer expected → strip both label and annotation). Also adds the gateway-attach label to tenant-root itself via cozystack-basics/templates/tenant-root.yaml when _cluster.gateway-enabled is true, so the Gateway in tenant-root's own namespace passes its own selector for the controller-owned http→https redirect HTTPRoute. Tests (TDD, red→green): * TestReconcile_HTTPSListenerUsesGatewayLabelSelector pins the new MatchLabels shape on HTTPS listeners. * TestReconcile_LabelsAttachedNamespaces pins the patch path: every namespace in spec.AttachedNamespaces (plus tgw.Namespace) carries namespace.cozystack.io/gateway=<owner> after reconcile. * TestReconcile_LabelGCRemovesDroppedAttachedNamespaces pins the GC path: removing an entry between reconciles strips the label. * TestReconcile_LabelGCDoesNotStripHelmOwnedLabels pins the safety contract: labels lacking the controller annotation (Helm-owned tenant labels) are left alone. * tenant-root_test.yaml adds the positive case for the conditional label render under _cluster.gateway-enabled=true. The HTTP listener (port 80) keeps the narrower static-name selector — it allows only the tenant namespace plus cozy-cert-manager (for HTTP-01 ACME challenges) and must NOT accept routes from app namespaces or child tenants, since unattended routes on the HTTP listener serve plaintext and leak credentials. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
97dff75b5f |
feat(tenant): inherit _namespace.gateway from parent
Mirror the existing _namespace.ingress inheritance for Gateway API attachment. A child tenant under a Gateway-owning ancestor publishes its routes through that ancestor's Gateway by default, without burning a dedicated LB IP / Certificate per child. Resolution rule (tenant.gatewayEffective): * gateway=true explicit → own Gateway (LB IP + cert burned) * gateway=false explicit → parked (no inherit, no own) * gateway unset → inherit $parentNamespace.gateway Removes the auto-default for derived-apex tenants — the previous shape gave every derived-apex child its own Gateway / IP / cert, which scales linearly with tenant depth and produces ~N×K LB IPs on a tree of N top-level × K derived-apex tenants. Inheritance collapses that to one LB IP per Gateway-owning ancestor. Custom-apex tenants (host explicitly set to a non-derived value) now fall through to inheritance instead of auto-firing a separate Gateway. The previous auto-fire conflated 'custom apex' with 'want public exposure' and would have silently failed TLS termination anyway (the ancestor's cert does not cover an unrelated apex). The new rule: explicit gateway=true is the only way to ask for a separate Gateway. The namespace.cozystack.io/gateway label and _namespace.gateway in the rendered cozystack-values Secret now both carry the owner-tenant name — could be self (when this tenant owns one) or any ancestor up the chain. Apps in the tenant render HTTPRoutes whose parentRefs.namespace points at that owner. helm-unittest pinned for the HelmRelease render side. The cross-namespace attach end of the flow (ReferenceGrants, listener / SAN expansion on the owner Gateway) lands separately in the TenantGateway controller. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
943441b0c7 |
ci: bump cozyvalues-gen pin to v1.5.0
v1.5.0 adds the @immutable annotation support (cozystack/cozyvalues-gen#24). Without the bump, pre-commit re-runs make generate with the v1.4.0 binary, silently strips any x-kubernetes-validations entries that downstream PRs add to values.schema.json / embedded openAPISchema, and reports drift against the committed artefacts. This is a no-op for the current main: no chart currently uses @immutable, so regeneration with v1.5.0 produces identical output to v1.4.0. Unblocks cozystack/cozystack#2639 (apply @immutable to storageClass across stateful apps). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
0994821bda |
feat(monitoring): toggle ingress↔HTTPRoute on _cluster.gateway-enabled
Adds Gateway API equivalents for the two ingress endpoints in the
monitoring chart — Grafana (web UI) and Alerta (alert console) — gated
on the same _cluster.gateway-enabled flag the keycloak / dashboard /
cozystack-api templates already use.
* Grafana's ingress is configured inline via Grafana CRD's spec.ingress
block; that block is now wrapped with {{ if ne gateway-enabled true }}
and a standalone HTTPRoute attaching to the tenant Gateway is rendered
in the alternate branch.
* Alerta uses a standalone Ingress object; the same Ingress↔HTTPRoute
swap is wired through an if/else.
Both routes attach to Gateway cozystack in the publishing namespace
(default tenant-root), matching the convention established by the rest
of the system charts.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
18fd155d2b |
feat(external-dns): enable gateway-tlsroute alongside gateway-httproute
The Gateway API source-flag toggle (gatewayAPI=true) already enabled gateway-httproute discovery. cozystack ships TLSRoute attachments for SSL-passthrough services (kubernetes control plane API, kubevirt CDI uploadproxy, kubevirt VM exportproxy) — without gateway-tlsroute in the external-dns sources list those hostnames are never picked up for DNS record management. Adds gateway-tlsroute under the same conditional and updates the value description / schema / README to reflect that the toggle now covers both route kinds. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
7c893bc653 |
docs(gateway): security model as three groups, mechanism-agnostic IP allocation
Security model section split into three groups by who they defend against, anchoring the framing in the apps.cozystack.io/* tenant API surface (tenants do not write Gateway API resources directly, so most of the seven layers are not protecting against tenant-user input): - tenant-user-input gates: Layer 4 (cozystack-tenant-host-policy) via cozystack-api admission. Tenant.spec.host is the user-supplied field that surfaces as a security boundary. - defense-in-depth: Layers 1, 2, 5, 6, 7. Cover bugs in cozystack-controller / Flux, supply-chain compromise of an app chart, confused-deputy admin mistakes. Fail-closed. - admin-against-themselves: Layer 3 (cozystack-gateway-attached- namespaces-policy). Catches kubectl edit on the platform Package. Layer 7 wording fix: drop the implication that a tenant user with HTTPRoute RBAC could exploit the cross-apex hostname surface — that RBAC is not granted in Cozystack. Reframed as defense-in-depth against an app chart bug or supply-chain compromise. External IP allocation section reframed as admin-side concern: allocator (MetalLB / Cilium LB-IPAM / robotlb / externalIPs) is configured at the platform layer, tenant API stays mechanism-agnostic. Per-Service IP uniqueness is the allocator's responsibility — same shape as for any LoadBalancer Service. Inheritance Known limitations entry: spell out both upstream gaps (Gateway API 64-listener cap, Cilium sharing-key inactive on 443/TCP via cilium#42756) so readers understand why per-tenant Gateway is the only scalable shape today. publishing.gateway.attachedNamespaces -> gateway.attachedNamespaces fix: the values block is at top-level, not under publishing. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
039924f59d |
feat(gateway): align _namespace.gateway propagation with tenant.gatewayEffective
Two related fixes for the gap where the tenant chart's gateway HelmRelease materialises but child apps (harbor, bucket) silently fall back to ingress-nginx: - packages/apps/tenant/templates/namespace.yaml: switch the _namespace.gateway resolution from raw .Values.gateway to tenant.gatewayEffective. This is the same helper consulted by gateway.yaml (HelmRelease render), so per-tenant Gateway materialisation and child-app HTTPRoute / Ingress selection now stay in lockstep — including the auto-default for derived-apex tenants. - packages/system/cozystack-basics/templates/cozystack-values-secret.yaml: add a conditional gateway field to the tenant-root cozystack-values Secret. cozystack-basics is the only writer of _namespace.* for tenant-root (apps/tenant chart skips tenant-root via the existing ne 'tenant-root' gate). Without this line, tenant-root apps read _namespace.gateway as empty and route to ingress even when the platform's gateway.enabled flag is on. Conditional on _cluster.gateway-enabled='true' so non-Gateway clusters keep the empty value. Test fixture in cozystack-basics/tests/cozystack-values-secret_test.yaml pins the conditional and the regression-guard that the other four _namespace keys (etcd, ingress, monitoring, seaweedfs) stay unconditional. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d2e851f63b |
feat: Gateway API support via Cilium
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
cb2f606c76 |
fix(csi): verify VMI Ready after kubevirt-csi Publish to surface stuck-PVC failures (#2659)
## What this PR does
Stops `ControllerPublishVolume` from silently reporting success when the
volume is in VM spec but the hotplug attach never actually completed.
### Root cause
Upstream's fast path (`EnsureVolumeAvailableVM`) checks only
`vm.Spec.Template.Spec.Volumes`. If a prior Publish wrote that entry but
timed out waiting for VMI Ready (e.g. the infra PVC is stuck
`ClaimPending` because LINSTOR cannot autoplace), the next Publish
short-circuits to success. external-attacher sets `Attached=true`, but
QEMU was never sent `device_add`. The tenant kubelet's NodeStageVolume
then fails with:
```
rpc error: code = Unknown desc = couldn't find device by serial id
```
Observed in production. Sequence from kubevirt-csi-driver controller
logs:
```
16:53:13 /ControllerPublishVolume called
16:55:13 volume failed to be ready in time (2m), context deadline exceeded
16:55:14 /ControllerPublishVolume called (retry)
16:55:14 Volume X already attached to VM Y - skipping hot-plug
16:55:14 /ControllerPublishVolume returned with response: {}
```
VMI volumeStatus during that window: `phase=Pending message="PVC is in
phase ClaimPending"`. VolumeAttachment then goes `Attached=true`; tenant
kubelet surfaces "couldn't find device by serial id".
### Fix
After upstream Publish returns success for a hotplug volume, the wrapper
re-reads VMI status and checks `Phase == VolumeReady` for this
DataVolume. If it is not Ready (or not present), the RPC returns
`codes.Unavailable` with the upstream reason, so external-attacher keeps
retrying instead of declaring the volume attached. The underlying
provisioning failure surfaces at the CSI layer instead of becoming a
confusing kubelet-side "device not found by serial id".
This is a wrapper-side workaround for a defect in upstream
`kubevirt.io/csi-driver`; the structural fix belongs upstream (tighten
the fast path to require VMI Ready) and will be filed separately. The
wrapper guard is safe to keep as defense-in-depth even after the
upstream fix lands.
Relates to #2634.
### Release note
```release-note
fix(csi): kubevirt-csi-driver Publish now verifies VMI Ready before reporting attached, surfacing stuck-PVC failures at the CSI layer instead of as "couldn't find device by serial id" on the tenant kubelet
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved hotplug volume publishing: non‑NFS volumes are now verified
on the VM before being marked published; if the volume is missing or not
ready, publishing returns unavailable, improving reliability for
hotplugged storage.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2659?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
60357eb4a6 |
fix(csi): route RWX Block volumes to upstream hotplug detach (#2658)
## What this PR does Fixes routing in `WrappedControllerService` so RWX **Block** volumes (KubeVirt live-migration disks) are no longer treated as NFS during unpublish and expand. ### Root cause `ControllerUnpublishVolume` selected the NFS cleanup branch based on `hasRWXAccessMode(pvc)` alone. RWX Block matched and went down the NFS path — which just deletes a `CiliumNetworkPolicy`. For a block volume that CNP does not exist, so `removeCNPOwnerReference` returns successfully without doing anything, and the function reports success without ever calling upstream's hotplug detach. The DataVolume entry stays in `VM.spec.template.spec.volumes` permanently. When the same PVC later attaches to a different VM, linstor-csi's anti-split-brain check sees pods from two different VMs referencing the same RWX block PVC and blocks the attach with: ``` RWX block volume <ns>/<pvc> is being used by pods from different VMs (<vm-a> and <vm-b>); this is not supported - RWX block volumes with allow-two-primaries are only for live migration of a single VM ``` `ControllerPublishVolume` was unaffected because it routes on the explicit `nfsVolumeKey` from `VolumeContext`. `ControllerUnpublishVolume` does not receive `VolumeContext` and has to infer from the PVC, and the previous inference was incomplete. ### Fix `isNFSVolume(pvc)` now requires `ReadWriteMany` access mode **and** `VolumeMode == Filesystem` (always set by our NFS `CreateVolume` path). RWX Block correctly delegates to upstream `ControllerUnpublishVolume`, which removes the entry from VM spec via `RemoveVolumeFromVM`. The same predicate replaces the previous access-mode check in `ControllerExpandVolume`, which was also clearing `NodeExpansionRequired` for RWX Block — block volumes still need in-VM disk expansion. Fixes #2634. ### Release note ```release-note fix(csi): route RWX Block volumes in kubevirt-csi-driver to upstream hotplug detach so VM spec entries are cleaned up on unpublish ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * More accurate detection of network-attached (NFS) vs block volumes, preventing incorrect cleanup of block volumes. * NFS volumes continue to receive special cleanup; non-NFS volumes now follow upstream cleanup paths. * **Refactor** * Node-side expansion logic refined to base decisions on improved NFS classification (RWX + filesystem), reducing expansion errors. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2658?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
e42a5828fe |
refactor(build): skip duplicate versioned tag when it matches IMAGE_TAG
On release builds IMAGE_TAG = github.ref_name = vX.Y.Z, and Makefiles that pass v$(COZYSTACK_VERSION) as the versioned arg evaluate to the same vX.Y.Z on the tag commit. The macro then emitted `--tag :vX.Y.Z --tag :vX.Y.Z --tag :latest`. docker buildx dedupes internally so there is no functional impact, but the build logs carried the redundant flag for every affected package. Replace the empty-string guard with filter-out so the versioned tag is also skipped when it would duplicate IMAGE_TAG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
1bcaa3de54 |
fix(flux-plunger): correct common-envs.mk include path
The include pointed at ../../../scripts/common-envs.mk, but that directory does not exist — only hack/common-envs.mk does. flux-plunger is not part of the root `make build`, so the broken path went unnoticed until the new image-tags macro made the recipe depend on it. Without the include, `make -C packages/system/flux-plunger image` fails immediately with "No such file or directory". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
6c8a67db3d |
fix(talos): fail fast on skopeo copy errors in image-talos
The image-talos recipe chains skopeo copy with `;` in a single shell. Without `set -e`, the recipe's exit status is the last command's, so a failed copy in the middle of the chain can be masked by a later successful command — or worse, by a false-branch `if`/`fi` (which exits 0 and would silently hide a push failure on PR builds where both PUBLISH_* flags are 0). Prepend `set -e;` to abort the recipe on the first failure. Also quote "$SRC" defensively. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
f874ba93b8 |
test(kubernetes): functional regression test for config_path seds
The helm-unittest assertions added in the previous commit cover the
structural shape of the sed lines (they must exist in
preKubeadmCommands). They do not catch the failure mode this PR exists
to fix: a future containerd release renames the registry plugin
section, the sed silently stops matching, and per-registry mirroring
under /etc/containerd/certs.d quietly breaks.
Add a bash test that:
1. Drift-checks the script's copy of the two sed lines against the
verbatim strings embedded in templates/cluster.yaml, so the test
cannot diverge from the template without failing.
2. Runs both seds against minimal fixtures of containerd 1.x and 2.x
default configs (tests/fixtures/containerd-{1,2}x.toml) and asserts
that config_path is rewritten exactly once in each. If containerd
3.x renames the section again, the fixture stops matching, the test
fails, and CI surfaces the regression instead of letting it ship
silently.
Wired into the package's `make test` target, which is already invoked
in CI alongside helm-unittest.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
47953c48b8 |
test(kubernetes): cover containerd config_path patching for both plugin schemas
Add helm-unittest assertions for both sed lines in preKubeadmCommands so that a future regression to a single-regex form, which silently fails against containerd 2.x where the plugin section was renamed and quoted with single quotes by default, is caught at template render. Add a Helm comment above the two seds explaining the rename from io.containerd.grpc.v1.cri (containerd 1.x) to io.containerd.cri.v1.images (containerd 2.x), so future readers understand why both lines exist and that the wrong-version one is intentionally a no-op on each image version. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Elaugaste
|
dbcd09a0fd |
fix(kubernetes): add config_path patch for containerd 2.x
This adds support for containerd 2.x CRI config_path configuration, as the CRI plugin section was renamed from io.containerd.grpc.v1.cri to io.containerd.cri.v1.images. It also supports both single and double quotes around plugin names. Assisted-By: Gemini <noreply@google.com> Signed-off-by: Elaugaste <elaugaste@gmail.com> |
||
|
Timur Tukaev
|
1810263f9d |
ci: pin Scorecard workflow actions by SHA (#2721)
## What this PR does Fixes the OpenSSF Scorecard workflow added in #2720. The initial workflow used floating major-version tags (`@v2`, `@v4`, `@v3`). `ossf/scorecard-action` does not publish a floating `v2` tag — only specific patch releases — so the first scheduled run failed: > Unable to resolve action `ossf/scorecard-action@v2`, unable to find version `v2` This PR pins every action by its full commit SHA with a trailing comment showing the human-readable version. This also satisfies the **Pinned-Dependencies** Scorecard check itself. - `actions/checkout@de0fac2` (v6.0.2) - `ossf/scorecard-action@4eaacf0` (v2.4.3) - `actions/upload-artifact@043fb46` (v7.0.1) - `github/codeql-action/upload-sarif@03e4368` (v3) ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated CI/CD infrastructure dependency versions to maintain security and stability. **Note:** This release contains no user-facing changes. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2721?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timur Tukaev
|
688f73f8f0 |
ci: pin Scorecard workflow actions by SHA
The initial Scorecard workflow used floating major-version tags (@v2, @v4, @v3). ossf/scorecard-action does not publish a floating v2 tag — only specific patch releases — so the first scheduled run failed with `Unable to resolve action ossf/scorecard-action@v2`. Pin every action by its full commit SHA with a trailing comment showing the human-readable version. This also satisfies the `Pinned-Dependencies` Scorecard check itself. - actions/checkout@de0fac2 (v6.0.2) - ossf/scorecard-action@4eaacf0 (v2.4.3) - actions/upload-artifact@043fb46 (v7.0.1) - github/codeql-action/upload-sarif@03e4368 (v3) Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io> |
||
|
Timur Tukaev
|
3a8d1c5090 |
ci: add OpenSSF Scorecard workflow (#2720)
## What this PR does Adds a GitHub Actions workflow that runs the [OpenSSF Scorecard](https://github.com/ossf/scorecard) on this repository. ### Schedule - Weekly on Mondays at 06:00 UTC - On branch protection rule changes - On pushes to `main` - Manual dispatch ### Where results go - **Public** [`scorecard.dev`](https://scorecard.dev/viewer/?uri=github.com/cozystack/cozystack) — transparency for adopters and reviewers (CNCF DD). - **Code-scanning dashboard** (Security tab) — for in-team triage of individual findings. - SARIF artifact retained for 5 days for local inspection. ### Token permissions Default `read-all`, with the minimum required scopes elevated only inside the analysis job: - `security-events: write` — upload to code scanning - `id-token: write` — publish to `scorecard.dev` via OIDC, no long-lived secret - `contents: read`, `actions: read` ### Why no README badge yet This PR enables the workflow but **does not add a badge to the README**. Once the first scans have completed and quick-win improvements (token permissions across other workflows, dependency pinning in workflows, etc.) have landed, a follow-up PR will add the badge. ### Context - Tracked under the project's CNCF Incubation readiness work and the in-progress security self-assessment. - Companion to the OpenSSF Best Practices passing badge (https://www.bestpractices.dev/en/projects/10177) — Scorecard adds machine-verifiable, continuously-updated evidence; Best Practices captures self-attested controls. ### Release note \`\`\`release-note NONE \`\`\` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Added automated security scanning to the development process to continuously monitor codebase health and security posture. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2720?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timur Tukaev
|
2396d08ee5 |
ci: add OpenSSF Scorecard workflow
Runs the OpenSSF Scorecard action on: - weekly schedule (Mon 06:00 UTC) - branch protection rule changes - pushes to main - manual dispatch Publishes results to: - the public OpenSSF Scorecard API (scorecard.dev) for transparency - the GitHub code-scanning dashboard (Security tab) for triage No README badge is added yet — the badge will be introduced in a follow-up PR once the first scans have run and any quick-win improvements (token permissions, dependency pinning) have landed. Aligns with the project's CNCF Incubation readiness work and the security self-assessment in progress. Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io> |
||
|
Timur Tukaev
|
b689613151 |
chore(maintainers): mark @lllamnyp as Core Maintainer (#2719)
## What this PR does Updates @lllamnyp's responsibility in `MAINTAINERS.md` from `Etcd-operator Lead` to `Core Maintainer`. The etcd-operator is hosted in the separate [aenix-io/etcd-operator](https://github.com/aenix-io/etcd-operator) repository — its leadership scope is tracked there. In `cozystack/cozystack`, @lllamnyp's role is a project-level Core Maintainer, which this change reflects. ### Release note ```release-note NONE ``` |
||
|
Timur Tukaev
|
b7c6f2793a |
docs(contributor-ladder): document Emeritus technical offboarding and reactivation (#2718)
## What this PR does Expands the **Stepping Down / Emeritus Process** section in `CONTRIBUTOR_LADDER.md`: - Adds an explicit PR-based mechanism for moving a maintainer's row from `Active` to `Emeritus` in `MAINTAINERS.md`. - Adds a five-business-day technical offboarding checklist: GitHub team membership, org role, direct collaborator entries, `CODEOWNERS`, Actions secret rotation, internal channels. - Defines a path for an Emeritus maintainer to return to active status. This codifies the offboarding flow actually exercised in #2717 and removes ambiguity for future maintainer lifecycle events. Useful evidence for the CNCF Incubation Due Diligence review (Governance section, maintainer-lifecycle criteria). ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated contributor documentation with details on the Emeritus transition process, including offboarding procedures and reactivation guidelines. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2718?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timur Tukaev
|
5da0052a48 |
chore(maintainers): mark @lllamnyp as Core Maintainer
The etcd-operator lives in the aenix-io/etcd-operator repository, not cozystack/cozystack. Listing it as a responsibility in this project's MAINTAINERS.md was a leftover from the earlier sub-project layout. Replace with the project-level "Core Maintainer" role, which reflects the actual scope of @lllamnyp's work on this repository. Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io> |
||
|
Timur Tukaev
|
3af8c98c9a |
docs(contributor-ladder): document Emeritus technical offboarding and reactivation
Expands the Stepping Down / Emeritus Process section with: - An explicit PR-based mechanism for recording an Emeritus transition. - A five-business-day technical offboarding checklist (team membership, org role, direct collaborator entries, CODEOWNERS, secret rotation, internal channels). - A defined path for an Emeritus maintainer to return to active status. Reflects the actual offboarding flow exercised in #2717 and removes ambiguity for future maintainer lifecycle events. Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io> |
||
|
Timur Tukaev
|
cd5b4d00dc |
chore(maintainers): update roster and add Emeritus section (#2717)
## What this PR does - Add `Active Maintainers` and `Emeritus Maintainers` sub-sections. - Update @gecube affiliation to Independent (was Ænix). - Update @lllamnyp affiliation to Ænix (was 3commas). - Mark @nbykov0 as Independent (was Ænix). - Move @aobort to Emeritus Maintainers. - Move @klinch0 to Emeritus Maintainers. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated project maintainers information in the maintainers list. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2717?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timur Tukaev
|
fa43065225 |
chore(maintainers): update roster and add Emeritus section
- Add `Active Maintainers` and `Emeritus Maintainers` sub-sections. - Update @gecube affiliation to Independent (was Ænix). - Update @lllamnyp affiliation to Ænix (was 3commas). - Mark @nbykov0 as Independent (was Ænix). - Move @aobort to Emeritus Maintainers. - Move @klinch0 to Emeritus Maintainers. Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io> |
||
|
Andrey Kolkov
|
debb9f85db |
feat(platform): add implementation of backup-strategy controller for etcd (#2641)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note add implementation of backup-strategy controller for etcd ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Cluster-scoped Etcd backup strategy CRD (S3-only) plus controller support for BackupJob (snapshot → Backup) and destructive in-place RestoreJob flows; snapshot metadata persisted and destination validation enforced. * **Documentation** * Complete etcd backup/restore README and admin/tenant scenario guides. * **Examples & Tests** * End-to-end demo scripts (helpers, stepper, cleanup, runner), Bats e2e, and extensive unit tests. * **Chores** * RBAC updates for etcd access, chart version bumped, demo env files added to .gitignore. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2641?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
d80f67e799 |
feat(platform): add implementation of backup-strategy controller for etcd
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
