mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 14:02:56 +09:00
8f4c19daaeb00a2aeccf761cd96879867a4d86ce
5128
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
6fc2606111 |
test(api): add read-path authorization tests for TenantNamespace registry
Cover the per-event filtering paths of REST.Watch directly instead of only transitively through the bookmark test: authorization filtering via RoleBindings, privileged groups (system:masters, cozystack-cluster-admin), non-tenant namespace and foreign object drops, backing bookmark forwarding, field and label selector filtering, missing user rejection, access-check error resilience, ADDED deduplication by resourceVersion, terminating bookmark flush on backing watcher close, and goroutine shutdown on Stop and context cancellation. Also close the remaining gaps on the sibling read paths sharing the same authorization logic: List/filterAccessible (per-user filtering, privileged groups, missing user, RoleBindings outside the tenant set, backing list errors) and the Get error branches. Branches the controller-runtime fake client cannot drive (backing bookmarks, non-Namespace objects, chosen resourceVersions, backing watch errors) are exercised through a stub client.WithWatch backed by a test-owned FakeWatcher. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
9e3ae7447b |
fix(ingress): keep tls section in operator wildcard mode
Dropping the whole tls block in wildcard mode disabled the forced HTTP-to-HTTPS redirect: ingress-nginx only issues the 308 redirect when TLS is enabled for the Ingress, while a host without a tls section is still served on 443 with the default certificate but stays reachable over plain HTTP. Keep the tls section with the host list and omit only the per-host secretName, so ingress-nginx falls back to the operator wildcard configured as the default SSL certificate and the redirect stays enforced, matching the non-wildcard behaviour. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
b1f5d9579c |
feat(ingress): serve operator wildcard via default SSL certificate
When publishing.certificates.wildcardSecretName is set, the root tenant's ingress-nginx controller now serves the operator-supplied wildcard Secret as its --default-ssl-certificate, and the platform service Ingresses stop requesting their own per-host ACME certificates. The flag is set only on the publishing controller (Release.Namespace == expose-ingress) because ingress-nginx loads the Secret from its own namespace; child-tenant controllers cannot reach a Secret in another namespace, so they keep the existing ACME path. Each service Ingress (dashboard, keycloak, grafana, alerta, bucket, linstor-gui, harbor, seaweedfs S3, bootbox) drops its cert-manager cluster-issuer annotation, the http01 challenge-class annotation, and its tls block in wildcard mode; nginx then falls back to the default certificate for those hosts. Other annotations and routing are unchanged. Each file keeps its existing render guard intact, and the annotations block collapses cleanly where the cert annotations were its only content. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
6e219c241b |
feat(platform,gateway): distribute operator wildcard secret name
Add publishing.certificates.wildcardSecretName to the platform chart and propagate it through the cozystack-values channel as _cluster.wildcard-secret-name. Only the Secret NAME travels — the cert/key material is never broadcast, unlike the public kube-root-ca that already rides this channel. When the key is set, the gateway chart switches the rendered TenantGateway to certMode=existingSecret with a wildcardSecretRef pointing at the operator-supplied Secret, and skips the ACME solver/provider/issuer validations that no longer apply. Wildcard mode takes precedence over solver=dns01. TLS-passthrough services and attached namespaces continue to render unchanged. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
4262658f0f |
feat(tenantgateway): add existingSecret cert mode for operator wildcard
Operators who already hold a wildcard TLS certificate (purchased, or issued by a corporate CA) had no way to make a tenant Gateway serve under it: the only cert modes were http01 (per-listener ACME) and dns01 (controller-minted wildcard via ACME). Add a third mode, existingSecret, that references a pre-existing TLS Secret in the TenantGateway's own namespace via spec.wildcardSecretRef. In this mode the controller mints no ACME Issuer and no Certificate; the wildcard, apex, and per-child-apex HTTPS listeners reference the supplied Secret directly. A missing wildcardSecretRef fails fast with Ready=False rather than producing a Gateway pointing at a nonexistent Secret. Switching into existingSecret from http01/dns01 reclaims the now-unused Issuer and Certificates so no ACME machinery is left behind. The non-cert listeners (http->https redirect, TLS passthrough) are unaffected and continue to render. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c6f6fee617 |
docs(gpu-operator): document passthrough variant host-driver prerequisite (#2767)
## What this PR does Documents an undocumented prerequisite of the `cozystack.gpu-operator` `default` (passthrough) variant: it assumes the GPU is not already claimed by a host-installed NVIDIA driver. The chart's `vfio-manager` DaemonSet runs the upstream NVIDIA `k8s-driver-manager` init container, which is hard-coded to exit non-zero and label the node `nvidia.com/gpu.deploy.driver=pre-installed` the moment `chroot /host nvidia-smi` returns a driver version (a `PATH` lookup, not a fixed file path). There is no env-var override in `k8s-driver-manager` v0.10.0, so today the only working path is a clean host. The new "Passthrough variant: host preparation" section in `docs/gpu-vgpu.md` covers: - The exact symptom (init log + sandbox-validator crashloop + `lspci` evidence) so operators can recognise it without spelunking. - A clean-host workaround: a build-from-installed purge list (`dpkg-query`, so a zero-match pattern can't silently abort the whole `apt` transaction the way a hardcoded `'cuda-*'` pattern does on a host without the CUDA repo), `/etc/modprobe.d/blacklist-nvidia.conf` (incl. `nvidia_peermem`), `update-initramfs -k all`, reboot. Warns that `autoremove` is dangerous on converged GPU + NVIDIA DOCA / Mellanox hosts because of the shared `nvidia-` prefix. - Post-reboot verification (no `nvidia` modules loaded, no `nvidia-smi` on `PATH`, no leftover NVIDIA DKMS module) and a label-cleanup step — the DaemonSet pods crash-loop and retry on their own, but the operator's success path never resets the `pre-installed` label, so it must be cleared manually. - A "Known limitation" subsection that names the upstream constraint and the planned mitigation path (upstream env-var override + a future `container` variant of the package). Linked to #2763. ### Release note ```release-note docs(gpu-operator): document that the passthrough (`default`) variant requires a clean host and explain how to purge a pre-installed NVIDIA host driver before enabling the variant. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Renamed GPU guide to cover passthrough and vGPU scenarios; updated verification/last-verified info. * Added passthrough host-preparation and troubleshooting, including a clean-host workaround with driver removal, kernel-module blacklist, initramfs update, reboot, and steps to force a fresh device bind. * Clarified FORCE_REINSTALL does not bypass early host-driver detection and noted pending mitigation paths. * Updated OS support table: passthrough requires clean-host on Ubuntu 20.04/22.04/24.04; Ubuntu 26.04 includes usr-merge notes and vGPU marked as caution (⚠️). <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.5.0-rc.2 api/apps/v1alpha1/v1.5.0-rc.2 |
||
|
|
cd36b0d3e3 |
docs(gpu-operator): document passthrough variant host-driver prerequisite
The vfio-manager DaemonSet's k8s-driver-manager init container exits non-zero and labels the node pre-installed whenever it finds a host NVIDIA driver, leaving vfio-pci unbound and the sandbox validator in CrashLoopBackOff. Upstream offers no override knob, so the only working path today is a clean host. Document the symptom (including the Init:CrashLoopBackOff pod state), a build-from-installed purge plus module-blacklist workaround (with the autoremove-vs-DOCA warning and the apt zero-match-abort caveat), the post-reboot verification, and clearing the stale pre-installed label that the operator's success path never resets. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
1d747b3d8d |
feat(platform): auto-wire KubeVirt permittedHostDevices and HostDevices feature gate when gpu-operator is enabled (#2768)
Closes #2765. ## What this PR does Makes the passthrough and vGPU variants of `cozystack.gpu-operator` produce a KubeVirt CR that is immediately usable, without an out-of-band `kubectl patch`. Today the gpu-operator stack reports green, `nvidia.com/<model>` shows up under node `.status.allocatable`, but every `VirtualMachine` referencing the resource via `domain.devices.hostDevices` is rejected by the admission webhook because `HostDevices` is not in `developerConfiguration.featureGates` and the resource is not listed under `permittedHostDevices`. That manual reconciliation is now driven by the platform. The change has three parts. `packages/system/kubevirt` now parameterizes the KubeVirt CR template. `developerConfiguration.featureGates` keeps its existing static defaults and appends `.Values.extraFeatureGates`; `spec.configuration.permittedHostDevices` renders verbatim from `.Values.permittedHostDevices` when set; `spec.configuration.mediatedDevicesConfiguration` does the same. Defaults stay empty so the rendered CR is byte-identical to the previous template when no overrides are supplied. The chart's `make update` target re-applies the four sed transforms after fetching the upstream CR, so refreshing the KubeVirt release version stays a single command. `packages/core/platform` adds two defaults tables under `files/`. `gpu-passthrough-defaults.yaml` covers Hopper (H100/H200), Ada Lovelace (L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10), Turing (T4) and Volta (V100/V100S); PCI vendor:device pairs are stable, `resourceName` slugs follow the `<arch>_<model>_<form>_<mem>` convention `nvidia-sandbox-device-plugin` generates as of v25.x, and `externalResourceProvider: true` is set everywhere because those resources are advertised by the sandbox plugin rather than KubeVirt's in-tree device plugin. `gpu-vgpu-defaults.yaml` covers the Pascal–Ampere mdev profile range (A100-40C/80C, A40-24Q/48Q, A30-24C, A10-24Q, V100D-32C, T4-16Q) — the same family span the upstream `vgpu-device-manager` walks `/sys/class/mdev_bus/` for. Ada Lovelace and Blackwell SR-IOV vGPU is intentionally out of scope for the default table and is expected to be advertised via the user-override hook described below. `templates/bundles/iaas.yaml` now selects between the two tables via a new `bundles.iaas.gpuOperatorVariant` knob (`default` for passthrough, `vgpu` for mediated; default is passthrough). The same variant flows through to the gpu-operator Package CR, replacing the previous always-default call. When the package is enabled, the platform injects `HostDevices` into `extraFeatureGates` plus the chosen defaults into `permittedHostDevices` for the kubevirt component's values. Operators extend or replace via top-level platform values: ```yaml gpu: replaceDefaults: false # append (default) vs. wipe-and-replace permittedHostDevices: pciHostDevices: [] # cluster-specific PCI passthrough entries mediatedDevices: [] # cluster-specific vGPU entries mediatedDevicesConfiguration: {} # extra mediated-device types / per-node mappings ``` `docs/gpu-vgpu.md` is updated to drop the manual `kubectl patch` step and describe the auto-wiring contract, the per-variant defaults, the escape hatches, and the sandbox-device-plugin `resourceName` slug convention so an operator whose node publishes a different name knows where to override. ## Test coverage `helm-unittest` covers eight branches in `packages/core/platform/tests/bundles_gpu_kubevirt_wiring_test.yaml`: gpu-operator disabled emits no extras; gpu-operator enabled at default variant emits the HostDevices gate plus `pciHostDevices`; vgpu variant switches to `mediatedDevices` plus `mediatedDevicesConfiguration`; user overrides append by default; `replaceDefaults: true` wipes the platform table; an unknown variant fails fast with a descriptive error. `packages/system/kubevirt/tests/kubevirt_cr_test.yaml` pins the empty-values render and the three populated shapes at the CR template level. `make helm-unit-tests` passes locally on the full repo. ### Release note ```release-note feat(platform): the KubeVirt CR is now auto-configured for GPU host-device passthrough (HostDevices feature gate plus permittedHostDevices populated from NVIDIA defaults) whenever cozystack.gpu-operator is in bundles.enabledPackages. Operators no longer need a manual kubectl patch to make VMs schedule against a GPU. The variant is selected via bundles.iaas.gpuOperatorVariant (default: passthrough; vgpu: mediated). Extend or replace the NVIDIA defaults via .gpu.permittedHostDevices and .gpu.replaceDefaults. **Upgrade note:** the bundle now OWNS spec.configuration.permittedHostDevices on the KubeVirt CR. If you previously hand-edited it (kubectl edit kubevirt), the first reconcile after upgrade overwrites your manual entries with the rendered NVIDIA default table. Before upgrading, move any custom entries into .gpu.permittedHostDevices (set .gpu.replaceDefaults: true to keep only your list) and verify each resourceName against what your nodes advertise (kubectl describe node <node> | grep nvidia.com/) — the default table now carries the exact slug the sandbox device plugin generates, which upper-cases the card's PCI-IDs name and keeps the Tesla/GL tokens (e.g. a Tesla T4 is nvidia.com/TU104GL_TESLA_T4), but a different plugin build can publish a different string; a mismatch makes the admission webhook reject GPU VMs on their next restart or migration. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Bundle-managed GPU Operator with selectable passthrough vs vGPU variants; platform now injects GPU settings into KubeVirt and forwards vGPU manager overrides when enabled. * Default platform mappings for PCI passthrough and vGPU mediated-device resources added. * **Documentation** * Expanded GPU/vGPU deployment guide with bundle-managed vs manual guidance, wiring/precedence rules, resource-name conventions, and verification steps. * **Tests** * New and expanded tests for bundle→KubeVirt wiring, variant behaviors, default/override merging, and idempotent template parameterization. * **Chores** * Added Helm test target and hardened KubeVirt template update/idempotency checks. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
b6108039b7 |
feat(gpu-operator): add container variant for preinstalled host driver (#2766)
## What this PR does Add a third architectural variant to `cozystack.gpu-operator` for hosts where the NVIDIA driver and `nvidia-container-toolkit` are already installed by the distro package manager (the typical Ubuntu/Debian GPU node shape: apt-installed driver + apt-installed CT). The existing variants do not produce a working setup in that shape. `default` (passthrough) runs `vfioManager`, which unbinds the host driver to bind `vfio-pci` — with the host driver active, `k8s-driver-manager` early-exits and the bind never happens, so the GPU never reaches a pod. `vgpu` requires the proprietary NVIDIA vGPU host driver and a license server; the user-installed open or proprietary driver does not satisfy it. The new `container` variant maps onto upstream gpu-operator's third mode (`sandboxWorkloads.enabled=false` + `devicePlugin.enabled=true`) with every component that would conflict with the host install switched off: - `driver.enabled=false` — use the host driver. - `toolkit.enabled=false` — host already has `nvidia-container-toolkit`. Enabling the operator's toolkit DaemonSet would overwrite the `/etc/containerd/config.toml` the host package configured (via `nvidia-ctk runtime configure`), breaking the host runtime wiring. Matches NVIDIA's documented guidance and the existing `examples/values-native-talos.yaml` reference, which pins the same flag for the same reason. - `devicePlugin.enabled=true` — publish `nvidia.com/gpu` to the kubelet. - `vfioManager.enabled=false` — do not unbind the host driver. Upstream renders this `true` unconditionally; it only stays at zero pods because no node carries a `vm-passthrough` sandbox assignment when `sandboxWorkloads` is off — pinned off so the rendered policy matches intent. - `cdi.enabled=false` — upstream defaults this `true`, but CDI spec generation is serviced by the operator-managed toolkit DaemonSet this variant disables. An apt host doesn't auto-generate CDI specs (`nvidia-ctk cdi generate` is a manual step) and uses the classic `nvidia-container-runtime` hook, so leaving CDI on would point the device plugin at CDI injection with no specs on the host and silently break allocation. - `sandboxWorkloads.enabled=false` — regular pod workloads, not VMs. With this off, upstream `defaultWorkload` is already `container`, so no override is needed. Host prerequisite: because the operator's toolkit component is disabled, the node's containerd must already have the `nvidia` runtime registered (`nvidia-ctk runtime configure --runtime=containerd`); `apt install nvidia-container-toolkit` installs binaries only and does not wire containerd. With `driver.enabled=false` the operator uses the pre-installed host driver at its standard location, so a stock apt install needs no `hostPaths.driverInstallDir` override (Talos lands the driver under a non-standard prefix — see `examples/values-native-talos.yaml`). The bundle wiring (`packages/core/platform/templates/bundles/iaas.yaml`) already enables `cozystack.gpu-operator` as an optional package — no bundle changes required. Users opt into this variant by setting `variant: container` in their `Package` CR. Verified locally with `helm template` against `values.yaml` + `values-container.yaml`: ClusterPolicy renders with `driver.enabled=false`, `toolkit.enabled=false`, `devicePlugin.enabled=true`, `vfioManager.enabled=false`, `cdi.enabled=false`, `sandboxWorkloads.enabled=false`. Platform `helm unittest` and the `gpu-operator` variant bats suite are green. Not yet smoke-tested on a real Ubuntu GPU node — the config matches the documented apt-host wiring (legacy `nvidia-container-runtime` hook, CDI off), but an end-to-end CUDA-pod run on hardware is still owed. Closes #2764 ### Release note ```release-note feat(gpu-operator): add `container` variant for hosts with a pre-installed NVIDIA driver and nvidia-container-toolkit, exposing containerized GPU workloads via the standard NVIDIA device plugin without VFIO passthrough or vGPU. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a "container" GPU operator variant to support containerized GPU workloads on hosts with the NVIDIA driver and nvidia-container-toolkit. * **Documentation** * Updated variants documentation and OS support table to include the container variant and per-host requirements (including Ubuntu and Talos notes). * Clarified Talos native pod example wording. * **Tests** * Added regression tests to verify container-variant configuration and component enablement/disablement. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
fb070ffa21 |
feat(hack): rewrite check-readiness as a Go command with test suite (#2755)
## What this PR does Replaces the `hack/check-readiness.sh` diagnostic with a Go command at `cmd/check-readiness`, integrated into the build the same way as `cozypkg`: - `make check-readiness` builds `_out/bin/check-readiness`, with `Version` injected via `-ldflags`. - The tool shells out to `kubectl` and reports only **non-ready** cozystack / Flux / Kubernetes resources, parsed from status conditions (not column heuristics). Flags: `--wait`/`--timeout`, `--watch`, `--core`, `--parallel`, `--verbose`, `-n`/`-l` scoping, `--no-color`, `--version`. Adds a black-box golden test suite under `test/check-readiness/`: - A mock `kubectl` serves canned fixtures; the suite diffs stdout/stderr/exit against golden files across 30 scenarios (arg parsing, condition/phase/suspend handling, CRD-not-installed, kubectl arg propagation, wait/timeout, edge cases) plus a PTY test for color output. - `make test-check-readiness` runs it; wired into `make unit-tests`. Supersedes #2740, which added the same feature set to the bash script. ### Release note ```release-note feat(hack): replace check-readiness.sh with a `check-readiness` Go command (build with `make check-readiness`); reports non-ready cozystack/Flux/Kubernetes resources with --wait/--watch/--core/--parallel/--verbose and namespace/label scoping. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added `check-readiness` command for verifying Kubernetes resource readiness status across namespaces * Supports multiple execution modes: one-shot, wait with timeout, and watch with configurable intervals * Includes namespace filtering, label selectors, parallel resource fetching, and color output control * **Tests** * Added comprehensive test suite validating command behavior across various scenarios <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2755?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8b4d31866e |
ci: add Go-scoped Renovate configuration (#2850)
## What this PR does Raises the OpenSSF Scorecard **Dependency-Update-Tool** check (was **0/10**) by adding a Renovate configuration at `.github/renovate.json`. Scorecard scores this check on the presence of a recognized config file, so the check turns green as soon as this merges. ### Why Renovate (not Dependabot) Renovate is already the cozystack org standard — it actively raises and merges PRs on `cozystack/ansible-cozystack`, and `cozystack/talm` + `cozystack/boot-to-talos` already ship `renovate.json`. The Renovate GitHub App is authorized on the org, so this config is picked up **without creating a new credential**. There is no Dependabot usage anywhere in the org. > If the app installation is scoped to "selected repositories", `cozystack/cozystack` needs to be toggled into the existing installation by an org admin — a one-click change, not a new auth artifact. The Scorecard check does not depend on that toggle. ### Scoped to Go dependencies only (per the 2026-05-27 decision) - **`enabledManagers: ["gomod"]`** — Renovate only ever touches Go modules. It will **not** open PRs for Dockerfiles, Helm charts, shipped component/operator versions, or GitHub Actions. Components and images stay hand-managed. - **`gomodTidy` + `gomodUpdateImportPaths`** run after each bump. The `gomod` manager auto-discovers both the root module and the `api/apps/v1alpha1` submodule. - **Kubernetes libraries grouped** (`sigs.k8s.io/controller-runtime` + `k8s.io/**`) so `client-go`/`apimachinery` never bump out of lockstep with controller-runtime (a split bump fails to compile). - **Go toolchain directive pinned** (`matchDepTypes: ["golang"]` disabled) — `go.mod` is generated and a toolchain bump needs a matching `setup-go` + codegen run, out of scope for an automated PR. ### "do not merge" on every bot PR (requirement) - `automerge` and `platformAutomerge` are **off**. - Every Renovate PR is labeled **`do-not-merge/hold`** + **`automated`** on creation. Both labels already exist in `.github/labels.yml`; `stale.yaml` already exempts `do-not-merge/hold` PRs from auto-close, so held bumps won't rot. - Bumps therefore stay parked until tests pass and a maintainer removes the label after a hands-on check. > Note: `do-not-merge/hold` is a **signal/marker** (cozystack uses native GitHub, not Prow Tide). Combined with automerge being off, nothing can auto-merge a held PR. Hard merge-blocking would need a status check tied to the label, which would require branch-protection changes that were explicitly excluded. ### Verification - `renovate-config-validator` (Renovate v42): **Config validated successfully.** - Valid JSON. ### Release note ```release-note ci: add a Go-scoped Renovate configuration (gomod only, no automerge, bot PRs held with do-not-merge/hold) to automate Go dependency updates ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Configured automated dependency management for Go modules, including synchronized updates for Kubernetes-related dependencies, rate limiting on concurrent updates, and manual approval requirements to ensure controlled rollouts. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
28dfb40579 |
fix(dashboard): grant tenant dashboard read on cozy-public PVCs (#2843)
## What this PR does The vm-disk **source image** dropdown in the console lists `PersistentVolumeClaims` in `cozy-public` (the `vm-default-images-*` golden images) to offer them when creating a VM disk. That list is issued by the browser as the **tenant identity** (e.g. `system:serviceaccount:tenant-root:tenant-root`), but the `cozy:tenant:dashboard` Role in `cozy-public` only granted read on Flux `helmrepositories`/`helmcharts`. The list returned `403 Forbidden` and the dropdown stayed empty even when images had been enabled (e.g. via `bundles.enabledPackages`). This is identity-dependent, which is why it can look intermittent: a cluster-admin login bypasses tenant RBAC and sees the images, while any real tenant — including `tenant-root-super-admin` — hits the 403. This PR adds `get`/`list`/`watch` on `persistentvolumeclaims` to the Role. `cozy-public` is a shared public namespace, so read on its PVCs is safe to expose to every tenant dashboard subject (the Role is already bound to the tenant groups and the tenant ServiceAccount). A helm-unittest regression pins the grant. Verified with `kubectl auth can-i list pvc -n cozy-public`: - `--as=system:serviceaccount:tenant-root:tenant-root` → no (before this PR) - `--as-group=tenant-root-super-admin` → no (before this PR) - cluster-admin → yes (why it "worked" for admins) No console code changed — this is an RBAC-only fix. ### Release note ```release-note fix(dashboard): allow tenant dashboards to list VM images in cozy-public, fixing the empty source-image dropdown when creating a vm-disk ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Dashboard access permissions have been expanded to include read-level access to persistent volume claims, enabling users to manage and select images through the dashboard more effectively. * **Tests** * Added test suite to verify dashboard role permissions are correctly configured and functional. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
d037ff3c71 |
fix(platform): order cozystack-basics after the APIs its admission policies type-check (#2842)
## What this PR does `cozystack-basics` ships cluster-wide ValidatingAdmissionPolicies — the gateway / route / tenant host policies — that match `tenants.apps.cozystack.io` (served by `cozystack-api`) and the Gateway API types (`gateways`, `httproutes`, `tlsroutes`). kube-controller-manager's ValidatingAdmissionPolicy *status* type-checker panics with a nil-pointer dereference in `k8s.io/apiserver/pkg/cel/openapi` when it type-checks a policy against a matched type whose schema it cannot resolve yet. During a cold install this is a deadlock. `cozystack-basics` can install before `cozystack-api` is Ready; the tenant-host policy is created; KCM panics type-checking it against the not-yet-served `tenants` schema and CrashLoops; that stalls cert/token issuance, so `cozystack-api` never starts and the schema never resolves — KCM keeps crashing and every HelmRelease times out on `DependencyNotReady`. This adds `dependsOn` edges so `cozystack-basics` installs only after `cozystack-engine` (which serves `tenants.apps.cozystack.io` via `cozystack-api`) and `gateway-api-crds` are Ready. The policies are then never created before the APIs they type-check are served. A template capability gate (`.Capabilities.APIVersions.Has`) was considered and rejected: operator-generated HelmReleases run without drift detection, so helm-controller is checksum-driven — it would render the policy out at first install and never re-add it once the API appears, silently dropping the protection. Ordering via `dependsOn` gates the first install on Ready, which is reliable. No cycle risk: nothing `dependsOn` `cozystack-basics`, and no `cozystack-engine` component consumes a `cozystack-basics` output, so engine becomes Ready independently. Includes a helm-unittest regression test asserting the `dependsOn` edges stay. The underlying nil-pointer panic is an upstream Kubernetes bug (the best-effort status type-checker should never crash kube-controller-manager); it is unguarded as of the current release and will be reported separately. This change removes the trigger on the cozystack side. ### Release note ```release-note fix(platform): order cozystack-basics after the APIs its admission policies type-check, fixing a cold-install deadlock where kube-controller-manager could crash type-checking policies against APIs (tenants, Gateway API) not yet served ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Bug Fixes** * Improved cluster stability by implementing proper dependency ordering for validation policies during initialization, preventing resource configuration conflicts. * **Tests** * Added integration tests to validate cluster-wide policy dependencies and ordering, ensuring continuous system stability verification. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
5982018629 |
feat(monitoring): add tenant overview dashboard (#2809)
## What Adds a cross-tenant **Tenant Overview** dashboard to the root/infra Grafana. ## Why Platform admins currently have no single view of which tenants consume what. This shows resource utilization, app counts, and health per tenant at a glance. ## Panels - **Fleet summary** — tenants, apps, running pods, CPU/memory/storage totals - **Tenant leaderboard** — per-tenant table: CPU & memory (usage vs requests), storage, pods, PVCs, app count, not-ready apps - **Top consumers** — Top-N bar gauges by CPU, memory, storage, app count - **Trends** — per-tenant CPU & memory usage over time - **Health** — pod restart rate and not-ready HelmReleases per tenant ## Details - Registered in `dashboards-infra.list` only, so it deploys to the root/`cozy-monitoring` Grafana and never exposes cross-tenant data on per-tenant instances (same pattern as `gpu/gpu-tenants`). - Apps are counted from `gotk_resource_info` HelmReleases, excluding the paired `*-system`/`cozy-*` charts. - Uses only standard cadvisor / kube-state-metrics / Flux metrics. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## New Features * Added a Tenant Overview dashboard for comprehensive tenant infrastructure monitoring, including: * Fleet summary with tenant counts, app deployments, and aggregate resource usage * Tenant performance leaderboard with CPU, memory, and storage metrics * Top consumers visualization by resource type and app count * Historical usage trends and pod health metrics <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c74964bf25 |
docs(release): expand release.md into release-engineer playbook (#2732)
## What this PR does Expands `docs/release.md` from a ~165-line process-design doc into a ~573-line single source covering both the process design (kept verbatim — mermaid diagrams and the RC/regular/patch numbered walkthroughs) and the operational playbook for cutting a release. New sections added under the existing diagrams: - **What CI does on a tag push** — phase-by-phase walk through `tags.yaml` (`prepare-release`, `generate-changelog`, `update-website-docs`), `pull-requests-release.yaml` (finalize), and `update-releasenotes.yaml` (sync release body). - **Automated patch tag** — `auto-release.yaml`'s nightly behavior. - **Backports** — bot mechanism, recursive-backport regression history (fix #2584), manual fallback, cherry-pick triage include/skip rule, monotonic-counter trap for migration backports. - **Changelogs** — pointer to `agents/changelog.md` plus the recurring failure modes that have shipped in real changelogs, with a verification snippet (chart-bump coverage, range-vs-changelog PR sweep, URL liveness, website PR-author check). - **Release-time fires** — three customer-impacting regressions (v1.3.0 cert-manager ingressClassName, v1.4.0 kubevirt-instancetypes upgrade, v1.4.0 Flux 2.7 → 2.8 readiness deadlock) with "what would have caught this" notes. - **Pre-release verification checklist**, **CI failures release engineers commonly hit**, **Force-retagging** (with open RFC #2677 context), **Splitting a release-blocking bundle PR**, **Cleanup after release**. Example version numbers across the walkthrough switched from the upstream `v0.42.x` legacy placeholders to simple semver-style (`v1.2.0`, `v1.2.1`, `release-1.2`) consistent with semver.org examples and applicable at any point in v1.x. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Enhanced release process documentation with clarified CI workflows, comprehensive verification checklists, expanded troubleshooting guidance, and case studies to ensure more reliable releases. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2732?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c961ffe286 |
feat(ci): test-impact analysis E2E (default-on) + release E2E workflow (#2559)
## What this PR does
Adds **test-impact analysis (TIA)** to the PR E2E workflow so that only
bats files affected by the diff are exercised, and a **release-time E2E
workflow** that runs the full suite on every release tag to keep
coverage on the shipped code.
### 1. Test-impact analysis selector — `hack/select-e2e.sh`
Reads `packages/core/platform/sources/*.yaml` (the PackageSource
dependency graph) and emits the bats files affected by a PR diff. The
walk is the same one `cozypkg dot` renders.
| Input | Output |
|---|---|
| `packages/apps/postgres/values.yaml` | `postgres` |
| `packages/system/postgres-operator/...` | `postgres harbor`
(transitive) |
| `packages/system/cilium/...` | full suite (cilium has many transitive
`*-application` dependents) |
| `packages/library/cozy-lib/...` | full suite (library affects
everything) |
| `hack/e2e-install-cozystack.bats` | full suite (shared install affects
all apps) |
| `hack/e2e-apps/redis.bats` | `redis` (per-app edit, never escalates) |
| `docs/agents/contributing.md` | (empty — no E2E impact) |
| `dashboards/gpu/gpu-fleet.json` | (empty) |
Conservative fallbacks:
- An unrecognised `packages/*` path or a system source with no
`*-application` descendants escalates to the full suite, so a path
inside the graph is never silently dropped.
- Per-app bats edits are matched **before** the full-suite trigger, so
editing one bats file selects only that app rather than escalating.
### 2. Default-on TIA + `full-e2e` opt-out — `pull-requests.yaml`
- The `Select E2E tests` step runs by default and produces `apps` +
`skip` outputs.
- `Run E2E tests` runs unless `skip=true` (docs / dashboards / `*.md`
only).
- Adding the `full-e2e` label skips the selector step (output empty) and
falls through to the unchanged full-bats-list path — the safety hatch
when reviewers want the full suite on a PR.
### 3. `release-e2e.yaml` — full E2E on every release tag
Triggered on `v*.*.*`, `v*.*.*-rc.*`, `v*.*.*-beta.*`, `v*.*.*-alpha.*`.
Builds images + Talos image, then runs the full suite. Independent of
`tags.yaml`'s release publish flow — failure here is an alarm, not a
release blocker.
This closes the coverage gap left by making PR runs default to TIA: the
full suite is now exercised at tag-cut time against the shipped code.
## Tests
`hack/select-e2e_test.bats` covers 11 cases — single app, transitive
operator, networking/library full-suite, docs/dashboards skip,
`kubernetes-application` two-bats mapping, shared E2E helper, install
bats, per-app bats no-escalation, release-e2e workflow change.
YAML parse verified for both new workflow files.
## Workflow injection safety
All `${{ }}` contexts that could carry untrusted data
(`github.base_ref`, label arrays) flow through env vars, never directly
into shell text. `github.workspace` is also indirected through
`WORKSPACE` env in the release workflow.
### Release note
```
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* CI now runs a Test Impact Analysis to pick only affected apps,
reducing E2E runtime.
* Added a release-level E2E workflow that runs full acceptance tests for
version tags with deterministic sandboxing, artifact capture,
conditional retries, teardown, and report upload.
* E2E jobs accept selected-apps metadata and support conditional
full-suite overrides.
* **Tests**
* New test suite validates selection logic, operator expansion,
full-suite escalation, and docs-only/no-op behaviors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
931f286b9f |
fix(ingress,platform): deliver publishing.proxyProtocol to host ingress-nginx (#2799)
## What this PR does `publishing.proxyProtocol: true` is documented to enable PROXY-protocol on the host ingress-nginx, but the three nginx config keys (`use-proxy-protocol`, `real-ip-header`, `enable-real-ip`) never reached the controller. They were written into the `ingress-system` component values of the `cozystack.ingress-application` Package, but that component has no `install:` block, so the package controller never builds a HelmRelease for it and the values are dropped. External traffic behind a PROXY-protocol L4 load balancer kept breaking. This routes the switch through `_cluster.proxy-protocol` in the `cozystack-values` Secret — the same path `expose-external-ips` already travels to reach the host ingress release — and the `extra/ingress` chart applies it to the external-facing ingress (the namespace named by `expose-ingress`). The dead component-values injection is removed. The `extra/ingress` chart also gains a first-class `proxyProtocol` value, so PROXY-protocol can be enabled per Ingress application (a per-app boolean knob like `cloudflareProxy`). Two `fail` guards keep the new knobs safe: the per-app `proxyProtocol` value is rejected on the external-facing (host) ingress — that path must go through `publishing.proxyProtocol`, which also installs ouroboros (hairpin handling) and arms the disable-hazard guard — and `cloudflareProxy` + `proxyProtocol` together are refused, since Cloudflare's HTTP proxy does not speak PROXY-protocol to origins and its `real_ip_header` server-snippet would override `real-ip-header: proxy_protocol`. `use-forwarded-headers` / `compute-full-forwarded-for` are deliberately not set: without a paired `proxy-real-ip-cidr` they would let clients spoof `X-Forwarded-For`, and the PROXY-protocol frame is already the source of truth for the client address. helm-unittest coverage pins runtime delivery on the host path, the per-app path, and that non-host tenants stay untouched; the previous test that only checked template emission into the (non-consumed) Package component is replaced. Affected versions: v1.4.0–v1.4.2 and current main (introduced in #2561). **Upgrade note (action required).** Because the flag was a silent no-op before this fix, a v1.4.x cluster that set `publishing.proxyProtocol: true` aspirationally — without its L4 load balancer actually injecting PROXY frames — kept working by accident. After this fix the flag really enables `use-proxy-protocol`, which makes ingress-nginx require a PROXY frame on every connection; if the LB does not send one, all external ingress breaks. Before upgrading, confirm the upstream LB injects PROXY-protocol v1 frames, or unset `publishing.proxyProtocol`. Closes #2796 ### Release note ```release-note fix(ingress): publishing.proxyProtocol now enables PROXY-protocol on the host ingress-nginx (the config was silently dropped before reaching the controller); the Ingress application also gains a proxyProtocol value to enable PROXY-protocol per-app. ACTION REQUIRED on upgrade: the flag was previously a no-op, so a cluster that set it without the upstream LB injecting PROXY-protocol v1 frames will lose all external ingress once it takes effect — verify the LB, or unset publishing.proxyProtocol before upgrading. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tenant ingress support for PROXY-protocol via a new proxyProtocol option (default: false). * Platform-level publishing.proxyProtocol now controls external/host ingress behavior. * **Bug Fixes / Validation** * Rejects enabling per-app proxyProtocol on external-facing host ingresses. * Rejects using proxyProtocol together with cloudflareProxy to prevent conflicting real‑IP handling. * **Documentation** * Updated docs describing proxyProtocol usage, defaults, and safety requirements. * **Tests** * Added tests covering proxyProtocol wiring, config emission, and failure cases. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
479929ebfb |
ci(e2e): drop 3x retry on Run E2E + Install Cozystack, wait out gateway.bats tenant teardown (#2558)
## What this PR does
Drops the 3× retry loop on `Run E2E tests` and `Install Cozystack into
sandbox`. `Prepare environment` keeps its 3× retry — that step is pure
infrastructure (Talos image download, sandbox VM boot, network) where
transient runner hiccups warrant a retry.
On failure, the test step now captures `kubectl get hr -A -o wide` and
`kubectl get events -A` under a collapsible group so triage starts with
the actual broken-state snapshot.
> [!NOTE]
> An earlier revision of this PR also doubled every bats timeout. That
commit was dropped in a rebase and is intentionally **not restored**:
the timeout class that actually matters (per-app HR-Ready waits) has
since been standardized at 5m on `main` (
|
||
|
|
ed367f30dd |
test(packages/system/seaweedfs): cover s3.trafficDistribution version gating
Pin the native s3.trafficDistribution behaviour that replaced the dropped s3-traffic-distribution.patch: PreferClose is kept on Kubernetes 1.33, auto-converts to PreferSameZone on 1.35, and the field is omitted below the 1.31 gate. Adds the helm-unittest harness (test target + suite) the package previously lacked. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
068af203ac |
chore(packages/system/seaweedfs): bump seaweedfs to 4.31
Re-vendor the SeaweedFS Helm chart from 4.05 to 4.31. Improvements that land with the binary: the PostgreSQL filer store moves to pgx v5 with PgBouncer support and an ON CONFLICT upsert default, HTTPS certificates hot-reload without a pod restart on cert-manager rotation, and more S3/volume Prometheus metrics are exported. The S3 service now exposes a metrics port by default. Adaptations: - Drop s3-traffic-distribution.patch: native s3.trafficDistribution support landed upstream in chart 4.12. The native field is gated to Kubernetes >=1.31 and auto-converts to PreferSameZone on >=1.35, which the hardcoded patch did not do. Set s3.trafficDistribution: PreferClose in values instead. - Migrate global.* to global.seaweedfs.* in values: chart 4.18 namespaced the app-specific globals to avoid polluting the shared global namespace when the chart is used as a subchart. Covers enableSecurity, serviceAccountName, extraEnvironmentVars and monitoring. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c32812ccf9 |
fix(platform): deliver publishing.proxyProtocol to host ingress-nginx
The host PROXY-protocol switch wrote the nginx config keys into the ingress-system component values of the cozystack.ingress-application Package. That component has no install block, so the package controller never built a HelmRelease for it and the values were dropped — the host ingress-nginx never received use-proxy-protocol and external traffic behind a PROXY-protocol L4 load balancer kept breaking. Route the switch through _cluster.proxy-protocol in the cozystack-values Secret instead, the same path expose-external-ips travels to reach the host ingress release, and drop the dead component-values injection. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
8df4b0b0f1 |
feat(ingress): add proxyProtocol knob to ingress-nginx
Expose a proxyProtocol value on the Ingress application that turns on ingress-nginx use-proxy-protocol with real-ip-header=proxy_protocol and enable-real-ip. The host control plane also enables it through the _cluster.proxy-protocol switch, applied only to the external-facing ingress (the namespace named by expose-ingress). use-forwarded-headers / compute-full-forwarded-for are deliberately not set: without a paired proxy-real-ip-cidr they let clients spoof X-Forwarded-For, and the PROXY-protocol frame is already the source of truth for the client address. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
0d6aaf93ac |
ci: use autobuild build-mode for CodeQL Go
CodeQL 2.25.x rejects build-mode 'none' for Go ('Go does not support the
none build mode; use autobuild or manual'), which failed the initial run.
Switch to autobuild and add a SHA-pinned setup-go (go-version-file: go.mod)
so the autobuilder uses the project's Go toolchain rather than whatever the
runner ships. go build ./... already compiles cleanly, so autobuild
succeeds.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
|
||
|
|
540c4a1315 |
ci: add CodeQL workflow for Go (SAST)
Raise the OpenSSF Scorecard SAST check (was 0/10) by adding GitHub's recommended CodeQL analysis for Go. Runs on push and PR to main plus a weekly schedule, and uploads results to the code-scanning dashboard. - build-mode: none extracts Go source without compiling, so the job cannot fail on the monorepo's generated files, build tags, or the api/apps/v1alpha1 submodule. It still produces real alerts and is credited by Scorecard. Can be switched to autobuild later for deeper build-aware analysis. - Top-level token is read-only; the analyze job opts into security-events: write only. - codeql-action pinned to the same v3 SHA already used by scorecard.yml; checkout pinned by SHA. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-By: Claude <noreply@anthropic.com> |
||
|
|
5d72a32e11 |
fix(build): read --cache-from from ghcr :latest so PR builds warm-start (#2855)
## What this PR does [#2711](https://github.com/cozystack/cozystack/pull/2711) moved PR builds to unique `pr-<N>-<sha>` tags (`PUBLISH_FLOATING=0`) to end concurrent-push 409 conflicts, but left `--cache-from` pointing at `$(REGISTRY)/<img>:latest`. PR builds set `REGISTRY` to the per-CI registry, which never publishes `:latest`, so every PR build's registry cache lookup `404`s and rebuilds cold — slowing builds and, on a loaded runner, pushing them into the 30-minute job timeout. This adds a `CACHE_REGISTRY` knob (default `ghcr.io/cozystack/cozystack`, where release builds *do* publish `:latest`) and points every `--cache-from` at it. Push targets (`$(REGISTRY)`) and the unique per-PR tags are unchanged, so #2711's anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release. - `hack/common-envs.mk`: new `CACHE_REGISTRY ?= ghcr.io/cozystack/cozystack` with rationale. - 31 package Makefiles: 37 `--cache-from type=registry,ref=$(REGISTRY)/…` → `$(CACHE_REGISTRY)/…`. Verified with `make -n image REGISTRY=iad.ocir.io/…`: push tag stays on OCIR (`…/cozystack-controller:pr-<N>-<sha>`), cache-from now reads `ghcr.io/cozystack/cozystack/cozystack-controller:latest`. Makefile-only change — no generated artifacts affected. > Note: cache freshness is bounded by the last release's `:latest`. PR-to-PR freshness (a `main`-built `:buildcache` tag) is a possible follow-up, intentionally out of scope here. ### Release note ```release-note fix(build): PR CI image builds now read layer cache from ghcr.io `:latest` (published by releases) instead of the per-CI registry where `:latest` was never pushed after #2711, restoring warm-start caching for PR builds. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated Docker build configuration across multiple components to use a dedicated cache registry for improved build layer reuse. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
9868c43e53 |
fix(build): read --cache-from from ghcr :latest so PR builds warm-start
PR #2711 switched PR builds to push only :pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to stop concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry (iad.ocir.io) and never publish :latest there, so every PR build's registry cache lookup 404s and rebuilds cold. Add a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and point every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so the #2711 anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
ac56d4cea1 |
ci: add Go-scoped Renovate configuration
Raise the OpenSSF Scorecard Dependency-Update-Tool check (was 0/10) by adding a Renovate config. Renovate is already the cozystack org standard (active on ansible-cozystack, talm, boot-to-talos), so the app is authorized org-wide and this file is picked up without a new credential. Scoped deliberately so automated bumps only ever touch Go dependencies: - enabledManagers: [gomod] — Renovate ignores Dockerfiles, Helm charts, shipped component/operator versions, and GitHub Actions. Components and images stay hand-managed, per the 2026-05-27 decision. - gomodTidy + gomodUpdateImportPaths run after each bump; the gomod manager auto-discovers both the root module and the api/apps/v1alpha1 submodule. - controller-runtime and k8s.io/** are grouped so client-go and controller-runtime never bump out of lockstep. - The Go toolchain directive is pinned (matchDepTypes golang disabled): go.mod is generated and a toolchain bump needs a matching setup-go and a codegen run, out of scope for an automated PR. - automerge/platformAutomerge are off and every Renovate PR is labeled do-not-merge/hold + automated, so bumps are held until tests pass and a maintainer reviews them by hand. Validated with renovate-config-validator (config validated successfully). Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-By: Claude <noreply@anthropic.com> |
||
|
|
117d317fd8 |
fix(platform): drop fragile numeric vGPU mediatedDeviceTypes defaults
The vgpu defaults shipped a mediatedDevicesConfiguration block whose mediatedDeviceTypes were hardcoded nvidia-NNN ids. Those ids are not stable identifiers — they are the per-(SKU, driver) indices under which a host enumerates mdev types in /sys/.../mdev_supported_types/, so the same profile gets a different number on a different card or driver branch. Auto-injecting them made KubeVirt try to create mdev types that usually do not exist on the operator's nodes, and duplicated work the GPU Operator's vGPU Device Manager already does. Delegate mdev creation to the vGPU Device Manager, which is name-based and stable, and keep only the name-based permittedHostDevices.mediatedDevices exposure list (mdevNameSelector) in the defaults. KubeVirt-driven creation stays available through the .gpu.mediatedDevicesConfiguration override for operators who supply host-verified type ids; document the lookup recipe. Update the bundle wiring, the gpu/vgpu values comments, the vGPU guide and the tests accordingly (the default vgpu path now emits no mediatedDevicesConfiguration; the override path is covered explicitly). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
919073601d |
fix(platform): match NVIDIA passthrough defaults to the resourceName the plugin advertises
The auto-injected permittedHostDevices only works if each resourceName equals what nvidia-sandbox-device-plugin advertises on the node; otherwise the admission webhook rejects the VM — the very manual step this bundle removes. Several defaults did not match the slug the plugin (nvcr.io/nvidia/kubevirt-gpu-device-plugin) derives from the PCI-IDs database name, which uppercases the name and keeps the die suffix, the Tesla brand and the form-factor/memory tokens: 10DE:1EB8 TU104GL_T4 -> TU104GL_TESLA_T4 10DE:20B7 GA100_A30 -> GA100GL_A30_PCIE 10DE:2331 GH100_H100_PCIE_80GB-> GH100_H100_PCIE 10DE:2321 GH100_H100_NVL_94GB -> GH100_H100L_94GB 10DE:1DB1/1DB4/1DF6 + new 1DB5 -> GV100GL_TESLA_V100_* family Device 10DE:1DB1 was also mislabelled: it is the SXM2 16GB part, not the 32GB one — the 32GB SXM2 V100 is a distinct device (10DE:1DB5) that was missing entirely. Relabel 1DB1 as 16GB and add 1DB5. Restate the header comment as the actual generation rule and pin the corrected slugs (T4, A30, both V100 SXM2 parts) with a regression test that also asserts the tidy/short forms never reappear. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
285f42aaeb |
fix(kubevirt): reinsert each GPU CR patch independently in make update
The update target gated the insertion of both the permittedHostDevices and mediatedDevicesConfiguration directives behind a single `grep permittedHostDevices` guard. A hand-merge from a newer upstream KubeVirt CR that dropped only mediatedDevicesConfiguration left permittedHostDevices in place, so the guard skipped the combined sed and mediatedDevicesConfiguration was never reinserted — then the sanity-check tail aborted the target with a non-zero exit instead of self-healing. Split the two directives into independent guarded inserts so each self-heals on its own. Reword the sanity-check failures to name the directive that failed to land and the anchor its insert keys off, instead of implying the anchor itself is missing. Add a shell regression test that strips one directive at a time and asserts `make update` restores it without duplicating the other; the update target gains a CR_FILE override so the test can drive it against a throwaway copy. Wired into the package test target and guarded to skip on BSD sed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
da0dcf9edf |
test(platform): pin nil-safe GPU bundle wiring against null gpu values
The .Values.gpu accesses in the iaas bundle are fully parenthesized
((( .Values.gpu ).x).y) so the field walk stays nil-safe when an operator
nulls the whole gpu block or leaves a sub-key blank (an empty YAML key
parses to null, which Helm then drops from .Values). A bare dotted walk
would panic with "nil pointer evaluating interface {}.<field>" on the
first nil hop.
Add regression cases covering gpu: null in both passthrough and vgpu
variants plus blank sub-keys, and a comment warning against flattening
the parenthesized form. A mutation check confirms the cases fail with the
nil-pointer panic the moment the parentheses are removed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
|
cb3c84e24d |
docs(gpu-vgpu): drop the manual KubeVirt patch step
The platform now wires permittedHostDevices and the HostDevices feature gate into the KubeVirt CR automatically whenever cozystack.gpu-operator is in bundles.enabledPackages. The KubeVirt configuration section now describes the auto-wiring contract, the per-variant defaults, the .gpu.permittedHostDevices / .gpu.replaceDefaults escape hatches, and calls out the sandbox-device-plugin resourceName slug convention so operators know where to look when a node's allocatable name does not match the platform default. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
91efa1f4eb |
feat(platform): auto-wire KubeVirt host-device support when gpu-operator is enabled
When cozystack.gpu-operator is in bundles.enabledPackages, the platform
now mirrors the chosen GPU variant into the KubeVirt CR — the HostDevices
feature gate is appended to developerConfiguration.featureGates, and a
starter permittedHostDevices block is filled from a per-variant defaults
table. Without this, a passthrough-variant GPU shows up under
.status.allocatable on the node but the KubeVirt admission webhook
rejects every VM that references it as a hostDevices.deviceName.
Two defaults tables ship under packages/core/platform/files/:
- gpu-passthrough-defaults.yaml covers Hopper (H100/H200), Ada
Lovelace (L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10),
Turing (T4), Volta (V100/V100S). PCI vendor:device pairs are
stable; resourceName slugs mirror nvidia-sandbox-device-plugin's
<arch>_<model>_<form>_<mem> convention. externalResourceProvider
is set on every entry because the resources are advertised by
the sandbox-device-plugin DaemonSet, not by KubeVirt itself.
- gpu-vgpu-defaults.yaml covers the Pascal-Ampere mdev profile
range the upstream vgpu-device-manager walks /sys/class/mdev_bus/
for. Ada Lovelace and Blackwell SR-IOV vGPU is out of scope for
the chart's default list (NVIDIA driver path differs) and is
expected to be advertised via the user-override hook below.
bundles.iaas.gpuOperatorVariant picks which table applies (default =
passthrough, vgpu = mediated). The same variant flows through to the
gpu-operator Package CR, replacing the previous always-default call.
Operators extend or replace the defaults via .gpu in platform values:
gpu:
replaceDefaults: false
permittedHostDevices:
pciHostDevices: [...]
mediatedDevices: [...]
mediatedDevicesConfiguration: {...}
replaceDefaults: false (the default) appends the user list to the
NVIDIA table; true drops the table entirely for clusters with only
non-NVIDIA accelerators or a strict allowlist requirement.
Companion helm-unittest suite covers the eight branches: no gpu
package emits no extras; gpu-operator at default variant emits the
HostDevices gate plus pciHostDevices; vgpu variant emits the
HostDevices gate plus mediatedDevices and mediatedDevicesConfiguration;
user overrides append by default; replaceDefaults wipes; unknown
variant fails fast.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
|
8c507fe4c5 |
feat(kubevirt): parameterize CR for host-device GPU configuration
The KubeVirt CR's developerConfiguration.featureGates list, the
permittedHostDevices block, and mediatedDevicesConfiguration are now
driven by chart values:
- extraFeatureGates is appended after the static default set so
HostDevices (or any other future gate) can be enabled without
forking the template.
- permittedHostDevices is rendered verbatim when set, allowing the
pciHostDevices (passthrough) and mediatedDevices (vGPU) shapes the
KubeVirt admission webhook requires for VMs that reference a GPU
by resourceName.
- mediatedDevicesConfiguration is rendered verbatim when set,
carrying mediatedDeviceTypes / nodeMediatedDeviceTypes for the
vGPU profile auto-creation loop.
Defaults stay empty so the rendered CR is byte-identical to the
previous template when no overrides are supplied.
The Makefile's update target re-applies the four sed transforms
in-place on templates/kubevirt-cr.yaml — it does not download anything
from upstream KubeVirt because the published CR is a near-empty stub
and would clobber the hand-curated cozystack defaults. To refresh from
a newer upstream release the operator hand-merges the new fields into
the in-tree template, then runs `make update` to re-assert the four
cozystack parameterization patches; grep sanity-checks at the end of
the target fail loudly if any anchor (developerConfiguration,
- VMExport, evictionStrategy) gets dropped during the hand-merge.
Companion helm-unittest coverage pins the default render and the
three populated shapes.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
|
9352d3ee0e |
feat(gpu-operator): add container variant for preinstalled host driver
The two existing variants assume the host has neither the NVIDIA driver
nor the container toolkit: default unbinds the host driver to bind
vfio-pci for KubeVirt VMs, vgpu requires the proprietary vGPU host
driver and a license server. Neither path produces a working setup on a
node that already ships the driver and nvidia-container-toolkit via the
distro package manager — the most common shape of a Linux GPU node.
The new container variant surfaces the third architectural mode that
upstream NVIDIA gpu-operator already supports: sandboxWorkloads
disabled, devicePlugin enabled, with every component that would
conflict with the host install switched off.
- driver.enabled=false use the host driver
- toolkit.enabled=false host already has nvidia-container-toolkit;
enabling overwrites the containerd
config.toml the host configured via
nvidia-ctk runtime configure
- devicePlugin.enabled=true publish nvidia.com/gpu to the kubelet
- vfioManager.enabled=false do not unbind the host driver
- cdi.enabled=false CDI spec generation is serviced by the
disabled toolkit; an apt host uses the
legacy nvidia-container-runtime hook, so
pin off to match the host runtime wiring
- sandboxWorkloads.enabled=false regular pod workloads, not VMs
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
|
80fbfa1ed8 |
fix(networking): point host ouroboros proxy at the root-tenant ingress (#2800)
## What this PR does With `publishing.proxyProtocol: true`, the host emits a `cozystack.ouroboros` Package using the no-override Package macro, so the proxy inherits the wrapper chart default backend `ingress-nginx-controller.cozy-ingress-nginx`. That layout describes a managed Kubernetes tenant cluster, not the host: on the host, `extra/ingress` deploys ingress-nginx into the root tenant namespace as `root-ingress-controller` in `tenant-root`. The composed backend FQDN never resolves, so `ouroboros-proxy` never reaches Ready. This emits the host ouroboros Package with a `proxy.target` override derived from `publishing.ingressName` — namespace plus `<short>-ingress-controller` service name — mirroring the `trimPrefix "tenant-"` transform `extra/ingress` applies to its release namespace. The derivation reuses the existing platform invariant that `publishing.ingressName` is the namespace where the host ingress controller runs. Adds bundle helm-unittest coverage for the default ingress name and a non-default one, and corrects the misleading wrapper `values.yaml` comment that claimed the host uses `cozy-ingress-nginx`. Scope: this fixes the ouroboros readiness defect only. The separate defect where the injected nginx PROXY-protocol config keys are not consumed by the host ingress is tracked independently and not touched here. Closes #2797 ### Release note ```release-note fix(networking): point the host ouroboros PROXY-protocol proxy at the root-tenant ingress so it becomes Ready when publishing.proxyProtocol is enabled ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enhanced proxy protocol configuration to automatically route through the appropriate ingress-nginx service based on your ingress name settings. * **Tests** * Added test coverage for proxy protocol bundle wiring and ingress service routing. * **Documentation** * Clarified proxy target configuration behavior and fallback mechanisms. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
be13ac833e |
fix(api): emit initial-events-end bookmark for core.cozystack.io watches (#2786)
## What this PR does While preparing for my CozySummit talk of last week (creating commits from CozySummit edits with [gitops-reverser](https://github.com/ConfigButler/gitops-reverser)) I noticed warnings on my console: ``` Warning: event bookmark expired err="reflector.go:343: hasn't received required bookmark event marking the end of initial events stream, received last event 59.998946102s ago" The warning fires on every reflector restart and continuously while the cache is unsynced from the WatchList perspective. ``` I've been searching a bit better and it turns out that `TenantSecret`, `TenantModule`, and `TenantNamespace` (group `core.cozystack.io`) all ignore `ListOptions.SendInitialEvents`. Interestingly enough `apps.cozystack.io/Application` already does implement this in the right way. Under the [WatchList / streaming-list protocol](https://kubernetes.io/docs/reference/using-api/api-concepts/#streaming-lists) (client-go's `WatchListClient`, default-on since v1.35), clients expect the initial `ADDED` events to be terminated by a [`Bookmark`](https://kubernetes.io/docs/reference/using-api/api-concepts/#watch-bookmarks) annotated with `k8s.io/initial-events-end: "true"`. These three resources only forwarded the backing watcher's bookmarks (which lack the annotation), so the marker was never sent and client-go reflectors never reached `HasSynced` — looping on `reflector.go:343: hasn't received required bookmark event marking the end of initial events stream`. **Fix:** extracted the contract into a shared `registry.InitialEventsBookmarker` helper (`pkg/registry/utils.go`) and wired it into all four `Watch` implementations. It annotates the first backing bookmark as the terminating marker, with fallbacks to emit it before the first live event and on watcher close. The annotation reuses the upstream `metav1.InitialEventsAnnotationKey` constant rather than a string literal. The full bookmark machinery in `k8s.io/apiserver/pkg/storage/cacher` isn't reusable here: it's internal to the watch-cache layer and only fires for cacher-backed storage, whereas these resources translate a controller-runtime watch in custom REST storage that bypasses the cacher. `Application` is refactored onto the helper with its `WorkloadMonitor` event-draining preserved. Adds unit tests for the helper and watch tests for each core resource (`ADDED` → annotated `Bookmark` → live event). ### Screenshots N/A — no UI changes. ### Release note ```release-note fix(api): core.cozystack.io resources (TenantSecret, TenantModule, TenantNamespace) now emit the k8s.io/initial-events-end bookmark for SendInitialEvents=true (WatchList) watches, so client-go informers reach HasSynced instead of looping on the missing-bookmark warning. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Watch endpoints reliably emit the "initial-events-end" bookmark after initial events, improving watch/list contract compliance and client reflector synchronization. * **Refactor** * Watch implementations consolidated around a centralized initial-events bookmarker and stop-aware event forwarding for consistent bookmark behavior. * **New Features** * Introduced a reusable initial-events bookmarker to coordinate bookmark emission and resourceVersion observation across translated watch streams. * **Tests** * Added unit and integration tests validating bookmark emission, ordering, resourceVersion seeding/observation, and close-time behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
bb322138ec |
fix(dashboard): grant tenant dashboard read on cozy-public PVCs
The vm-disk source image dropdown in the console lists PersistentVolumeClaims in cozy-public (the vm-default-images-* golden images). The browser issues that list as the tenant identity (e.g. system:serviceaccount:tenant-root:tenant-root), but the cozy:tenant:dashboard Role only granted read on Flux helmrepositories and helmcharts. The list returned 403 and the dropdown stayed empty even when the images existed. Add get/list/watch on persistentvolumeclaims to the Role. cozy-public is a shared public namespace, so read on its PVCs is safe to expose to every tenant dashboard subject. A helm-unittest regression pins the grant. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
56014f1fdc |
fix(platform): order cozystack-basics after the APIs its admission policies type-check
cozystack-basics ships cluster-wide ValidatingAdmissionPolicies matching
tenants.apps.cozystack.io and the Gateway API types. KCM's VAP status
type-checker panics on a nil schema when it type-checks a policy against a
matched type that isn't resolvable yet; during a cold install this crashes
KCM, which stalls cert/token issuance, so cozystack-api never starts and the
tenants schema never resolves — a bootstrap deadlock.
Add dependsOn edges so the package installs only after cozystack-engine
(cozystack-api) and gateway-api-crds are Ready. Drift detection is off on
operator-generated HelmReleases, so a template capability gate would drop the
policy at first install and never re-add it; ordering is the reliable fix.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
|
03f1e7a55a |
fix(platform): order cozystack-basics after the APIs its admission policies type-check
cozystack-basics ships cluster-wide ValidatingAdmissionPolicies matching tenants.apps.cozystack.io and the Gateway API types. KCM's VAP status type-checker panics on a nil schema when it type-checks a policy against a matched type that isn't resolvable yet; during a cold install this crashes KCM, which stalls cert/token issuance, so cozystack-api never starts and the tenants schema never resolves — a bootstrap deadlock. Add dependsOn edges so the package installs only after cozystack-engine (cozystack-api) and gateway-api-crds are Ready. Drift detection is off on operator-generated HelmReleases, so a template capability gate would drop the policy at first install and never re-add it; ordering is the reliable fix. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
53edfc1f50 |
test(e2e): capture COSI bucket state in failure diagnostics and cozyreport
Run 27057500380 failed on harbor's BucketClaim never reaching bucketReady=true, and neither artifact could explain why: the per-app diagnostics dump has no bucket objects, and cozyreport collects neither COSI resources nor objectstorage-controller / seaweedfs provisioner logs — by the time the report is taken, the bucket state has moved on. Add a COSI module to cozyreport.sh (bucketclaims/bucketaccesses/ buckets/classes as custom-column tables plus full YAML, controller and per-namespace provisioner logs) and a bucket-readiness snapshot to the per-app failure dump in the PR workflow. The COSI CRDs ship no printer columns, so plain kubectl get would show only NAME/AGE — readiness fields are pulled explicitly. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
5b986e0a47 |
test(e2e): wait out nested-tenant teardown in gateway.bats
The last three gateway tests create nested tenants (gwprop, gwparent/gwchild, rparent/rchild) and deleted them fire-and-forget, with parent and child deletes back-to-back. The file exits in ~25s while the uninstalls run on for minutes: each tenant uninstall blocks on a cleanup Job in cozy-system, and a parent deleted alongside its child wedges on the child's still-terminating namespace. On the tenants helm-controller shard (--concurrent=5) the leftovers from one gateway.bats run — three stuck parent uninstalls, one stuck child uninstall, and the rchild HR still mid-install — occupied exactly 5 workers, so the next test in line (harbor) sat unreconciled for its whole 5m HR-Ready budget and failed without ever erroring. Sequence the teardown instead: delete child, wait for its HR to go away, then delete parent and wait again. The waits are hard failures on purpose — if a tenant uninstall genuinely wedges, gateway.bats is the test that should go red, not whichever app test runs next. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
d8d4c71e0f |
ci(pull-requests): dump cluster state on Install Cozystack failure
Mirror the diagnostics group already on `Run E2E tests` to the `Install Cozystack into sandbox` step. Without this, install-step failures (e.g. an HR not reaching Ready inside the bats `kubectl wait` window) surface only as "error: timed out waiting for the condition on helmreleases/<name>" with no cluster snapshot — re-running is the only way to triage, which is the exact pattern the parent commit removed. On failure, dump three groups: - `kubectl get hr -A -o wide` (last 100 lines): which HRs exist and their Ready/Status columns. - `kubectl describe hr` for each not-Ready HR (tail -50): the Conditions block surfaces helm install/upgrade errors, missing CRDs, and chart-render failures — the actual cause. - `kubectl get events -A --sort-by=.lastTimestamp` (last 50): cluster events that often pinpoint the failing pod/PVC/job. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
199781a7a5 |
ci(pull-requests): report total attempt count in prepare-env retry log
Address review feedback from coderabbitai on .github/workflows/pull-requests.yaml:235: $attempt is incremented only on failure, so first-try success printed "completed after 0 attempts". Use $((attempt + 1)) so the message reflects the total number of runs. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
e3b54cb201 |
ci(pull-requests): drop 3x retry on Run E2E + Install Cozystack
Audit of 30 successful PR runs found that across 5 sampled failure attempts, **25/25 retries** on `Run E2E tests` failed — the retry loop never recovered a flake, only stretched deterministic failures and tripled diagnostic wall-time. Same data shape on `Install Cozystack`. Drop both. `Prepare environment` keeps its 3x retry — that step is pure infrastructure (Talos image download, sandbox VM boot, network) where transient runner hiccups warrant a retry. On failure, the test step now captures `kubectl get hr -A -o wide` and `kubectl get events -A` under a collapsible group so triage starts with the actual broken-state snapshot. Depends on: - #2508 — installer namespace bootstrap (Helm namespace-ownership conflict) - #2509 — operator HelmRelease config knobs (`seaweedfs-system` 2-min wait race) - #2528 — harbor bucket-secret + BucketInfo gating (harbor ValuesError) - #2529 — objectstorage-controller BucketAccess conflict retry - the daniil/split-vminstance PR (vminstance disk race + VM IP/ready timeouts) - the daniil/split-event-driven PR (existence backstops surfacing real errors) Until those land, dropping the retry will fail CI for unrelated PRs that hit the seaweedfs / harbor / installer / vminstance races. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-By: Claude <noreply@anthropic.com> |
||
|
|
36b07dd703 |
fix(platform): install velero by default (#2833)
## What this PR does `backupstrategy-controller` has been installed by default since |
||
|
|
bc03142c08 | feat(ouroboros): bump to v0.8.0 (#2807) | ||
|
|
2e23198b44 |
fix(platform): install velero by default
backupstrategy-controller is installed by default since |
||
|
|
797cac40ab |
fix(dashboard): keep nodes with no running pods in unrequested panels
The == 1 phase filter drops the requests vector for nodes that have no running pods, so the allocatable - requests subtraction silently omits those nodes from the Unrequested CPU/Memory panels and totals. Default the right-hand side to 0 per node via 'or sum by (node) (allocatable * 0)'. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
26574f08f3 |
ci(release): repair orphaned draft tag_name on retag (#2761)
## What this PR does When a release-cut PR is merged after the underlying git tag was deleted and re-created (e.g. the release branch was force-updated between the initial tag push and the merge), GitHub orphans the corresponding draft release — its `tag_name` becomes `untagged-<hash>` while `name` is preserved. The "Publish draft release" step in `pull-requests-release.yaml` looks up the draft strictly by `tag_name === tag` and throws `Draft release for <tag> not found`, blocking the release. Observed on run [#26558920814](https://github.com/cozystack/cozystack/actions/runs/26558920814/job/78236959976) cutting v1.4.2, and the same error fired on the earlier [v1.4.1 finalize run](https://github.com/cozystack/cozystack/actions/runs/26271301257). This change adds a fallback: when the primary lookup misses, search for a draft where `name === tag` and `tag_name` matches `/^untagged-/`. If found, repair the draft's `tag_name` to the real tag via `updateRelease` before publishing, so the released artifact correctly points at the git tag created earlier in the same job. The strict-fail path is preserved when neither lookup matches. The fix is forward-only and does **not** auto-recover the currently stuck v1.4.2 draft — that needs a one-time manual `tag_name` patch + job re-run, or publishing the draft via the GitHub UI. ### Release note ```release-note ci(release): the release-finalize workflow now repairs an orphaned draft release (tag_name == `untagged-<hash>` after a tag delete + re-create) by re-linking it to the actual tag before publishing. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Improved the robustness of the release publication process to handle edge cases when locating draft releases, ensuring more reliable and consistent release deployments. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2761?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.5.0-alpha.2 api/apps/v1alpha1/v1.5.0-alpha.2 |