mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-04 02:57:27 +09:00
fix/vm-disk-cross-storageclass-clone
417
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
b6108039b7 |
feat(gpu-operator): add container variant for preinstalled host driver (#2766)
## What this PR does Add a third architectural variant to `cozystack.gpu-operator` for hosts where the NVIDIA driver and `nvidia-container-toolkit` are already installed by the distro package manager (the typical Ubuntu/Debian GPU node shape: apt-installed driver + apt-installed CT). The existing variants do not produce a working setup in that shape. `default` (passthrough) runs `vfioManager`, which unbinds the host driver to bind `vfio-pci` — with the host driver active, `k8s-driver-manager` early-exits and the bind never happens, so the GPU never reaches a pod. `vgpu` requires the proprietary NVIDIA vGPU host driver and a license server; the user-installed open or proprietary driver does not satisfy it. The new `container` variant maps onto upstream gpu-operator's third mode (`sandboxWorkloads.enabled=false` + `devicePlugin.enabled=true`) with every component that would conflict with the host install switched off: - `driver.enabled=false` — use the host driver. - `toolkit.enabled=false` — host already has `nvidia-container-toolkit`. Enabling the operator's toolkit DaemonSet would overwrite the `/etc/containerd/config.toml` the host package configured (via `nvidia-ctk runtime configure`), breaking the host runtime wiring. Matches NVIDIA's documented guidance and the existing `examples/values-native-talos.yaml` reference, which pins the same flag for the same reason. - `devicePlugin.enabled=true` — publish `nvidia.com/gpu` to the kubelet. - `vfioManager.enabled=false` — do not unbind the host driver. Upstream renders this `true` unconditionally; it only stays at zero pods because no node carries a `vm-passthrough` sandbox assignment when `sandboxWorkloads` is off — pinned off so the rendered policy matches intent. - `cdi.enabled=false` — upstream defaults this `true`, but CDI spec generation is serviced by the operator-managed toolkit DaemonSet this variant disables. An apt host doesn't auto-generate CDI specs (`nvidia-ctk cdi generate` is a manual step) and uses the classic `nvidia-container-runtime` hook, so leaving CDI on would point the device plugin at CDI injection with no specs on the host and silently break allocation. - `sandboxWorkloads.enabled=false` — regular pod workloads, not VMs. With this off, upstream `defaultWorkload` is already `container`, so no override is needed. Host prerequisite: because the operator's toolkit component is disabled, the node's containerd must already have the `nvidia` runtime registered (`nvidia-ctk runtime configure --runtime=containerd`); `apt install nvidia-container-toolkit` installs binaries only and does not wire containerd. With `driver.enabled=false` the operator uses the pre-installed host driver at its standard location, so a stock apt install needs no `hostPaths.driverInstallDir` override (Talos lands the driver under a non-standard prefix — see `examples/values-native-talos.yaml`). The bundle wiring (`packages/core/platform/templates/bundles/iaas.yaml`) already enables `cozystack.gpu-operator` as an optional package — no bundle changes required. Users opt into this variant by setting `variant: container` in their `Package` CR. Verified locally with `helm template` against `values.yaml` + `values-container.yaml`: ClusterPolicy renders with `driver.enabled=false`, `toolkit.enabled=false`, `devicePlugin.enabled=true`, `vfioManager.enabled=false`, `cdi.enabled=false`, `sandboxWorkloads.enabled=false`. Platform `helm unittest` and the `gpu-operator` variant bats suite are green. Not yet smoke-tested on a real Ubuntu GPU node — the config matches the documented apt-host wiring (legacy `nvidia-container-runtime` hook, CDI off), but an end-to-end CUDA-pod run on hardware is still owed. Closes #2764 ### Release note ```release-note feat(gpu-operator): add `container` variant for hosts with a pre-installed NVIDIA driver and nvidia-container-toolkit, exposing containerized GPU workloads via the standard NVIDIA device plugin without VFIO passthrough or vGPU. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a "container" GPU operator variant to support containerized GPU workloads on hosts with the NVIDIA driver and nvidia-container-toolkit. * **Documentation** * Updated variants documentation and OS support table to include the container variant and per-host requirements (including Ubuntu and Talos notes). * Clarified Talos native pod example wording. * **Tests** * Added regression tests to verify container-variant configuration and component enablement/disablement. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
fb070ffa21 |
feat(hack): rewrite check-readiness as a Go command with test suite (#2755)
## What this PR does Replaces the `hack/check-readiness.sh` diagnostic with a Go command at `cmd/check-readiness`, integrated into the build the same way as `cozypkg`: - `make check-readiness` builds `_out/bin/check-readiness`, with `Version` injected via `-ldflags`. - The tool shells out to `kubectl` and reports only **non-ready** cozystack / Flux / Kubernetes resources, parsed from status conditions (not column heuristics). Flags: `--wait`/`--timeout`, `--watch`, `--core`, `--parallel`, `--verbose`, `-n`/`-l` scoping, `--no-color`, `--version`. Adds a black-box golden test suite under `test/check-readiness/`: - A mock `kubectl` serves canned fixtures; the suite diffs stdout/stderr/exit against golden files across 30 scenarios (arg parsing, condition/phase/suspend handling, CRD-not-installed, kubectl arg propagation, wait/timeout, edge cases) plus a PTY test for color output. - `make test-check-readiness` runs it; wired into `make unit-tests`. Supersedes #2740, which added the same feature set to the bash script. ### Release note ```release-note feat(hack): replace check-readiness.sh with a `check-readiness` Go command (build with `make check-readiness`); reports non-ready cozystack/Flux/Kubernetes resources with --wait/--watch/--core/--parallel/--verbose and namespace/label scoping. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added `check-readiness` command for verifying Kubernetes resource readiness status across namespaces * Supports multiple execution modes: one-shot, wait with timeout, and watch with configurable intervals * Includes namespace filtering, label selectors, parallel resource fetching, and color output control * **Tests** * Added comprehensive test suite validating command behavior across various scenarios <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2755?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
c961ffe286 |
feat(ci): test-impact analysis E2E (default-on) + release E2E workflow (#2559)
## What this PR does
Adds **test-impact analysis (TIA)** to the PR E2E workflow so that only
bats files affected by the diff are exercised, and a **release-time E2E
workflow** that runs the full suite on every release tag to keep
coverage on the shipped code.
### 1. Test-impact analysis selector — `hack/select-e2e.sh`
Reads `packages/core/platform/sources/*.yaml` (the PackageSource
dependency graph) and emits the bats files affected by a PR diff. The
walk is the same one `cozypkg dot` renders.
| Input | Output |
|---|---|
| `packages/apps/postgres/values.yaml` | `postgres` |
| `packages/system/postgres-operator/...` | `postgres harbor`
(transitive) |
| `packages/system/cilium/...` | full suite (cilium has many transitive
`*-application` dependents) |
| `packages/library/cozy-lib/...` | full suite (library affects
everything) |
| `hack/e2e-install-cozystack.bats` | full suite (shared install affects
all apps) |
| `hack/e2e-apps/redis.bats` | `redis` (per-app edit, never escalates) |
| `docs/agents/contributing.md` | (empty — no E2E impact) |
| `dashboards/gpu/gpu-fleet.json` | (empty) |
Conservative fallbacks:
- An unrecognised `packages/*` path or a system source with no
`*-application` descendants escalates to the full suite, so a path
inside the graph is never silently dropped.
- Per-app bats edits are matched **before** the full-suite trigger, so
editing one bats file selects only that app rather than escalating.
### 2. Default-on TIA + `full-e2e` opt-out — `pull-requests.yaml`
- The `Select E2E tests` step runs by default and produces `apps` +
`skip` outputs.
- `Run E2E tests` runs unless `skip=true` (docs / dashboards / `*.md`
only).
- Adding the `full-e2e` label skips the selector step (output empty) and
falls through to the unchanged full-bats-list path — the safety hatch
when reviewers want the full suite on a PR.
### 3. `release-e2e.yaml` — full E2E on every release tag
Triggered on `v*.*.*`, `v*.*.*-rc.*`, `v*.*.*-beta.*`, `v*.*.*-alpha.*`.
Builds images + Talos image, then runs the full suite. Independent of
`tags.yaml`'s release publish flow — failure here is an alarm, not a
release blocker.
This closes the coverage gap left by making PR runs default to TIA: the
full suite is now exercised at tag-cut time against the shipped code.
## Tests
`hack/select-e2e_test.bats` covers 11 cases — single app, transitive
operator, networking/library full-suite, docs/dashboards skip,
`kubernetes-application` two-bats mapping, shared E2E helper, install
bats, per-app bats no-escalation, release-e2e workflow change.
YAML parse verified for both new workflow files.
## Workflow injection safety
All `${{ }}` contexts that could carry untrusted data
(`github.base_ref`, label arrays) flow through env vars, never directly
into shell text. `github.workspace` is also indirected through
`WORKSPACE` env in the release workflow.
### Release note
```
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* CI now runs a Test Impact Analysis to pick only affected apps,
reducing E2E runtime.
* Added a release-level E2E workflow that runs full acceptance tests for
version tags with deterministic sandboxing, artifact capture,
conditional retries, teardown, and report upload.
* E2E jobs accept selected-apps metadata and support conditional
full-suite overrides.
* **Tests**
* New test suite validates selection logic, operator expansion,
full-suite escalation, and docs-only/no-op behaviors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
479929ebfb |
ci(e2e): drop 3x retry on Run E2E + Install Cozystack, wait out gateway.bats tenant teardown (#2558)
## What this PR does
Drops the 3× retry loop on `Run E2E tests` and `Install Cozystack into
sandbox`. `Prepare environment` keeps its 3× retry — that step is pure
infrastructure (Talos image download, sandbox VM boot, network) where
transient runner hiccups warrant a retry.
On failure, the test step now captures `kubectl get hr -A -o wide` and
`kubectl get events -A` under a collapsible group so triage starts with
the actual broken-state snapshot.
> [!NOTE]
> An earlier revision of this PR also doubled every bats timeout. That
commit was dropped in a rebase and is intentionally **not restored**:
the timeout class that actually matters (per-app HR-Ready waits) has
since been standardized at 5m on `main` (
|
||
|
Myasnikov Daniil
|
9868c43e53 |
fix(build): read --cache-from from ghcr :latest so PR builds warm-start
PR #2711 switched PR builds to push only :pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to stop concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry (iad.ocir.io) and never publish :latest there, so every PR build's registry cache lookup 404s and rebuilds cold. Add a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and point every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so the #2711 anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
9352d3ee0e |
feat(gpu-operator): add container variant for preinstalled host driver
The two existing variants assume the host has neither the NVIDIA driver
nor the container toolkit: default unbinds the host driver to bind
vfio-pci for KubeVirt VMs, vgpu requires the proprietary vGPU host
driver and a license server. Neither path produces a working setup on a
node that already ships the driver and nvidia-container-toolkit via the
distro package manager — the most common shape of a Linux GPU node.
The new container variant surfaces the third architectural mode that
upstream NVIDIA gpu-operator already supports: sandboxWorkloads
disabled, devicePlugin enabled, with every component that would
conflict with the host install switched off.
- driver.enabled=false use the host driver
- toolkit.enabled=false host already has nvidia-container-toolkit;
enabling overwrites the containerd
config.toml the host configured via
nvidia-ctk runtime configure
- devicePlugin.enabled=true publish nvidia.com/gpu to the kubelet
- vfioManager.enabled=false do not unbind the host driver
- cdi.enabled=false CDI spec generation is serviced by the
disabled toolkit; an apt host uses the
legacy nvidia-container-runtime hook, so
pin off to match the host runtime wiring
- sandboxWorkloads.enabled=false regular pod workloads, not VMs
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Myasnikov Daniil
|
53edfc1f50 |
test(e2e): capture COSI bucket state in failure diagnostics and cozyreport
Run 27057500380 failed on harbor's BucketClaim never reaching bucketReady=true, and neither artifact could explain why: the per-app diagnostics dump has no bucket objects, and cozyreport collects neither COSI resources nor objectstorage-controller / seaweedfs provisioner logs — by the time the report is taken, the bucket state has moved on. Add a COSI module to cozyreport.sh (bucketclaims/bucketaccesses/ buckets/classes as custom-column tables plus full YAML, controller and per-namespace provisioner logs) and a bucket-readiness snapshot to the per-app failure dump in the PR workflow. The COSI CRDs ship no printer columns, so plain kubectl get would show only NAME/AGE — readiness fields are pulled explicitly. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
5b986e0a47 |
test(e2e): wait out nested-tenant teardown in gateway.bats
The last three gateway tests create nested tenants (gwprop, gwparent/gwchild, rparent/rchild) and deleted them fire-and-forget, with parent and child deletes back-to-back. The file exits in ~25s while the uninstalls run on for minutes: each tenant uninstall blocks on a cleanup Job in cozy-system, and a parent deleted alongside its child wedges on the child's still-terminating namespace. On the tenants helm-controller shard (--concurrent=5) the leftovers from one gateway.bats run — three stuck parent uninstalls, one stuck child uninstall, and the rchild HR still mid-install — occupied exactly 5 workers, so the next test in line (harbor) sat unreconciled for its whole 5m HR-Ready budget and failed without ever erroring. Sequence the teardown instead: delete child, wait for its HR to go away, then delete parent and wait again. The waits are hard failures on purpose — if a tenant uninstall genuinely wedges, gateway.bats is the test that should go red, not whichever app test runs next. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
67b1a960dd |
test(e2e): fail install test when HelmReleases are not Ready
The HR readiness gate in e2e-install-cozystack.bats was toothless in two
layers: the backgrounded kubectl-wait fan-out discarded child exit codes
(POSIX `wait` without args returns 0), and the final check echoed
"Some HelmReleases failed to reconcile" without exiting non-zero — a
regression from
|
||
|
Aleksei Sviridkin
|
2363f2d9be |
fix(backups): carry dropdown option sources in CRD annotations, not schema
The four backups.cozystack.io CRDs injected the x-cozystack-options vendor
extension into their OpenAPI validation schema. apiextensions JSONSchemaProps
is a closed struct that only preserves x-kubernetes-* vendor extensions, so
helm-controller's server-side apply rejected the unknown key ("field not
declared in schema") and the CRDs never applied — the controller crash-looped
and backups were non-functional.
Move the field-to-source mapping the dashboard needs into CRD
metadata.annotations (preserved by the apiserver), emitted via
+kubebuilder:metadata:annotations markers, and drop the post-processing awk
injector. The dashboard reattaches the extension client-side.
A helm-unittest suite pins the contract: each dropdown field carries its source
in metadata.annotations and must not carry x-cozystack-options in the schema,
guarding against re-introducing the injector.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Andrei KvapilandClaude
|
4dc2367fe4 |
test(tests): exclude loop devices from host LVM scanning
Add an /etc/lvm/lvm.conf overwrite to the e2e Talos machine config so the host LVM ignores /dev/loop devices (alongside drbd/dm/zd). This prevents the host from scanning and activating volume groups inside loop-mounted disk images used during e2e. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
4c34e0f427 |
build(codegen): reproduce x-cozystack-options on backup CRDs after controller-gen
controller-gen cannot emit the x-cozystack-options vendor extension, so the annotations on the backup CRDs (backupjobs/backups/plans/restorejobs) were hand-edited into generated output — the next make generate would strip them, breaking the appkind/plan/backup form dropdowns and failing the codegen-drift check. Add a deterministic awk pass that re-applies the extension after controller-gen, driven by per-CRD field targets. It inserts lines without reserializing the YAML, so it reproduces the committed CRDs byte-for-byte and is independent of the host yq version. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
myasnikovdaniil
|
c435efa199 |
feat(flux): upgrade to v2.8.0 + chart fixes for strict SSA & kstatus (folds #2612) (#2602)
## What this PR does Upgrades Flux v2.7.3 → v2.8.0 across both the vendored tenant chart and the embedded management-cluster manifests, and folds in the chart fixes that become hard errors under the new strict helm-controller v1.5. Flux v2.8's helm-controller v1.5.0 ships: - **Server-Side Apply with `--force-conflicts`** — strict CRD schema enforcement; misplaced fields (silently dropped on v2.7) now fail the apply. - **kstatus-based health checking** by default — parent HR waits for every applied resource (including child HRs) to be Ready before flipping its own Ready, surfacing latent ordering deadlocks. - **Helm v4 embedded** — `upgrade.force: true` is deprecated and now conflicts with SSA. - **`HelmRelease.spec.healthCheckExprs`** — prerequisite for proper readiness gating (used in PR #2601 split). Folds in PR #2612 (kubevirt-instancetypes null TPM fix) since the same Flux upgrade triggers it. ## Commits **Flux upgrade itself:** - `feat(fluxcd)`: bump `flux-operator` / `flux-instance` vendored charts to v0.48.0; web UI opt-in. - `feat(flux)`: regenerate embedded management-cluster manifests via `make update` in `packages/core/flux-aio` (timoni bundle build). **Chart fixes for strict SSA — fields the chart sent that v2.7 silently dropped, v2.8 rejects:** - `fix(kubevirt-instancetypes)`: drop persistent strip that produced null `preferredTPM` (folds #2612). - `fix(foundationdb)`: relocate `faultDomain`, `imageType`, `labels`, `minimumUptimeSecondsForBounce` from inside `automationOptions` to direct children of `spec`. - `fix(kafka)`: place `enableServiceLinks` under `template.pod`, not a phantom `template.spec`. - `fix(vm-instance)`: emit `disk: {}` (not `disk:`/null) when no bus is set. - `fix(platform)`: drop deprecated `upgrade.force: true` from HelmReleases; fix `kafka` WorkloadMonitor `replicas` paths. **Ordering / deadlock fixes under v2.8 kstatus:** - `fix(vpa)`: break circular wait between parent install and nested `vpa-for-vpa` HR. - `fix(kubernetes)`: drop lookup-guarded parent-HR `dependsOn` on tenant addon child HRs (parent waits on child via kstatus, child waited on parent — deadlock). **E2E waits for v2.8 kstatus timing:** - `test(e2e)`: bump app HR-Ready waits to 5m (was 20s–100s under v2.7's faster dispatch). - `test(e2e)`: wait for parent HR Ready before downstream asserts in `run-kubernetes.sh` and `vminstance.bats`. ## Scope discipline This PR is part of the split of #2619 (the consolidated CI fixes branch) into review-friendly pieces. Companion PRs: - **PR #2601** (seaweedfs split) — folded into this PR (commits `29c6afc8`, `0e8b46d7`, `7157158c`, `dccdeb52`, `f880b324`): the seaweedfs-system → seaweedfs-db + seaweedfs-system split, its adoption migration 43 (targetVersion 44), and the configurable db resources all land here, because the strict-SSA `upgrade.force` removal and the kstatus parent-HR timeout bump only make sense together with the split. #2601 is superseded. - **PR #2558** (drop 3× retry on `Run E2E` + `Install Cozystack`) — independent, lands separately. - Several smaller standalone fixes lifted out of #2619 (startup probes, cert-manager `dependsOn`, prepull machinery, CSI HR timeout, NFS/OIDC test improvements) — opened as separate PRs. ## Verification - `helm template` renders cleanly for both `fluxcd` and `fluxcd-operator` packages with `web.enabled=false` (default) and `web.enabled=true`. - Embedded `cmd/cozystack-operator` binary contains the v1.5.0 / v1.8.0 / v2.1.0 controller image strings. - No references to the v0.39-removed `--disable-wait-interruption` flag anywhere in `packages/` or `internal/`. ### Release note ```release-note Flux upgraded to v2.8.0 (helm-controller v1.5 — Helm v4 Server-Side Apply with --force-conflicts, kstatus health checking). When upgrading existing clusters: - Kubernetes 1.33+ is now required for the platform (management) cluster, and for any tenant cluster that enables the optional (default-off) Flux addon — that addon ships the bumped Flux too. - HelmReleases no longer set `upgrade.force: true`. Helm v4 SSA resolves field-ownership conflicts automatically (`--force-conflicts`), but that is not the old client-side replace: immutable-field changes (e.g. StatefulSet volumeClaimTemplates/serviceName) no longer self-heal and require manual recreation — delete the object (e.g. `kubectl delete sts <name> --cascade=orphan`) and let Flux recreate it. - KubeVirt: persistent TPM/EFI is re-enabled for the Windows 11/2k22/2k25 preferences (KubeVirt 1.8 VMPersistentState); each affected VM provisions an extra RWO backend-storage PVC from the default StorageClass. - KubeVirt: the EOL centos.7*/centos.stream8* preferences are retained as deprecated, hidden aliases (`tags: hidden`, `instancetype.kubevirt.io/deprecated: "true"`) — existing VMInstances on these profiles keep rendering and need no action on upgrade, but the profiles are no longer offered for new VMs; repoint to centos.stream9/10 when convenient. The gn1.* GPU instancetypes are likewise retained. - FoundationDB: imageType now reaches the operator (silently dropped pre-SSA); it is pinned to `split` to match the value existing clusters effectively ran, so upgrades stay non-disruptive. Set `imageType: unified` to migrate deliberately. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Flux Status web UI: optional web server, config secret, service port, HTTPRoute/Ingress, network policy, and RBAC roles. * New SeaweedFS DB Helm chart and optional managed DB release. * **Improvements** * CRD/schema enhancements: new provider kinds, validations, variant option, and external checksum refs. * Raised Kubernetes prerequisite to 1.30+; extended e2e timeouts for reliability. * **Chores** * Bumped Flux Operator and Flux versions; documentation links updated to fluxoperator.dev. * **Bug Fixes** * Removed aggressive HelmRelease force-upgrade/install flags. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2602?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
2d58479f06 |
feat(platform): add kuberture as optional system package (#2647)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Adds [kuberture](https://github.com/lexfrei/kuberture) as an optional system package. Kuberture is a small controller that translates the `default/kubernetes` EndpointSlice into headless Services with external-dns-style annotations, closing the gap that external-dns cannot use EndpointSlice as a source directly: ```text EndpointSlice (kubernetes) → kuberture → headless Service(s) with annotations → external-dns → DNS ``` The package is opt-in via `bundles.enabledPackages: [cozystack.kuberture]`. Operators populate `config.outputs` per HelmRelease — each output targets a specific external-dns instance by its `annotationPrefix`, so a single kuberture install can serve multiple external-dns instances (e.g. one publishing to the public zone, another to an internal corporate prefix) simultaneously. The vendored chart at `packages/system/kuberture/charts/kuberture/` is pulled from `oci://ghcr.io/lexfrei/kuberture/charts/kuberture` and pinned by both version AND OCI manifest digest in the Makefile (mirroring the ouroboros lockstep pattern). The controller image at `ghcr.io/lexfrei/kuberture` is pinned by tag+sha256 in the shim values.yaml. Both pins advance in lockstep with each upstream release; the procedure is documented inline. Install-order dependencies declared in `PackageSource`: `cozystack.networking` (CNI before pod scheduling) and `cozystack.prometheus-operator-crds` (ServiceMonitor CRD before the chart's enabled-by-default ServiceMonitor manifest). External-dns is deliberately NOT a dependency — kuberture creates annotated Services regardless of whether external-dns is installed; external-dns is a downstream consumer, not a runtime prerequisite. Coverage: - **helm-unittest** (`packages/system/kuberture/tests/kuberture_test.yaml`, 10 cases): fail-fast on empty outputs, ServiceMonitor enabled by default, source pinned to `default/kubernetes`, multi-output prefix isolation, empty-string `annotationPrefix` schema rejection, omitted-prefix fallback to controller default, ConfigMap → Deployment volume mount, controller image pinned by tag+sha256 digest, Role permits full Service lifecycle, metrics containerPort derived from `metricsBindAddress`. - **bats e2e** (`hack/e2e-apps/kuberture.bats`): applies a `Package cozystack.kuberture` with two outputs (different `annotationPrefix` values), waits for HelmRelease and controller readiness, asserts both output Services are headless and carry the right `hostname`/`target`/`ttl` annotations under their configured prefix, cross-checks per-output isolation (each Service only carries its own prefix's annotations). ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> Not applicable — no UI changes. ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(platform): add kuberture as an optional system package — translates the default/kubernetes EndpointSlice into annotated headless Services so external-dns can publish the Kubernetes API endpoint to DNS. Opt-in via bundles.enabledPackages: [cozystack.kuberture]; populate config.outputs to target one or more external-dns instances by annotationPrefix. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Adds kuberture as an optional system package to publish the Kubernetes API endpoint to DNS via configurable external-dns outputs; includes Helm chart, RBAC, NetworkPolicy, PDB, ServiceMonitor, and validated chart values/schema. * **Tests** * Adds Helm unit tests and an end-to-end smoke test validating controller readiness, headless Service annotations, and external-dns routing/probe behavior. * **Documentation** * Adds a full README with usage, configuration examples, and vendoring/pinning guidance. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2647) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- Documentation companion: [cozystack/website#539](https://github.com/cozystack/website/pull/539). |
||
|
Arsolitt
|
17b2fb70a3 |
feat(kafka): add TLS support via Strimzi listener configuration (#2681)
## What this PR does Adds TLS support to the Kafka managed service via Strimzi listener configuration. - Introduces a `tls.enabled` field that controls the external listener (port 9094). When unset, defaults to the value of `external` (auto-on when external access is enabled). Explicit `tls.enabled` always wins. - Strimzi manages the cluster PKI automatically; the operator publishes `<release>-cluster-ca-cert` and `<release>-clients-ca-cert` for client trust setup. No cert-manager chain is rendered by this chart — the operator owns the chain end-to-end. - The internal TLS listener on 9093 is always on (Strimzi default); this field only governs the external listener on 9094. - External LoadBalancer listener is decoupled from `tls.enabled`: it is gated on `external: true` only. Previously, an explicit `tls.enabled: true` with `external: false` would force the external listener to be rendered. Verified end-to-end on a sandbox cluster: external listener exposes a TLS endpoint signed by the Strimzi-managed cluster CA, client connections with the published trust bundle succeed. Note: this chart does not configure listener authentication. With TLS on the external listener, clients can connect without credentials unless `KafkaUser` resources with `tls`/`scram-sha-512`/`oauth` authentication are configured outside this chart. ### Release note ```release-note feat(kafka): add TLS support on the external listener via Strimzi (tri-state tls.enabled) ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tri-state TLS for the external Kafka listener: explicit on, explicit off, or inherit from the external toggle; chart values now include a top-level tls object. * **Documentation** * Expanded chart docs/README clarifying tls.enabled inheritance, plaintext risk when disabled with external access, and authentication guidance. * **Tests** * Added/updated schema, rendering, RBAC/secret-naming, helm unit, and end-to-end tests covering TLS tri-state and external-listener behavior. * **Bug Fixes** * RBAC and secret-name expectations updated to use the `-cert` suffixed secret names. * **Chores** * Added a Makefile test target to run helm unit tests. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2681?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
a3a7a6d516 |
test(e2e): add HR existence backstop for external-dns ${name}-system
The two external-dns bats tests already poll for the user-facing ${name}
HelmRelease before calling kubectl wait, but they immediately wait on the
${name}-system HR without the same backstop. The system HR is materialised
asynchronously by the controller, so kubectl wait can race and exit non-zero
when the object does not exist yet. Apply the same existence poll pattern
to both occurrences.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
401993ac62 |
test(e2e): wait for parent HR Ready before downstream asserts
Two timing bugs unmasked by Flux v2.8 kstatus, with the same root shape:
the bats steps assumed a side-effect ordering ("by the time X is true,
the parent HR's helm install is done") that held under v2.7's faster
dispatch but breaks under v2.8 where helm-controller walks kstatus on
every applied resource before flipping the HR Ready.
run-kubernetes.sh:
After every addon HR (cilium, coredns, csi, vsnap-crd, ingress-nginx)
reaches Ready, the remediation guard reads `.status.history` on the
parent kubernetes-${test_name} HR. The parent's helm install can still
be Reconciling at that point because kstatus is finishing its full
sweep; `.status.history` is empty until install completes, so the
guard exits 1 with "empty .status.history on a Ready HR". kubernetes-
latest masked this because the ouroboros assertions add ~10m of
buffer; kubernetes-previous (no ouroboros) hit the guard with the
parent still installing. Add an explicit
kubectl wait hr/kubernetes-${test_name} --for=condition=ready
before the guard.
vminstance.bats:
The test waited 120s for `kubectl get vmi vm-instance-$name` to
return a VMI with an assigned IP, then waited for the parent HR Ready
separately afterwards. Under v2.7 the parent HR's helm install
applied the VirtualMachine CR fast enough that kubevirt had created
the VMI before the 120s budget expired. Under v2.8 the dispatch is
slower and the VMI poll returned NotFound for its entire budget.
Reorder: wait for HR Ready first (which implies the VirtualMachine
CR is applied + kstatus-ready), then poll for the VMI with IP.
Loosen the VMI poll budget from 120s to 5m for nested-virt slow paths.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
7b9f286faa |
test(e2e): bump app HR-Ready waits for Flux v2.8 kstatus
Flux helm-controller v1.5.0 (shipped in Flux v2.8) replaced the legacy
Helm readiness logic with kstatus-based health checking, which polls
Deployment/StatefulSet/Service status and waits for kstatus to assess
rollout-ready. This typically adds 10-30s of latency between resource
apply and HR condition Ready=True compared to v2.7.
Our per-app bats tests had `kubectl wait hr/... --for=condition=ready
--timeout=` budgets of 20s, 30s, 60s, 100s — sized for v2.7's faster
readiness flip and previously masked by the 3x retry on Run E2E tests
(dropped in #2558). Under v2.8 with no retry, every app that runs after
the first few hit its HR wait timeout exactly, surfacing as 13 of 16
test failures on PR #2619.
Standardise on 5m for HR Ready across all app tests. Tight enough to
catch genuine bring-up failures within the per-app job budget, loose
enough to absorb kstatus dispatch jitter on a management cluster that
is concurrently reconciling multiple tenant-test HRs.
Post-HR-Ready checks for app-specific resources (StatefulSet/Service/
PVC/operator CRs) remain — for CR-based apps kstatus only observes
that the chart applied, not that the operator finished. Audit of
which of those are actually redundant under v2.8 kstatus is queued
as a follow-up.
Also bumps the 10s "kamajicontrolplane appears" wait in
run-kubernetes.sh to 2m, and the kubernetes-addon HR Ready loop from
1m to 5m for the same reason.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
ea7a8c8675 |
test(e2e): quote HelmRelease names in kubectl wait
Both \${test_name} and \${component} are literal-safe today (no
whitespace, no glob characters), so this is a style-only change — but
quoting variable expansions in kubectl arguments matches the rest of
the file and removes a future-foot-gun if a test name ever grows a
hyphen-y suffix that interacts oddly with completion or globbing.
Scoped to the two HelmRelease wait lines flagged by CodeRabbit and
echoed by the human reviewer; the machinedeployment wait on the
preceding line keeps its existing style.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
64c443caf1 |
test(e2e): split local k8s_version declaration from yq invocation
\`local k8s_version=\$(yq ...)\` masks the exit status of the command substitution — the assignment to a local variable always succeeds, so a failing yq is swallowed. Declare the local first, then assign, so non-zero yq exits propagate via shell errexit semantics if the script ever runs under \`set -e\`. Pure shell hygiene; no behavioral change on the success path. Surfaced by CodeRabbit and confirmed by the human reviewer as non-blocking but worth applying. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
8c6e149177 |
test(e2e): delete standalone ouroboros.bats now folded into kubernetes-latest
The previous commit (3c81261b) folded the ouroboros hairpin-NAT reconciliation assertions into the kubernetes-latest tenant under run-kubernetes.sh, but did not remove the standalone bats file. The e2e workflow auto-discovers every hack/e2e-apps/*.bats and dispatches test-apps-<name> via the wildcard rule in packages/core/testing/Makefile. Leaving ouroboros.bats in the tree meant the assertions ran twice: once on the standalone ~25m Kamaji-backed tenant and again as the folded addon on the kubernetes-latest tenant — the opposite of the consolidation the fold was intended to deliver. The folded assertions in hack/e2e-apps/run-kubernetes.sh cover the same surface (HR Ready, controller pod Running, coredns-custom rewrite-line injection, and end-to-end DNS resolution from inside the tenant), so the standalone file is dead weight after the fold. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
3ecc030a64 |
test(e2e): set keycloakInternalUrl when enabling OIDC
The dashboard gatekeeper (oauth2-proxy) does OIDC discovery against the
external host at container startup. The e2e sandbox uses
host=example.org which does not resolve, so the auth-proxy crashloops:
Get "https://keycloak.example.org/realms/cozy/.well-known/openid-configuration":
dial tcp: lookup keycloak.example.org: no such host
Under Flux v2.8 kstatus the gatekeeper Deployment then flips to
'Failed' (rather than the pre-v2.8 'InProgress') and stalls the
cozy-dashboard/dashboard HelmRelease, which in turn blocks
cozy-fluxcd/flux-plunger via dependsOn. Previous runs masked this with
an unrelated ImagePullBackOff on cozy-dashboard-console; with the
console image fixed the OIDC discovery failure now surfaces.
packages/core/platform/values.yaml:194-198 already declares
authentication.oidc.keycloakInternalUrl. When set, the dashboard
chart (packages/system/dashboard/templates/gatekeeper.yaml:59-66)
renders --skip-oidc-discovery plus explicit redeem/jwks/validate/
backend-logout URLs pointing at the in-cluster keycloak Service.
Browser-facing redirect-url and login-url stay on the external host
(they are not exercised by the e2e flow).
The patch sets the value alongside oidc.enabled in the same
kubectl-patch so the configmap and gatekeeper rollout happen in one
reconcile pass.
Observed on CI run 26142986969 (PR #2619).
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
68a9a6e5ad |
test(e2e): bump NFS pod wait to 10m to fit slow-path chain
Previous run failed kubernetes-latest (and then kubernetes-previous via
cascade) on the 5m budget for nfs-test-pod to reach Succeeded. The
chain is host linstor prime PVC + CDI import + tenant CSI attach +
busybox image pull + containerd CreateContainer (which occasionally
stalls past kubelet's 2m CRI deadline and retries with name-reservation
conflict). ~40s on the happy path, 6m+ in observed worst case.
When kubernetes-latest times out, its tenant cluster (2x 20Gi worker
PVCs) leaks because set -e exits before any cleanup. kubernetes-
previous then runs, adds another 40Gi, and the host tenant-quota
(100Gi) is exhausted -- every downstream app test (mariadb, mongo,
postgres, qdrant, redis, openbao, vminstance) fails with
ErrCreatingPVCPrime / exceeded quota.
Bumping the wait to 10m gives the slow path room. Headroom costs
nothing on green runs since the pod is busybox echo.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
04a54b01cf |
test(e2e): wait for tenant CSI HR Ready before NFS test
kubernetes-previous fails on the NFS pod step. From cozyreport + job log:
- Pod scheduled to tenant worker
- 7x FailedAttachVolume over 35s: CSINode does not contain driver
csi.kubevirt.io (kubevirt-csi-node DaemonSet still rolling out)
- Attach succeeds at T+1m08s after the driver registers
- busybox pull queued ~1m25s, completes at T+2m48s
- CreateContainer in containerd then stalls past kubelet's CRI
deadline; retry trips "failed to reserve container name ... is
reserved for <prev>" -- the 5m pod-Succeeded wait then expires.
Root cause: run-kubernetes.sh creates the NFS PVC+Pod before asserting
kubernetes-${test_name}-csi is Ready. That HR lives in the parent
cluster but installs cozy-kubevirt-csi-node into the tenant via
kubeConfig.secretRef; helm-controller walks kstatus on the tenant
DaemonSet using the tenant kubeconfig. HR Ready in parent therefore
implies kubevirt-csi-node Pods Ready on every tenant worker, which
implies csi.kubevirt.io is published in CSINode. The existing wait at
line 354 runs AFTER the NFS test, so it does not gate the race.
Move the wait earlier with a 10m budget. Post-NFS sibling waits stay
as is, so component coverage is unchanged.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
5035ac1414 |
test(e2e): fold ouroboros assertions into kubernetes-latest
The standalone hack/e2e-apps/ouroboros.bats provisioned a full Kamaji-
backed tenant Kubernetes cluster (control plane + 2 CAPI workers + base
addons) just to flip `addons.ouroboros.enabled=true` on top, then ran
~15m of hairpin-NAT reconciliation assertions. Cluster shape was
identical to what hack/e2e-apps/run-kubernetes.sh already provisions for
kubernetes-latest, so the second Kamaji bringup was pure duplication —
roughly 25-45m of CI wall-time per run.
Add an `enable_ouroboros` 4th parameter to `run_kubernetes_test()`. When
"true", inject the ouroboros addon block into the Kubernetes CR's
addons map and run the post-cluster assertions inline:
- Wait for the ouroboros HR Ready in the host namespace.
- Verify coredns-custom ConfigMap exists in tenant kube-system.
- Wait for the ouroboros controller pod Ready.
- Apply an Ingress with a TLS host in tenant default namespace.
- Poll coredns coredns-custom for the rewrite line (5m budget).
- Resolve the hairpin host from inside the tenant via a netshoot pod
and assert it points at the ouroboros-proxy ClusterIP.
- Clean up dnscheck pod + Ingress.
kubernetes-latest passes "true" (gets ouroboros coverage); kubernetes-
previous leaves it empty (control-plane-only smoke remains).
(The standalone ouroboros.bats was removed in the preceding commit
alongside an unrelated foundationdb chart fix — the git rm staging had
already happened when the foundationdb commit landed.)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Arsolitt
|
532fff8a8c |
test(kafka): assert scalar boolean tls.enabled type in rd-schema check
The drop-null refactor switched the embedded openAPISchema to a scalar "boolean" type (the ["boolean","null"] array form breaks the ApplicationDefinition CRD's own unmarshal of the embedded schema). The rd-schema bats guard still asserted the old array form and started failing; align it with the scalar type the chart now generates. Signed-off-by: Arsolitt <arsolitt@gmail.com> |
||
|
myasnikovdaniil
|
628ebce653 |
feat(platform): add deletion-protection guardrail via ValidatingAdmissionPolicy (#2650)
## What this PR does Adds a `ValidatingAdmissionPolicy` that blocks `DELETE` on critical platform objects labeled `platform.cozystack.io/no-delete=true`. Native, in-process CEL evaluation — no webhook DaemonSet, no Service, no TLS, no image to maintain. Requires Kubernetes 1.30+. This is an alternative to the closed #2402 (admission webhook approach). ### Protected objects (labeled in this PR) | Resource | Kind | Source | |---|---|---| | `cozy-system` | Namespace | `packages/core/installer/templates/cozy-system-{namespace,labels}.yaml` | | `tenant-root` | Namespace | `packages/system/cozystack-basics/templates/tenant-root.yaml` | | `tenant-root` | HelmRelease (ns: tenant-root) | same | | `cozystack-version` | ConfigMap (ns: cozy-system) | `packages/core/platform/templates/cozystack-version.yaml` | | `cozystack-packages` | OCIRepository (ns: cozy-system) | `packages/core/platform/templates/repository.yaml` | | `letsencrypt-prod`, `letsencrypt-stage`, `selfsigned-cluster-issuer` | ClusterIssuer | `packages/system/cert-manager-issuers/templates/cluster-issuers.yaml` | | `linstorcluster` | LinstorCluster | `packages/system/linstor/templates/cluster.yaml` | | `packages.cozystack.io`, `packagesources.cozystack.io` | CRD | label stamped in `internal/crdinstall/install.go` | ### Bypass ```bash kubectl label <kind> <name> platform.cozystack.io/no-delete- kubectl delete <kind> <name> ``` ### Why VAP instead of a webhook Same outcome as #2402 but with zero runtime infrastructure: no DaemonSet, Service, TLS certificate, or image. The kube-apiserver evaluates the CEL `expression: "false"` in-process and denies because `validationActions: [Deny]`. `objectSelector` on the binding scopes evaluation to labeled objects only — every other DELETE in the cluster is unaffected. ### Release note ```release-note feat(platform): add deletion-protection guardrail (ValidatingAdmissionPolicy) blocking DELETE on cozystack platform objects labeled `platform.cozystack.io/no-delete=true`. Bypass by removing the label first. Requires Kubernetes 1.30+. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Deletion-protection guardrail: objects labeled platform.cozystack.io/no-delete=true are denied DELETE requests; removal of the label allows deletion. * **Chores** * Platform migration to v43 backfills the no-delete label onto system resources and namespaces. * **Tools** * Added an "unprotect" teardown utility to remove the no-delete label for uninstall/disaster recovery. * **Documentation** * Helm NOTES now warns when the guardrail is inactive due to missing cluster API. * **Tests** * New tests and e2e checks validate deletion-protection behavior. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2650?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Arsolitt
|
615415d676 |
test(kafka): add cluster-ca-cert checks to kafka-rd secrets bats test
Mirror the existing clients-ca-cert presence/rejection tests for cluster-ca-cert so the suite enforces both Strimzi-managed CA secret names and rejects bare cluster-ca references. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Arsolitt <arsolitt@gmail.com> |
||
|
Arsolitt
|
377e7702ca |
fix(kafka): use printf in kafka-rd-schema bats test
dash's echo interprets backslash escapes by default (unlike bash), which mangles double-backslash sequences in the embedded JSON pattern strings and makes jq report an invalid-escape parse error. printf '%s' passes the value through untouched. Signed-off-by: Arsolitt <arsolitt@gmail.com> |
||
|
Arsolitt
|
ddc3b5204b |
fix(kafka): replace bash-ism in kafka-rd-schema bats test
Replace double-bracket [[ ... ]] with POSIX [ ... ] in hack/check-kafka-rd-schema.bats so it runs under dash, which is what /bin/sh resolves to on the Ubuntu CI runners. The test passed locally because /bin/sh resolves to bash on most developer machines. Signed-off-by: Arsolitt <arsolitt@gmail.com> |
||
|
Arsolitt
|
6f15714846 |
test(kafka): add TLS test coverage
- Add helm-unittest suite covering TLS listener config, tri-state nil/true/false cases - Cover external=false + tls.enabled=false negative combinations - Add dashboard resourcemap tests for CA secret inclusion - Add bats tests for cozyrds schema nullable field and Strimzi CA secret naming Signed-off-by: Arsolitt <arsolitt@gmail.com> |
||
|
myasnikovdaniil
|
d3f282cee1 |
test(e2e): pre-pull platform images via staged-busybox DaemonSet (#2724)
## What this PR does Pre-pulls every image referenced by `packages/system/kubeovn`, `packages/system/linstor`, and `packages/system/cert-manager` onto every node before those HelmReleases install. Cluster-member workloads (OVN raft, LINSTOR, cert-manager webhook) fail when replicas start at different times due to per-node image-pull stagger; pre-pulling means all replicas start with images already cached. ## Mechanics Renders each chart with `helm template`, then runs a `yq` filter that walks every PodSpec-shaped object and emits the images of each container — scoping the output to images the kubelet actually pulls (skips configmap fields and CRD examples that happen to contain an `image:` key). Each render is staged through a tmp file so a helm-template failure trips `set -e` cleanly (cozytest.sh runs `@test` bodies under `/bin/sh`, which is dash on Ubuntu CI — no `pipefail`). The pre-pull DaemonSet itself is the hard part: distroless images (cert-manager and friends) ship no shell and no `/bin/sleep`, so the obvious `command: ["sleep", "infinity"]` fails with `exec: ENOENT`. The script stages a statically-linked `busybox:musl` into a shared `emptyDir` from an initContainer; every prepull container then execs `/shared/sleep` regardless of what the image itself ships. The glibc `busybox` tag is unusable — it needs `/lib64/ld-linux-x86-64.so.2` which distroless lacks. ## Origin Lifted from #2619 — squash of `test(e2e): pre-pull cert-manager images` → POSIX rewrite → cert-manager removal (failed on distroless) → reintroduction with the busybox helper that solves the distroless problem. Final state only — the four iterative commits aren't review-friendly individually. ## Verification - `sh -n hack/e2e-prepull-images.sh` clean - All chart-referenced images held Ready under the DaemonSet on local kind v1.33.1 single-node ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved end-to-end test reliability for platform image pre-pulling by rendering charts separately and consolidating extracted images. * Simplified and consolidated readiness waits during tenant configuration for more robust application readiness checks. * **Chores** * Enhanced image pre-pulling to reliably handle distroless images using a staged helper binary and shared volume approach. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2724?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
f85ec76e19 |
feat(hack): rewrite check-readiness as a Go command with test suite
Replace hack/check-readiness.sh with a Go command at cmd/check-readiness, wired into the build like cozypkg (`make check-readiness`, Version via -ldflags). The tool shells out to kubectl and reports only non-ready cozystack / Flux / Kubernetes resources parsed from status conditions, supporting --wait/--timeout, --watch, --core, --parallel, --verbose, -n/-l scoping, --no-color and --version. Add a black-box golden test suite under test/check-readiness driven by a mock kubectl over fixtures; `make test-check-readiness` runs it and it is wired into `make unit-tests`. Supersedes #2740, which added the same feature set to the bash script. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
2ace581117 |
test(e2e): stage prepull image list to avoid yq pipefail trap
Without pipefail (dash-CI), `yq | hack/e2e-prepull-images.sh` would let a yq failure produce empty input; prepull-images.sh exits 0 on no-input and the test passes with pre-pull silently skipped. Stage the yq output to a tmp file and feed it via stdin so set -e trips on a yq failure. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
76fd5d1791 |
fix(platform): address lllamnyp review on PR #2650
unprotect.sh: add `gitrepository` to the namespaced-kind list so the
platform stays unprotectable when an operator configures
`sourceRef.kind: GitRepository`. Currently only `ocirepository` is
swept, while repository.yaml templates `kind: {{ $sourceRef.kind }}`
unrestricted.
run-migrations.sh: replace the silent skip on a missing migration file
with a hard error. All migrations 1..targetVersion exist in the shipped
image — a gap can only mean a packaging mistake (digest advanced
without the file landing), which previously stalled the cluster at a
stale version while reporting success.
migrations/42: drop the misleading "no 3-way-merge subtlety" claim from
the header. The apply records the no-delete label under the default
field manager's last-applied annotation, so a future label-less apply
by the same field manager would strip it. Document the constraint
instead of denying it.
hack/e2e-install-cozystack.bats: restore `helm.sh/resource-policy: keep`
on the reconstructed cozystack-version ConfigMap in the bypass-path
teardown so cluster state matches the chart template shape
(packages/core/platform/templates/cozystack-version.yaml) on exit.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
155c011dd5 |
test(platform): e2e check that no-delete VAP denies + bypass works
Address review feedback from @lllamnyp on packages/core/platform/templates/
deletion-protection.yaml: the VAP contract had no integration coverage.
A unit test that greps the rendered manifest is the wrong shape — the
contract is "the api-server denies a labeled DELETE with the documented
message" and that needs a real apiserver in the loop.
Add a post-install bats test to hack/e2e-install-cozystack.bats that runs
once cozystack is installed on the e2e cluster:
1. Skips on clusters that pre-date Kubernetes 1.30 (no VAP API).
2. Asserts the cozystack-version ConfigMap actually carries
platform.cozystack.io/no-delete=true — precondition, so the deny
assertion below would not misreport on a regressed binding.
3. Asserts `kubectl delete configmap cozystack-version -n cozy-system`
exits non-zero AND its error message contains the documented
"Deletion blocked: ... platform.cozystack.io/no-delete=true"
substring AND the `--namespace` bypass hint.
4. Confirms the ConfigMap survives the rejected delete.
5. Removes the label, deletes (must succeed), recreates the ConfigMap
with the original data.version and the label so the cluster ends
the test in the same state it started.
This single test catches every regression the PR is meant to prevent:
capability gate inverted, binding objectSelector mistyped,
validationActions flipped Deny→Warn, expression flipped false→true,
label-key drift between the binding and the manifests, and the bypass
docs missing --namespace.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Aleksei Sviridkin
|
3756f5c459 |
feat(networking): Gateway API support via Cilium (supersedes #2213) (#2470)
## What this PR does
Replaces ingress-nginx as the default ingress layer for cozystack-native
services with **Cilium Gateway API**, materialised per-tenant via a new
`TenantGateway` CRD reconciled by `cozystack-controller`. Supersedes
#2213 / #2208. Stacked on top of #2468.
## Architecture
The chart renders one `gateway.cozystack.io/v1alpha1 TenantGateway` CR
per tenant that owns its own Gateway. The cozystack-controller
reconciles the actual `Gateway`, per-tenant `Issuer`, `Certificate`
(wildcard or per-listener), and HTTP→HTTPS redirect `HTTPRoute` from
there. Helm does not render `Gateway` or `Certificate` directly — that
prevents the Helm-vs-controller race on `Gateway.spec.listeners` that
route-driven listener additions would otherwise cause.
```
┌── TenantGateway CR (rendered by extra/gateway chart for owning tenants)
│
└──▶ cozystack-controller reconciles:
├─ Gateway (with dynamic listeners, label-based allowedRoutes)
├─ Issuer (per-tenant ACME account)
├─ Certificate(s) — wildcard + per-child SANs in DNS-01, per-listener in HTTP-01
├─ HTTPRoute (controller-owned http→https redirect)
├─ Namespace label patching (cozy-* system namespaces from spec.AttachedNamespaces)
└─ Watches HTTPRoute / TLSRoute attachments to materialise listeners
```
## Inheritance: tenants attach by default, opt into their own Gateway
A tenant gets a dedicated Gateway (own LB Service, own LB IP, own
Certificate) only when it explicitly asks via
`tenant.spec.gateway=true`. Every other tenant in the tree publishes
through the Gateway of the nearest ancestor that owns one — same shape
as the existing `_namespace.ingress` inheritance.
The `apps/tenant` chart writes a `namespace.cozystack.io/gateway` label
on each tenant namespace, carrying either the tenant's own name (when
owning a Gateway) or the inherited ancestor name (when inheriting). The
Gateway's listener `allowedRoutes.namespaces.selector` is keyed on that
label, so the same selector admits routes from every namespace pointing
at the owner — descendants and cozy-* system namespaces alike.
cozystack-controller patches the same label onto every namespace in
`tgw.Spec.AttachedNamespaces` and garbage-collects labels it wrote when
an entry is removed from the attach list (annotation-gated GC so labels
written by the apps/tenant chart are never stripped).
In DNS-01 mode, the controller extends the parent's wildcard Certificate
with `<child-apex>` + `*.<child-apex>` SANs per inheriting child, and
adds a `*.<child-apex>` HTTPS listener per child apex referencing the
same cert. Without this expansion the parent's single-level wildcard
cannot match a child route's hostname (`harbor.alice.example.org` is two
labels past the parent's `*.example.org`).
Opt-out into own Gateway is the right choice when: a tenant needs its
own LB IP (DNS pinned, firewall rule), the apex is not derived from the
parent (custom `host` value the ancestor's cert can't cover), or the
tenant wants its own ACME account / cert authority. Otherwise leave
`gateway` unset and inherit.
## End-to-end flow
1. Platform: `gateway.enabled=true` toggles cert-manager solver,
switches per-system-app templates from Ingress to HTTPRoute / TLSRoute.
2. Per tenant: explicit `tenant.spec.gateway=true` renders a
`TenantGateway` CR and triggers the controller to materialise the owner
Gateway. Tenants with the field unset inherit through
`_namespace.gateway` propagation in the tenant chart.
3. Adding a published app under any tenant (owner or inheriting) is just
deploying its HTTPRoute. The controller picks up the hostname, adds a
per-listener Certificate (HTTP-01) or extends the wildcard cert's SANs
(DNS-01), and updates the owner Gateway's listeners.
Defaults stay legacy: `gateway.enabled=false`,
`tenant.spec.gateway=false`. Existing clusters upgrade unchanged.
## External IP allocation — mechanism-agnostic
The per-tenant Gateway's auto-created `LoadBalancer` Service draws its
IP from whatever LB allocator the cluster admin has configured at the
platform layer — same shape as ingress-nginx today. The tenant API stays
mechanism-agnostic — no `gatewayIP` field, no allocator-specific
manifest in the tenant chart. Cozystack itself ships MetalLB installed
but does not render any `IPAddressPool` / `L2Advertisement` /
`BGPAdvertisement` from this chart; admins set up the allocator that
suits their environment (MetalLB pool with L2 / BGP, Cilium LB-IPAM with
announcer, robotlb against a cloud provider, or
`Service.spec.externalIPs` pinning).
If a tenant needs a specific address (DNS already pinned, firewall rule,
etc.), the operator pre-allocates it on the admin side: either
pre-create the Service with `loadBalancerIP` set, or hand the tenant a
reference to a named admin-managed pool. Per-Service IP uniqueness is
the allocator's responsibility — same as for any other LoadBalancer
Service.
## Cert mode: HTTP-01 (default) vs DNS-01 (opt-in)
`publishing.certificates.solver` selects how the controller sources TLS
certs.
**HTTP-01 (default)** — per-app HTTPS listener + per-app Certificate.
New apps require zero platform-side config: deploy the HTTPRoute,
controller does the rest. Inheriting children's hostnames pick up
per-listener certs the same way the owner's own apps do.
**DNS-01 (opt-in)** — single wildcard Certificate (`<apex>` +
`*.<apex>`) covering every published app under the apex, plus per-child
SANs (`<child-apex>` + `*.<child-apex>`) for every inheriting tenant.
Pick this for clusters where many apps share the apex and Let's Encrypt
rate limits matter. The DNS provider account must be able to write TXT
records under every apex level the parent serves — for deeply-nested
inherited children that requires zone delegation or a wide-scope
provider credential.
| `publishing.certificates.dns01.provider` | Required
`publishing.certificates.dns01.<provider>` keys |
| --- | --- |
| `cloudflare` (default) | `cloudflare.secretName`,
`cloudflare.secretKey` |
| `route53` | `route53.region`, `route53.secretName` (and `accessKeyID`
if not on IRSA) |
| `digitalocean` | `digitalocean.secretName` |
| `rfc2136` | `rfc2136.nameserver`, `rfc2136.tsigKeyName`,
`rfc2136.secretName` |
The platform chart wires every provider's keys into `_cluster.dns01-*`;
the per-tenant gateway chart and the cluster-wide `cluster-issuers.yaml`
both read them.
## Strengthens admission coverage on apps.cozystack.io/*
The `cozystack-tenant-host-policy` probe in `gateway.bats` initially
looked like a flake — the apply that was supposed to be VAP-rejected
just succeeded. Investigation traced it to the custom REST handler at
`pkg/registry/apps/application/rest.go`. genericapiserver hands every
storage method a callback into the validating admission chain:
`createValidation` for Create, `updateValidation` for Update,
`deleteValidation` for Delete. `Update` was correctly invoking its
callback since
|
||
|
Aleksei Sviridkin
|
c85e473523 |
test(e2e): inheritance attach asserts controller writes, not Cilium Accepted
The previous e2e check waited for `HTTPRoute.status.parents[0].conditions[Accepted]=True` after applying a child tenant's HTTPRoute against the parent's Gateway. That condition is set by Cilium after the listener's TLS Secret becomes Ready — and in the e2e cluster every fresh certificate fails to issue (LE prod refuses every `.example.org` hostname by policy, visible across alerta/grafana/dashboard/seaweedfs-s3 cluster certs in cozyreport). So Cilium never set Accepted, the route timed out 3/3 retries, and the failure was environmental — not a regression in the inheritance code path. The contract the test exists to pin is cozystack-controller's behaviour, which has two observable writes: 1. The parent Gateway grows a per-listener HTTPS entry for the child route's hostname (proves collectHostnameClaims saw the route through the inheritance label and reconcileGateway appended the listener). 2. A per-listener Certificate object appears in the parent namespace with the route's hostname in spec.dnsNames (proves reconcilePerListenerCertificates ran for the same hostname). Both writes are deterministic and depend only on the inheritance label + the HTTPRoute existing — not on ACME, not on Cilium TLS readiness. The test now polls those two objects directly within shorter, scenario-appropriate timeouts (120s for the listener, 60s for the Certificate). The route Accepted check is dropped with an explanation in the test comment so a future cluster with a working ACME doesn't get a misleading pre-condition added back. The namespace inheritance label is also verified up front as the precondition collectHostnameClaims's filter reads. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4120f8795e |
test(e2e): cover child-tenant inheritance attach flow
Replaces the now-obsolete "Package admission rejects tenant-* in gateway.attachedNamespaces" e2e (the VAP it pinned was dropped in the inheritance refactor) and adds two new e2e cases that cover the end-to-end inheritance contract: * "Package admission accepts gateway.attachedNamespaces with tenant-* entries" — regression guard against a future refactor re-introducing the gate; the hijack surface is now handled by Layers 4/5/7 (Tenant.spec.host VAP, namespace label immutability VAP, HTTPRoute hostname VAP), not by entries in attachedNamespaces. * "child tenant without explicit gateway inherits _namespace.gateway from a Gateway-owning parent" — chart-level lockstep: a parent tenant with gateway=true produces both the namespace label and the cozystack-values Secret pointing at itself, and a child Tenant under that parent with the gateway field unset receives the parent's name in both places. Plus a negative assertion that the child does NOT get its own gateway HelmRelease. * "child tenant's HTTPRoute attaches to parent's Gateway via inheritance label" — full cross-namespace attach: parent owns the Gateway, child inherits, an HTTPRoute in the child namespace pointing at parent's Gateway reaches Accepted=True once Cilium evaluates the label-based allowedRoutes selector. Route hostname uses the child's derived apex so Layer 7 admits it. The Gateway-Programmed wait is intentionally omitted from the last test — Programmed depends on the LB allocator that ships with the e2e cluster, while Accepted is a pure spec evaluation on the parent Gateway's allowedRoutes selector and is the property the inheritance refactor is meant to deliver. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d2e851f63b |
feat: Gateway API support via Cilium
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.
The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.
publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.
Security model -- seven independent layers, all admission-time gates
fail-closed:
1. Listener allowedRoutes namespace whitelist via the
kube-apiserver-written kubernetes.io/metadata.name label;
plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
passthrough listeners restrict allowedRoutes.kinds to
HTTPRoute / TLSRoute respectively.
2. cozystack-gateway-hostname-policy VAP on Gateway.
3. cozystack-gateway-attached-namespaces-policy VAP on Package --
blocks tenant-* in gateway.attachedNamespaces.
4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
writes from non-trusted callers.
5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
writes to namespace.cozystack.io/host from non-trusted callers.
6. Render-time fail in cozystack-basics for tenant-* in
gateway.attachedNamespaces.
7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
scoped to tenant-* namespaces.
Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).
Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.
Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.
Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.
Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.
Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.
Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Myasnikov DaniilandClaude Opus 4.7
|
e42a5828fe |
refactor(build): skip duplicate versioned tag when it matches IMAGE_TAG
On release builds IMAGE_TAG = github.ref_name = vX.Y.Z, and Makefiles that pass v$(COZYSTACK_VERSION) as the versioned arg evaluate to the same vX.Y.Z on the tag commit. The macro then emitted `--tag :vX.Y.Z --tag :vX.Y.Z --tag :latest`. docker buildx dedupes internally so there is no functional impact, but the build logs carried the redundant flag for every affected package. Replace the empty-string guard with filter-out so the versioned tag is also skipped when it would duplicate IMAGE_TAG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
000d5165ca |
test(e2e): pre-pull platform images via staged-busybox DaemonSet
Cluster-member workloads (OVN raft, LINSTOR, cert-manager) fail when replicas start at different times due to per-node image-pull stagger. This test pre-pulls the images that those charts reference onto every node before the platform HelmReleases install, so all replicas can start with images already cached. Mechanics: - Source images directly from the rendered charts (kubeovn, linstor, cert-manager) via `helm template` + a `yq` filter that walks every PodSpec-shaped object and emits each container's image. Scopes the output to images the kubelet actually pulls — skips configmap fields and CRD examples that happen to contain an `image:` key. Each render is staged through a tmp file so a helm-template failure trips `set -e` cleanly without needing `pipefail` (cozytest.sh runs @test bodies under /bin/sh, which is dash on Ubuntu CI), and without capturing rendered YAML into a shell variable that `set -x` would expand and balloon the CI trace. - The pre-pull DaemonSet is the hard part: distroless images (cert-manager and friends) ship no shell and no /bin/sleep, so the obvious `command: ["sleep", "infinity"]` fails with exec ENOENT. Stage a statically-linked busybox:musl into a shared emptyDir from an initContainer; every prepull container then execs /shared/sleep regardless of what the image itself ships. The glibc busybox tag is unusable here — it needs /lib64/ld-linux-x86-64.so.2 which distroless lacks. Loop is written in POSIX shell (no bash arrays) for the same /bin/sh-is-dash reason. Verified locally on kind v1.33.1 single-node: every chart-referenced image is held Ready under the DaemonSet before the platform HelmRelease applies. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
ed0297eb87 |
refactor(build): standardize image tagging to fix concurrent PR push conflicts
Every package Makefile tagged images via `settag` which, on a non-release commit, collapsed to `:latest`. Concurrent PR builds therefore raced on every image's `:latest` manifest in the OCIR registry. Two outliers (metallb, redis-operator) used hardcoded version tags and collided across PRs even when `settag` would not have. Replace `settag` and the `TAG=latest` fallback with an `image-tags` macro driven by three env vars: - IMAGE_TAG — build-unique tag, always pushed - PUBLISH_VERSIONED — also push :<component-version> - PUBLISH_FLOATING — also push :latest PR CI sets IMAGE_TAG=pr-<N>-<sha> with both PUBLISH_* off, so every push is unique across runs. Release CI sets IMAGE_TAG=<git-tag> with both PUBLISH_* on, preserving the previous release output and adding a uniform :<git-tag> tag across every image. Local builds default to IMAGE_TAG=dev with PUSH=0 already off. Image consumers reference digests pinned into values.yaml by each Makefile, so the tag is decorative for cluster-side pulls; the build-unique tag is sufficient for E2E. Drops the redundant `:<component-version>-<git-tag>` form (cilium, linstor, matchbox, ubuntu-container-disk, kubevirt-*, cluster-autoscaler) — nothing in the repo or downstream values.yaml consumed it. The same information is preserved via the new uniform `:<git-tag>` plus the existing `:<component-version>` on release. ubuntu-container-disk builds a separate image per Kubernetes version in a loop. Switched its primary tag to `:<k8s-ver>-<IMAGE_TAG>` so loop iterations within a single build do not push to the same tag. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
e3a737a1d3 |
test(e2e): silence helm-template render in install-cozystack trace (#2615)
## What this PR does Trims the `Install Cozystack into sandbox` CI step from ~40k log lines back down to a normal size. The `Pre-pull platform images` bats test was capturing each rendered chart into a shell variable so a render failure would trip `set -e`: ```sh kubeovn_manifests=$(helm template packages/system/kubeovn) linstor_manifests=$(helm template packages/system/linstor) printf '%s\n%s\n' "$kubeovn_manifests" "$linstor_manifests" | yq ... | hack/e2e-prepull-images.sh ``` `hack/cozytest.sh` runs every bats test under `set -x`, and bash expands captured command output into the trace as `+ var=<entire stdout>`. The kubeovn and linstor renders are several thousand lines each, and the `printf` line re-expands both variables, so the same YAML was being dumped to the log three times. This PR stages each render to a tmp file instead: ```sh helm template packages/system/kubeovn > "$kubeovn_yaml" helm template packages/system/linstor > "$linstor_yaml" cat "$kubeovn_yaml" "$linstor_yaml" | yq ... | hack/e2e-prepull-images.sh ``` The original safety property is preserved — `helm template ... > file` is a simple command whose exit code still triggers `set -e` on failure, exactly like `var=$(helm template ...)` did. The trace now shows the command line, not its rendered output. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved image pre-pull test to render charts into temporary files, consolidate results, and feed them deterministically into the image pre-pull check. * Made failure handling more reliable so rendering or parsing errors now cause the test to fail predictably. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2615?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
9c87996da8 |
fix(kafka): remove ZooKeeper PVCs on uninstall (#2679)
## What this PR does Aligns ZooKeeper storage cleanup with the existing Kafka broker behavior so deleting a Kafka release reclaims all of its persistent storage. The Strimzi `Kafka` CR template already set `deleteClaim: true` on the broker JBOD volume, but the ZooKeeper persistent-claim was left at `deleteClaim: false`. As a result, removing a Kafka application freed the broker PVCs while orphaning the ZooKeeper PVCs in the tenant namespace, requiring manual cleanup before the same release name could be reused. The change is a single field flip in `packages/apps/kafka/templates/kafka.yaml` on the `spec.zookeeper.storage` block. ### Behavior change Users who previously relied on orphaned ZooKeeper PVCs as an implicit "soft delete" for ZooKeeper state recovery should be aware that uninstalling a Kafka release now also removes the underlying ZooKeeper storage. Take a backup before deletion if ZooKeeper data retention is required. ### Screenshots Not applicable — no UI changes. ### Release note ```release-note fix(kafka): ZooKeeper PVCs are now reclaimed when a Kafka release is deleted, matching the existing Kafka broker storage behavior. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Kafka storage behavior updated so persistent volumes are automatically removed when claims are deleted, changing storage lifecycle for deployments. * **Tests** * End-to-end cleanup adjusted to remove the Kafka resource and verify volumes are reclaimed automatically, adding waits to confirm PVC binding and removal. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2679?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei KvapilandClaude
|
4d5486cd84 |
chore(controller): remove dead dashboard controller
The dashboard controller used to project ApplicationDefinitions into a set of dashboard.cozystack.io CRDs (Sidebar, Factory, Navigation, Breadcrumb, CustomFormsOverride, CustomFormsPrefill, CustomColumnsOverride, MarketplacePanel, TableUriMapping, CFOMapping, BreadcrumbInside) consumed by the legacy UI. cozystack-ui now reads ApplicationDefinitions directly, so the controller, its API types, the generated CRDs and the related RBAC rules are no longer used. Removes: - internal/controller/dashboard - api/dashboard/v1alpha1 - packages/system/cozystack-controller/definitions/dashboard.cozystack.io_*.yaml - dashboard.cozystack.io wiring from cmd/cozystack-controller/main.go - dashboard.cozystack.io RBAC rules from cozystack-controller and cozy-dashboard charts - ./api/dashboard/... codegen path from hack/update-codegen.sh Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
39d46f2d13 |
test(e2e): stage yq output to a file instead of piping
The prior commit added `set -o pipefail` to catch yq failures in the
prepull pipeline. CI failed at runtime: hack/cozytest.sh has a
`#!/bin/sh` shebang, /bin/sh is dash on Ubuntu CI, and dash rejects
`set -o pipefail` ("Illegal option").
Drop the pipe entirely. Each step is now its own command with stdin/
stdout redirection — set -e catches failures at the helm-template, yq,
or prepull stage without needing pipefail and without `var=$(...)`
captures that would balloon the `set -x` trace with rendered YAML.
Verified under dash in Ubuntu 24.04 by running cozytest.sh against the
test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov DaniilandClaude Opus 4.7
|
5ddcb0ded2 |
test(e2e): enable pipefail in prepull pipeline
The previous form staged each helm-template render to a tmp file so a render failure would trip `set -e` outside the pipe. That avoided the `var=$(...)` trace-bloat trap, but yq still ran inside a pipe with pipefail off, so a yq failure (bad expression, schema drift) would be masked by the prepull script's exit code and the test would pass while silently pre-pulling nothing. Replace the tmp-file dance with a brace group + `set -o pipefail`: helm-template output flows through the pipe, never expanded into the `set -x` trace (verified under `hack/cozytest.sh`: 5 command lines, no YAML), and pipefail now surfaces a failure at any stage. Drops the mktemp/rm scaffolding and the `cat` subprocess. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Arsolitt
|
2086f3e6b6 |
test(kafka): assert ZooKeeper PVC-1 is created before deletion check
Address review feedback from coderabbitai on hack/e2e-apps/kafka.bats:58: The post-delete `while kubectl get pvc data-kafka-$name-zookeeper-1` loop exits immediately when the PVC has never been created, so a regression that prevents Strimzi from provisioning the second ZooKeeper PVC would silently pass the reclamation check. Mirror the PVC-0 readiness guard (existence + Bound) for PVC-1 before deleting the Kafka CR so the test only proceeds once both PVCs are observed. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Arsolitt <arsolitt@gmail.com> |
||
|
Arsolitt
|
c4b0606e67 |
test(kafka): verify ZooKeeper PVCs are reclaimed by the operator
Address review feedback from gemini-code-assist on hack/e2e-apps/kafka.bats:53-55: Replace the manual kubectl delete of the two ZooKeeper PVCs with a poll that waits for the Strimzi operator to reclaim them. With deleteClaim: true on the ZooKeeper persistent-claim storage, the operator removes the PVCs as part of Kafka CR deletion; manual deletion would either race the operator, fail with NotFound, or silently mask a regression in the reclamation path. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Arsolitt <arsolitt@gmail.com> |
