mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 14:02:56 +09:00
scooby87
fdbe6b299c
## What this PR does After deleting a `ClickHouse` app CR, data PVCs were left orphaned in the tenant namespace and consumed storage permanently: - the keeper PVC `default-chk-<release>-keeper-cluster1-0-0-0`, backing a `ClickHouseKeeperInstallation` reconciled by the clickhouse-keeper-operator; - the main ClickHouse data / log PVCs. The keeper operator does not delete data PVCs when the CR is removed (data-safety default) and the CRD exposes no `persistentVolumeClaimRetentionPolicy` equivalent, so the volumes leaked. This adds a `post-delete` cleanup hook (modeled on the existing MariaDB hook at `packages/apps/mariadb/templates/hooks/cleanup-pvc.yaml`) that deletes every PVC carrying the release label `app.kubernetes.io/instance=<release>`. It also adds that label to the keeper `volumeClaimTemplates` in `chkeeper.yaml`: the keeper PVC previously had **no** release label (the chart set labels only on the keeper pod template, not the PVC), so the operator never stamped one and a single selector would have missed it — leaving the reported leak unfixed. ### Existing releases (label backfill migration) The altinity clickhouse-keeper-operator only stamps `volumeClaimTemplates.metadata.labels` onto PVCs it creates **after** the chart change; it never re-labels PVCs that already exist. So for releases installed before this fix, the keeper PVC stays unlabeled and the cleanup selector would still miss it. To close #3057 for existing customers too, this PR adds migration `48` (`48 -> 49`, `packages/core/platform/images/migrations/migrations/48`) that back-fills `app.kubernetes.io/instance=<release>` onto pre-existing keeper PVCs, and bumps `migrations.targetVersion` to `49`. The migration: - routes its version stamp through the shared `lib/cozystack-version.sh` helper (`stamp_cozystack_version 49`), per the `cozystack-version-stamp.bats` convention; - no-ops when the keeper CRD is absent, and tolerates a transient apiserver error on the keeper list (`|| true`) so it cannot abort a platform-wide upgrade — the backfill is best-effort (worst case is the pre-existing leak, no worse than today); - is idempotent (skips PVCs already labeled). The main ClickHouse data/log PVCs have carried `app.kubernetes.io/instance` since 2025-06-03 (well before the migration era), so they need no backfill. ### Hook failure handling The cleanup hook does **not** mask failures: `kubectl delete` failing makes the Job exit non-zero. The **Job** uses `helm.sh/hook-delete-policy: before-hook-creation` only (no `hook-succeeded`) so a failed cleanup Job and its logs survive for debugging; the **ServiceAccount/Role/RoleBinding** keep `before-hook-creation,hook-succeeded` so the RBAC artifacts are reclaimed after a successful uninstall instead of lingering in the namespace. ### Scope The hook intentionally reclaims **only PVCs** of the release. Other release state (Secrets, ConfigMaps) is owned/garbage-collected through the normal Helm uninstall path and is out of scope here. > [!WARNING] > **This is destructive — deleting a ClickHouse app now permanently deletes its data.** > The hook removes **all** PVCs of the release: the keeper PVC **and** the main > ClickHouse data and log PVCs. After deletion nothing is left behind to recover > data from. This is intentional and matches the existing MariaDB cleanup-hook > behaviour and the general cozystack model (deleting a managed app reclaims its > storage). Back up before deleting. Fixes #3057. ### Release note ```release-note fix(clickhouse): garbage-collect the keeper and data/log PVCs when a ClickHouse app is deleted, so no orphaned volumes are left behind. A migration back-fills the release label onto keeper PVCs from releases installed before this fix so they are cleaned up too. WARNING: deleting a ClickHouse app now permanently removes all of its PVCs (keeper + data + logs) and the data they hold. ```
How to test packages local
cd packages/core/installer
make image-cozystack REGISTRY=YOUR_CUSTOM_REGISTRY
make apply
kubectl delete po -l app=source-controller -n cozy-fluxcd