mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-03 18:47:04 +09:00
Aleksei Sviridkin
bdf23e66d1
## What this PR does Closes #2412. On a cold tenant-Kubernetes bootstrap, the parent HelmRelease raced the admin-kubeconfig Secret that Kamaji provisions asynchronously. Three CP-side Deployments (cluster-autoscaler, kccm, kcsi-controller) mounted that Secret as a hard volume, flux helm-controller's default wait budget was too short for Kamaji cold start, and `install.remediation { retries: -1 }` then uninstalled the Cluster CR and restarted the cycle forever. Implements a defense-in-depth fix: - `optional: true` on the admin-kubeconfig Secret volume in all three Deployments so kubelet no longer FailedMounts while Kamaji is still bootstrapping. - A shared `wait-for-kubeconfig` init container (in `templates/_helpers.tpl`) that polls for `super-admin.svc` with a 10m deadline, strictly below the HelmRelease Install.Timeout so a broken tenant falls into CrashLoopBackOff visibly instead of hanging forever. - Per-Application HelmRelease Install/Upgrade timeout, driven by a new `release.cozystack.io/helm-install-timeout` annotation on ApplicationDefinition. Kubernetes-rd sets it to `15m`; other kinds leave it unset and keep flux defaults, so their failed installs remediate on the normal cadence. Parser rejects ns/us/µs (accepted by `time.ParseDuration`, rejected by Flux's CRD pattern) at startup. - Soft-skip when `_namespace.etcd` is empty: the CP-side Deployments, the Cluster/KamajiControlPlane/KubevirtCluster/WorkloadMonitor CRs, and every child HelmRelease that references admin-kubeconfig now render only when an etcd DataStore exists for this tenant. An `awaiting-etcd` ConfigMap is emitted as a user-visible status beacon so `helm install` still succeeds and flux retries on its 5m interval until the Tenant chart catches up. - e2e remediation guard built on `.status.history[].status` (the Snapshot shape), not on `.status.installFailures` - `ClearFailures()` zeroes the latter on every successful reconciliation, which made the previous guard vacuous. Tests: - Go unit tests for the annotation parser (accepted/rejected units) and the HR builder (table-driven across kinds). - helm unittest for the per-template structure (optional volume, init container, dataStoreName, awaiting-etcd beacon). - bats unit tests for the shell guard (every combination of empty/zero/positive history entries, plus pinned HR v2 shape). - Chart-wide bats invariants: every Deployment mounting admin-kubeconfig has the guards; zero such Deployments and zero HelmReleases render when etcd is empty. All wired into the existing `make unit-tests` target (`go-unit-tests` added alongside `helm-unit-tests` and `bats-unit-tests`). Option 2 from the ticket (separate HelmRelease with `dependsOn`) was intentionally not taken: the combination above closes the same race without restructuring the chart's HelmRelease topology. ### Release note ```release-note fix(kubernetes): close admin-kubeconfig race on tenant Kubernetes bootstrap. The parent HelmRelease no longer enters an uninstall/retry cycle when Kamaji control-plane cold start exceeds flux's default wait budget. A Kubernetes tenant created before the parent Tenant application has etcd enabled now renders only an awaiting-etcd beacon ConfigMap and waits quietly for the DataStore to appear, instead of producing half-installed Deployments that CrashLoopBackOff forever. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Per-application Helm install/upgrade timeout via metadata annotation. * Init-container guards that wait for admin kubeconfig before workloads start. * Chart resources now render conditionally based on etcd presence. * **Tests** * Helm-template tests for admin-kubeconfig invariants and remediation-cycle detection. * New Go unit tests and CI Helm/unittest coverage plus test value files. * **Chores** * Added BusyBox image pin and new Makefile test targets (including Go unit-tests). <!-- end of auto-generated comment: release notes by coderabbit.ai -->