mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-30 16:47:20 +09:00
Andrei Kvapil
93aac35121
## What this PR does Implements the tenant Flux sharding design from #2817: a new `flux-shard-operator` system package that spreads tenant HelmReleases across multiple helm-controller shards, so a noisy tenant (e.g. a HelmRelease stuck in infinite remediation) cannot degrade the others. - **Placement controller** owns the tenant→shard assignment: the unit of placement is the tenant (all of its HelmReleases share one shard label), distribution is greedy least-loaded weighted by HelmRelease count, with pinned tenants, threshold-gated rebalance, paced moves, and never moving deleting tenants. Watches are metadata-only, so the controller does not decode the helm-controller status-patch firehose. Assignments are recorded on tenant namespaces; HelmRelease labels remain the source of truth on restarts. - **Shard runtime** is provisioned by the operator: `shardCount` helm-controller Deployments cloned from the flux-aio `flux` Deployment and sanitised, so the image and feature-gates stay version-synced automatically. flux-aio itself is unchanged and keeps owning everything without a shard key. - **CREATE-only mutating webhook** (`failurePolicy: Ignore`) stamps the tenant's shard onto every HelmRelease at admission, so each one is born on the correct shard regardless of creation path; any miss degrades gracefully to the placement controller. - The hand-rolled `flux-tenants` Deployment is retired: the operator drains it (backfill `tenants` → `shard<i>`) and deletes it once empty; migration 44 removes a leftover only under the same guard. Ships with `shardCount: auto` (the default): the operator sizes the shard count from the tenant HelmRelease count — `K = clamp(ceil(H/100), 1, min(16, T))` with hysteresis (scale up past 120 HR/shard, down under 60) — so small clusters stay at one shard (≈ today's behaviour) and large fleets shard out automatically. An integer value pins the count explicitly. One deviation from the design doc: the webhook/controller scope is the `sharding.fluxcd.io/key` label rather than `internal.cozystack.io/tenantmodule` — every tenant HelmRelease is born with `key=tenants` (stamped by ApplicationDefinition `release.labels` and chart templates), so the webhook rewrites it and the placement controller continuously relabels stragglers; `tenantmodule` covers only the 6 tenant chart modules. ### Screenshots Not a UI change. ### Release note ```release-note feat(fluxcd): tenant HelmReleases are now reconciled by operator-managed helm-controller shards; the flux-shard-operator balances tenants across `shardCount` instances and retires the legacy flux-tenants deployment ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced **flux-shard-operator** to shard helm-controller workloads with deterministic tenant placement, optional rebalance controls, and autosized shard count. * Added a **CREATE-only mutating webhook** to stamp the correct shard label on tenant HelmReleases. * **Bug Fixes** * Application updates now preserve existing **flux-shard** label assignments to prevent unwanted shard re-stamping. * **Chores** * Added platform packaging for the new component, including an image build step. * Removed the legacy tenants deployment and added a migration to retire it safely. * **Tests** * Expanded unit and chart tests for placement, provisioning, webhook behavior, and configuration parsing. * **Documentation** * Added operator documentation describing configuration and metrics. <!-- end of auto-generated comment: release notes by coderabbit.ai -->