mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-06 03:57:52 +09:00
feat/upgrade-e2e-chainsaw
194
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Andrei Kvapil
|
7cca3d75e7 |
feat(kubernetes)!: remove the FluxCD tenant addon
The optional `addons.fluxcd` toggle of the Kubernetes app deployed a Flux Operator + FluxInstance into a tenant cluster. It was served by two vendored charts, packages/system/fluxcd-operator and packages/system/ fluxcd, which were the last AGPL-3.0 content in this Apache-2.0 repository and a governance blocker for CNCF Incubation. The management cluster does not use these charts (it runs classic Flux via internal/fluxinstall; migration 21 already removed the operator from it), and the addon defaulted to disabled. Rather than maintain an AGPL payload for an unused feature, drop it entirely: - delete both system packages and their platform source entries - remove the addons.fluxcd HelmReleases, values and generated schema from the Kubernetes app - drop the addons.fluxcd block still submitted by the e2e harness - drop the removed fluxcd package entries from CODEOWNERS BREAKING CHANGE: the addons.fluxcd option of the Kubernetes app is removed. Tenant clusters that had it enabled keep their running Flux release; see the companion migration for the upgrade path. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
3710541278 |
fix(kubernetes): validate MachineHealthCheck condition timeouts at render time
The CAPI webhook validates nodeStartupTimeout with a 30s floor but does
not validate unhealthyConditions timeouts at all, so a zero or negative
readyUnknownTimeout or readyFalseTimeout applied and remediated a
Machine the moment its Ready condition flipped. Route both fields
through the same render-time duration guard as maxNodeProvisionTime,
extracted into a kubernetes.positiveDuration helper in each chart.
The guard now also accepts the canonical form a duration serializes
back to ("20m0s", "1h0m0s"): metav1.Duration prints that shape, so a
value copied from kubectl output previously failed the render. The
first segment must still be positive and later segments may be zero,
which keeps every accepted string a positive duration.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
|
||
|
Aleksei Sviridkin
|
ddb15ebc6a |
fix(kubernetes): widen the worker node startup budget to 20m
nodeStartupTimeout is the window a Machine has to produce a Node at all, measured from the later of the Machine's creation and its InfrastructureReady transition. What has to fit inside it is the guest boot and the kubelet registering with the tenant apiserver, and on a host under load neither is quick. At 10m a worker that was still booting was deleted before it ever joined, and its replacement started the same race from the beginning. Also record what a percentage maxUnhealthy does at small group sizes, because the interaction is not what the field name suggests: the value is scaled against the machines the check covers and rounded down, and remediation proceeds only while the unhealthy count is at or below the result. At the default "50%" a group of one never remediates, and a group of two remediates one unhealthy machine but not two. The golden-parity gate pinned the child chart's MachineHealthCheck values to the parent's defaults as inputs, which made a default moved on both sides fail there for the wrong reason. Drop the pins so the comparison is between the two charts' defaults, which is the contract it exists to hold, and pin the defaults themselves in each chart's own unit tests instead. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
33701800d6 |
fix(kubernetes): make the worker MachineHealthCheck timeouts configurable
The two unhealthyConditions timeouts were fixed in the template at 30s for `Ready=Unknown` and 300s for `Ready=False`, and both are too tight for a worker joining on a loaded host. `Ready=Unknown` is what the node lifecycle controller writes when a kubelet stops reporting, on a grace period of its own, so 30s on top of that deleted the VM of a guest that had merely stopped being scheduled for a stretch, before it had a chance to answer again. And `Ready=False` is the condition a registered Node holds until its CNI is up, so 300s was the entire budget a first boot got for the CNI rollout to reach it -- a worker deleted mid-rollout takes the rollout with it and its replacement starts over. Move both to values, defaulting to 5m and 10m. The remediation of a worker that is genuinely dead is correspondingly later, which costs less than it appears to: Pods leave an unreachable Node on the node lifecycle controller's taint-based eviction and the toleration that carries, not on either of these timers. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
83f0eef645 |
fix(kubernetes): give the autoscaler the real worker join time
The cluster-autoscaler treats a machine as long-unregistered once its own max-node-provision-time elapses, and a worker that has not registered by then stops counting towards its node group's upcoming capacity. On a host slow enough that the Talos image import, the guest boot and the CNI rollout do not fit inside that window, the timer fires on machines that were still joining: the group reads as short of capacity, so the autoscaler asks for replacements, and the extra provisioning load makes the join it was waiting for slower still. Render the per-node-group override the autoscaler reads off the MachineDeployment, sourced from a new maxNodeProvisionTime value that defaults to 30m. The duration is checked at render time because the autoscaler discards one it cannot parse and falls back to its own default without failing, and because the values schema types the field as a string and so accepts every malformed duration. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
3cb131785c |
fix(kubernetes): settle what a pod CPU quantity of zero or less means
Both podCpuLimit and podCpuRequest were rendered under `with`, which skips a zero as readily as an absent field, and their guards were keyed the same way. The generated schema behind them admits a leading minus and a zero as readily as a sane value, so rather than patch whichever edge turned up, the whole value domain is settled in one place. Presence, not truth, is the predicate everywhere either field is read. A negative value is refused: it never reaches a running Pod. A zero request on its own is allowed and reserves nothing. A zero request beside a non-zero limit is refused: KubeVirt's defaulter tests the request with IsZero, which an explicit zero satisfies exactly as an absent key does, so it overwrites the zero with the limit and the worker asks the scheduler for its whole ceiling. That is the outcome podCpuRequest exists to prevent, reached silently and reported nowhere, which is why it is refused rather than written down. A zero limit removes the ceiling rather than being the tightest one -- the kubelet turns a zero CPU limit into no CFS quota -- but it reaches a Pod only paired with a zero request. KubeVirt keeps deriving a request from the vCPU count when the VM declares none, the defaulter that would have copied the limit across does not fire because the limit is itself zero, and API server validation refuses a request above a limit that exists. Zero is a limit that exists. So a lone zero limit is refused too. Both fields document the same rules and each class has a test on both charts. A group that sets neither field renders the bytes it rendered before, so the content-hash name of the worker VM template does not move. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
1b8340298e |
feat(kubernetes): let a node group set the CPU request of its worker VMs
Adds nodeGroups.<name>.podCpuRequest, rendering into domain.resources.requests.cpu on the worker VM, and the same field on the kubernetes-nodes chart, which is the other half of a byte-identity contract rather than a separate feature surface. podCpuLimit on its own cannot raise the ceiling of a worker without raising what that worker asks the scheduler for, because KubeVirt copies a declared CPU limit into the request when the VM declares none. The request is the half that has to hold still, so the pair can now be moved apart. Both fields conflict with instancetype sizing for the same reason and in the same upstream check, so the request gets the guard the limit already had. A request above its own limit fails the render too, rather than reaching API server validation on a Pod two objects away from the CR that named the values; the quantities are compared as numbers, so 500m under a ceiling of 1 is accepted where a lexical guard would refuse it. A node group that sets neither field renders the bytes it rendered before, which the content-hash name of the KubevirtMachineTemplate depends on, and the render-parity gate gains a case that compares the pair across the two charts. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
669c3e60ab |
docs(kubernetes): correct what podCpuLimit does to the CPU request
The field was documented as moving the ceiling alone and leaving the CPU request at the value KubeVirt derives from the guest vCPU count and the cluster CPU allocation ratio. KubeVirt does the opposite: its VMI defaulter copies a declared CPU limit into the CPU request whenever the VM declares no request of its own, so a node group that sets only the limit asks the scheduler for its whole ceiling, and a ceiling the nodes cannot spare leaves the workers Pending on Insufficient cpu. The same paragraph promised a floor that does not exist either. A limit below the derived request was said to leave the container with a request above its limit; the copy makes the two equal instead, whatever the limit is. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
78b9bc3e6e |
feat(kubernetes-nodes): mirror the worker pod CPU limit into the pool chart
The split-out pool chart renders the same worker VM as the monolithic one, and tests/render-parity.sh exists to prove it renders it byte-identically, because the KubevirtMachineTemplate is content-hash named and any divergence rolls every live worker VM. A field that only one of the two charts knows breaks that quietly rather than loudly: a pool adopted into a kubernetes-nodes release would render without the CPU limit it was given, the hash would change, and the ceiling would fall back to the vCPU count with nothing reporting it. So the field, its render-time guard against combining it with an instancetype, and its tests arrive here in the shape the parent chart has them, and the parity gate gains a case that compares the two renders with the limit set. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
a666b9b3bd |
feat(kubernetes): let a node group set the CPU limit of its worker VMs
The chart had no way to give a worker VM a CPU ceiling different from its vCPU count. `resources.cpu` sizes the guest, and the pod side is filled in by KubeVirt: with a ResourceQuota limiting CPU in the namespace, which every tenant carrying resourceQuotas has, it derives limits.cpu from the guest vCPU count. The QEMU emulator thread and the IO threads then come out of the same allowance as the vCPU threads, which is tight for a nested guest that pays emulation overhead. Declaring the limit on the VM is what stops KubeVirt deriving one, and it is the only way out that does not make things worse: dropping cpu from the tenant resourceQuotas would let the LimitRange apply its 250m default instead, four times tighter than what it replaced. The limit is usable only on a group sized by resources, because KubeVirt's instancetype applier reports domain.resources.limits.cpu as a conflict rather than resolving it, and a VM carrying both is refused by the admission webhook with nothing on the CR to say why. Sizing by resources is what drops the instancetype from the VM, so the chart fails the render for a group that kept its instancetype, beside the existing guard for the same conflict class one field over. The field renders no CPU request, so the request stays derived from the vCPU count and the cluster CPU allocation ratio, and the scheduler weight of a worker does not move with its ceiling. It renders nothing at all when unset, byte for byte: the enclosing KubevirtMachineTemplate is named by a hash of its content, so anything else there would roll every live worker VM on upgrade. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
dff56aa998 |
feat(backups): add a native MongoDB (psmdb) backup-strategy driver
Adds a MongoDB (Percona Server for MongoDB / psmdb) backup-strategy driver so `apps.cozystack.io/MongoDB` reaches feature parity with the CNPG, MariaDB and ClickHouse native drivers — logical backups, restore-to-differently-named instances, and point-in-time recovery through the unified BackupJob/RestoreJob/BackupClass/Plan interface (closes #3545). The driver orchestrates the operator's PerconaServerMongoDBBackup / PerconaServerMongoDBRestore CRDs: - BackupJob: creates a one-shot PerconaServerMongoDBBackup, polls status.state, and persists the S3 destination + storage descriptor (by-reference, no credentials) onto the Cozystack Backup artifact. - RestoreJob: drives PerconaServerMongoDBRestore via backupSource (read back from the source backup status, snapshot fallback) so the same artifact restores in-place or into a differently-named instance; PITR via spec.options.pitr (date/latest). Because psmdb resolves the backup storage by name against the cluster's spec.backup.storages, the driver names the chart's storage (s3-storage) via the cozy-default-mongodb Strategy rather than injecting it, and surfaces a clear Ready=False precondition when a target has backups disabled. Wiring: - api/backups/strategy/v1alpha1: MongoDB strategy type + generated CRD/deepcopy. - internal/backupcontroller: psmdbstrategy_controller + psmdbtypes/mongodbapp partial-schema mirrors; dispatch, supported-kinds and scheme registration. - backupstrategy-controller: cozy-default MongoDB route, cozy-default-mongodb Strategy CR, and psmdb.percona.com RBAC. - apps/mongodb: backup.insecureSkipTLSVerify knob (psmdb s3 storage has no CA-bundle field) so backups can target Cozystack's self-signed seaweedfs. Tests + e2e: - unit tests for the driver (validation, ensure-idempotency, precondition, backupSource resolution, snapshot round-trip, PITR option parsing). - examples/backups/mongodb + a mongodb-2-backup-roundtrip Chainsaw suite that drives run-all.sh end-to-end (sentinel document round-trips through S3 into a restored copy while the source stays untouched), per docs/agents/e2e-testing.md. - docs/operations/backup-classes.md updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
d0990299ed |
feat(clickhouse): let the backup sidecar trust a private S3 CA, unblocking the backup e2e round-trip (#3385)
## What this PR does Gives the ClickHouse `clickhouse-backup` sidecar a way to trust an S3 endpoint whose certificate is signed by a private CA, and uses that to make the ClickHouse backup/restore e2e round-trip actually run in CI. This actualizes #2600, whose feature code and example scripts have since landed on `main` independently, and whose remaining contribution — the e2e test — was still in the retired BATS format. #2600 can be closed once this merges. ### 1. `backup.endpointCA` on the ClickHouse chart The sidecar had **no TLS/CA surface at all**: bucket coordinates and credentials arrive as `S3_*` env, and nothing let it verify an endpoint signed by a private CA. Cozystack's own in-cluster SeaweedFS is exactly that case — it serves `:8333` behind the self-signed "SeaweedFS CA" — so a ClickHouse release could only ever back up to a publicly-trusted endpoint. MariaDB (`storage.s3.tls.caSecretKeyRef`) and Postgres (barman `endpointCA`) have had a CA field all along; this closes the gap for ClickHouse. `backup.endpointCA` takes `{name, key}`. When set, the chart mounts that Secret into the sidecar and points `SSL_CERT_DIR` at it. Empty — the default — renders no env, no mount and no volume, so existing releases are byte-for-byte unchanged. **Why `SSL_CERT_DIR` and not `AWS_CA_BUNDLE`.** Both are honoured by `altinity/clickhouse-backup:2.7.4`; I measured them against a self-signed endpoint and public AWS S3 in turn. Go reads `SSL_CERT_DIR` *in addition to* the system bundle, so both endpoints verify. `AWS_CA_BUNDLE` *replaces* the SDK's trust pool: the private endpoint verifies, and public S3 then fails `x509: certificate signed by unknown authority` — it would silently break a release that also talks to a public endpoint. `S3_DISABLE_CERT_VERIFICATION` was rejected outright, since it drops verification rather than trusting one more CA. ### 2. The e2e suite, and what it was really testing Two Tests added to the existing `hack/e2e-chainsaw/clickhouse/` suite (in name order, mirroring the `etcd`/`mariadb`/`postgres` shape — see the layout rationale below for why they live here rather than in a new directory): - **`clickhouse-1-backup-contracts`** (CI, `tenant-test`, ~98s): a backup-enabled ClickHouse asserted against the *running* cluster — the `clickhouse-backup` sidecar on the `ch-backup-api` `:7171` port, and the `<release>-backup-api-auth` Secret whose keys the sidecar's `API_USERNAME`/`API_PASSWORD` actually resolve `valueFrom`. Each is a silent break if a chart bump renames it. Needs no object store. - **`clickhouse-2-backup-roundtrip`** (CI, `tenant-root`): `Bucket` → `Altinity` strategy + `BackupClass` → source ClickHouse with a sentinel row → `BackupJob` → in-place `RestoreJob` → to-copy `RestoreJob` into a second instance, driving `examples/backups/clickhouse/run-all.sh` so test and docs cannot drift. The round-trip was originally gated behind `CLICKHOUSE_E2E_S3_ROUNDTRIP` and **skipped in CI** — its `PASS (1.01s)` meant the gate fired, not that a backup happened. `backup.endpointCA` removes the reason for the gate, so the suite now adopts the same three preconditions mariadb/postgres rely on: run in `tenant-root` (an isolated tenant's Cilium egress allowlist blocks its Pods from reaching `tenant-root`'s SeaweedFS), target the in-cluster endpoint instead of the `Bucket`'s external ingress URL (an unroutable placeholder in CI), and trust its certificate via the copied CA. It also pre-cleans with `cleanup.sh`, because `tenant-root` is shared and a leftover `Succeeded` `BackupJob` would satisfy the harness's wait without a backup ever running. Both backup Tests live in the existing `hack/e2e-chainsaw/clickhouse/` suite rather than a new suite directory, so no selector mapping is added: `hack/select-e2e.sh` already derives `clickhouse` both from `packages/apps/clickhouse/**` (the `*-application` rule) and from `examples/backups/clickhouse/**` (the harness path), and both paths therefore select the suite that actually runs the harness. That alignment is the reason for the layout — a separate `clickhouse-backup/` directory would have made an edit to the harness select a suite that cannot exercise it. `hack/select-e2e.sh` and `hack/select-install.sh` are otherwise untouched (the only change is a comment recording that invariant), and the existing `backup example harness edit selects its app suite` case in `hack/select-e2e_test.bats` covers the mapping. ### Verification - `helm unittest` — 17 pass, including 5 new `endpointCA` cases: the unset default renders nothing, the env/mount/volume wiring, a custom key still projected as `ca.crt`, composition with `useSystemBucket`, and `backup.enabled=false` renders nothing. - The CA-trust behaviour was measured against the real image, not assumed: without a CA knob the sidecar reports `x509: certificate signed by unknown authority`; with `SSL_CERT_DIR` the handshake succeeds. - `hack/select-e2e_test.bats` (16) and `hack/select-install_test.bats` (17) pass; `select-install.sh --validate` reports graph OK. - `make generate` is idempotent, so the pre-commit job's `git diff --exit-code` stays clean. - The contracts Test passed in CI on earlier runs of this branch. The round-trip only became CI-live with this work, so it still needs a green E2E run before merge — the branch has since been rebased onto current `main` to get one. ### Screenshots Not applicable — no UI changes. ### Downstream repositories Walked the trigger map against the diff, file by file. The load-bearing change is the new `backup.endpointCA` field in `packages/apps/clickhouse/values.yaml` / `values.schema.json`; everything else is `hack/e2e-chainsaw/`, `examples/backups/`, one docs paragraph, and regenerated artifacts, none of which appear in the map. - [ ] No downstream repository is affected by this change - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/18 — "add, remove or rename a field in an app's `values.schema.json`" reaches the provider's hand-maintained schema, model and expand/flatten pair, so the new nested object is not surfaced until it is added there. Opened as an issue rather than a PR because the natural moment is alongside that repo's next `api/apps/v1alpha1` pin bump, which is a maintainer call. Not ticked, with reasons rather than silence: **cozystack/website** — the trigger is *adding, renaming or removing a package*, and this adds a field to an existing one. `clickhouse` is already in the website `Makefile`'s app list and its reference page is generated from this package's `README.md`, so the regenerated `backup.endpointCA` rows are carried by the existing release-tag automation with no manual PR. **cozystack/external-apps-example** — vendors `hack/package.mk` and `update-crd.sh`, neither of which this PR touches. **cozystack/ansible-cozystack**, **ccp**, **talm**, **cozyhr**, **cozy-proxy**, **cozystack-telemetry-server**, **examples** — no installer values, `hack/` layout, namespace, variant, node-prerequisite, annotation, label or metric in their trigger rows is touched here. ### Release note ```release-note feat(clickhouse): add `backup.endpointCA` so the clickhouse-backup sidecar can verify an S3 endpoint whose certificate is signed by a private CA, such as Cozystack's in-cluster SeaweedFS. Leave it unset for publicly-trusted endpoints; existing releases are unaffected. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional private S3 endpoint CA support for ClickHouse backup/restore sidecars via `backup.endpointCA`, including configurable Secret name and CA bundle key (default `ca.crt`). * Sidecars mount the CA and extend trust while leaving public endpoints unchanged. * **Documentation** * Updated ClickHouse chart docs, operational backup guidance, and backup/restore examples to cover CA setup, overrides, and cleanup. * **Tests** * Added helm-unittest and end-to-end coverage validating CA rendering/mounting behavior and ClickHouse backup/restore workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
3ae1e10c92 |
docs(kubernetes): correct the registryMirrors field and README claims
State in the README that in-guest registry pulls are routable again via talos.registryMirrors, which knob covers which artifact (the mirror routes registry pulls; the worker OS disk image arrives as a raw HTTP artifact under talos.imageFactoryURL and no registry mirror touches it), and the limits: skipFallback is needed for air-gap, machine.registries.config is not passed through, and existing nodes keep their boot-time config until replaced. Trim the field doc comment. The regenerated schema, README tables and cozyrds openAPISchema follow from those sources. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
IvanHunters
|
58910c0edc |
feat(kubernetes): route tenant worker image pulls through a registry mirror
Add a talos.registryMirrors passthrough to the kubernetes and kubernetes-nodes
charts that renders into the worker TalosConfigTemplate machine.registries.mirrors,
so an operator can point a registry such as ghcr.io at an in-cluster pull-through
mirror and a worker boot no longer depends on live public egress. Default {}
keeps the rendered worker reconcile Job byte-identical to before, content-hash
Job name included. Free-form mirror values are shell-escaped for the unquoted
apply heredoc, covered by an execution-level bats test.
Also add the e2e ghcr-mirror (registry:2 pull-through plus a Cilium egress
policy) and wire it into the kubernetes chainsaw suites, so CI exercises the
mirror path and takes the flaky public ghcr.io kubelet-image pull off live
egress.
Refs #3513 (addresses the egress variant of the tenant-worker node-join
failures; the dominant variant is CI-runner CPU starvation, tracked in the
issue).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
|
||
|
Aleksei Sviridkin
|
c20d47a245 |
feat(kubernetes): let a node group log the guest serial console
A worker VM that stalls during boot before Talos apid answers is invisible to every diagnostic the platform has: no Node registers, no certificate request arrives, and talosctl needs the apid that never came up. The guest serial console does show that window, but KubeVirt attaches its guest-console-log container only when the VMI asks for it, and the platform turns console logging off cluster-wide. Add an opt-in per-node-group logSerialConsole that sets the KubeVirt field, which takes precedence over the cluster-wide setting. The key renders nothing unless true -- including when set to false -- because the KubevirtMachineTemplate is content-hash named, so a byte emitted for a group that did not ask for it renames the template and rolls every live worker VM. The description carries the two costs the field name does not show. Changing the value rolls the group it is set on, so the stuck VM whose console was wanted is replaced along with the rest and the setting buys visibility into the next boot rather than the current one. And the cluster-wide default it overrides exists because that same container has been seen holding virt-launcher in PodInitializing, which would take a group down rather than explain it. Worker VMs are emitted by two charts. Both carry the field, and the golden-parity check gains a case that keeps their output identical. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrey KolkovandClaude Opus 4.8
|
9379729b31 |
fix(clickhouse): address review notes on endpointCA and the CA copy
Review notes from IvanHunters on #3385 (LGTM with non-blocking findings). 1. The backup-s3-ca volume carries no `optional: true` while the adjacent useSystemBucket credential refs deliberately do, and the asymmetry was undocumented — an operator naming a Secret that does not exist gets the whole ClickHouse Pod stuck in ContainerCreating, not just the sidecar. Kept the fail-closed behaviour and wrote down why the two cases differ: credentials are projected by the platform after install, so a strict ref would wedge every fresh release until the first BackupJob, whereas this CA is named by the tenant, must already exist, and is trust material — optional would mount an empty dir, the Pod would look healthy, and the first backup would fail deep inside clickhouse-backup with an x509 error naming neither the Secret nor the chart. The comment also states the blast radius plainly. 2. The CA probe swallowed stderr, so an RBAC denial on the cross-namespace read was reported as "CA not found" followed by advice (set S3_CA_SECRET) that cannot fix a permissions problem. It now separates present / absent / not-allowed-to-look and says so, and the README no longer labels step 03 simply "tenant": copying the CA reads a Secret in tenant-root, which is a platform capability. Added a section spelling out the three ways to satisfy that (admin credentials, a one-off admin copy, or skipping the CA entirely on a publicly-trusted endpoint). 3. Caveat, and a usability regression this branch introduced: on a cluster with a publicly-trusted endpoint and no seaweedfs, step 03 hard-exited on a default the user never chose. It now distinguishes an explicitly named Secret (still an error when missing — a typo should not be guessed past) from the unset default, where it warns and continues with backup.endpointCA unset. Verified all four paths against a fake kubectl: Forbidden, absent+default, absent+explicit, and present. 4. Caveat: endpointCA reaches the clickhouse-backup sidecar only. The legacy `schedule` CronJob writes to S3 through restic and does not consume it, so a private-CA endpoint combined with `schedule` still fails TLS. Documented on the field itself, so it lands in values.schema.json, the README and the cozyrds openAPISchema. The PR body's claim about select-e2e.sh/select-install.sh mappings was stale after the restructure that removed them; corrected there rather than in code. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Timofei Larkin
|
e5d4940338 |
chore(codeowners): don't gate per-app generated files on architect review
api/apps/v1alpha1/*/types.go and packages/system/*/cozyrds/*.yaml are entirely produced by `make generate` from each app's own values.yaml, but the /api/ codeowner rule pulled every app's regenerated types.go into an architect-only review, regardless of which area actually changed. Correctness there is already enforced by the pre-commit make-generate CI hooks, not by human review, so list no owner for these generated paths and let that check carry the weight instead. Also fixes a stale api/.gitattributes pattern that never matched any real file (zz_generated_deepcopy.go vs the actual zz_generated.deepcopy.go), and adds a matching linguist-generated attribute for the cozyrds ApplicationDefinition manifests, so both collapse in the PR diff view like the Go files already do. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
2687e095f4 |
feat(clickhouse): let the backup sidecar trust a private S3 CA
The clickhouse-backup sidecar had no TLS/CA surface at all: bucket coordinates
and credentials arrive as S3_* env, and nothing let it verify an endpoint whose
certificate is signed by a private CA. Cozystack's own in-cluster SeaweedFS is
exactly that case — it serves :8333 behind the self-signed "SeaweedFS CA" — so a
ClickHouse release could only ever back up to a publicly-trusted endpoint, while
MariaDB (storage.s3.tls.caSecretKeyRef) and Postgres (barman endpointCA) have had
a CA field all along.
Add backup.endpointCA {name, key}. When set, the chart mounts that Secret into
the sidecar and points SSL_CERT_DIR at it. Empty (the default) renders no env, no
mount and no volume, so existing releases are untouched.
SSL_CERT_DIR rather than AWS_CA_BUNDLE, measured against
altinity/clickhouse-backup:2.7.4 with a self-signed endpoint and public AWS S3 in
turn: Go reads SSL_CERT_DIR in addition to the system bundle, so both verify;
AWS_CA_BUNDLE replaces the SDK trust pool, so the private endpoint verifies and
public S3 starts failing `x509: certificate signed by unknown authority`. That
would silently break a release that also talks to a public endpoint.
S3_DISABLE_CERT_VERIFICATION was rejected outright — it drops verification
instead of trusting one more CA.
Covered by helm-unittest: the unset default, the mount/env/volume wiring, a
custom key still projected as ca.crt, composition with useSystemBucket, and that
backup.enabled=false renders nothing.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrei Kvapil
|
8fc8d23dfd |
fix(computeplane): address review round three
Gate the Proxied ingress Service on _namespace.etcd like the addon HelmRelease it fronts, so an etcd-less tenant with the addon pinned on does not get a selector-less Service (+ test). Correct the secrets framing: the tenant-visible surface is an allowlist (include: [] withholds everything), so drop the never-load-bearing exclude entry, the bats cross-check and the rename-coupling comments, and state the real mechanism. Fix the etcd prerequisite docs (an ancestor's etcd satisfies the wrapped chart; the awaiting-etcd beacon is the diagnostic), reword the tenant toggle to stop advertising the not-yet-shipped placement routing, require maxReplicas >= 1 on the ingress-role node group, and fix comment drift (Service name after the release rename, placement-hook label, package-guard lookup semantics). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Andrei Kvapil
|
b2232904b6 |
feat(tenant): add the computeplane module toggle
Wire the computeplane module into the tenant chart as a bool toggle, matching the other tenant modules. The module's own Helm release takes the -module suffix so the canonical release name computeplane (and with it the computeplane-admin-kubeconfig Secret) stays reserved for the cluster HelmRelease the module renders. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
5a7e4c5391 |
feat(api)!: require the application release prefix to be dot-free (#3411)
## What this PR does Constrains `ApplicationDefinition.spec.application.release.prefix` to `^[a-z0-9-]*$`, which excludes the dot. Release names are `<prefix><app>`, and the tenant CA trust anchor is projected to `<release>.tenant-ca`; the dot is the separator that keeps that name unreachable by any engine, so a prefix carrying a dot would break the guarantee. Every in-tree prefix already conforms. Split out from #3299. ```release-note feat(api)!: `ApplicationDefinition.spec.application.release.prefix` must match `^[a-z0-9-]*$` (dot-free). A definition with a prefix containing a dot or any other character is rejected at admission; correct it before applying this CRD. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Added validation for release prefixes to allow only lowercase letters, numbers, and hyphens. * Updated guidance explaining how prefixes affect generated release names and tenant CA trust anchors. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
6ecb36ffc7 |
feat(cozystack-controller): extract application CA into a key-free tenant Secret (#3407)
## What this PR does Adds an engine-agnostic controller that publishes a key-free `<release>.tenant-ca` Secret (only `ca.crt`) for a managed application, so a tenant can verify the application's TLS without ever seeing a private key. The source is declared, not guessed: a chart renders a namespaced `TenantProjection` sentinel (group `internal.cozystack.io`) naming the CA Secret to lift; the controller watches the sentinel, extracts `ca.crt`, and writes the projection owner-referenced to the sentinel, so garbage collection is native. The projection carries the `internal.cozystack.io/tenant-ca` label the lineage webhook turns into tenant visibility. Postgres is the first consumer. The trust boundary is RBAC: no tenant role grants any verb on `internal.cozystack.io`, so a tenant cannot forge a sentinel. A companion `ValidatingAdmissionPolicy` that pins the writer to helm-controller ships separately as defence in depth. This is the controller-and-CRD slice, split out from #3299 for focused review. It is the only piece required for the feature to work; the render-time helper, the writer policy, the capability-gate, and the release-prefix validation land as their own PRs. ### Downstream repositories Walked the trigger map against the diff. This adds an internal CRD and controller and a sentinel to the Postgres chart; no app is added or renamed and no `values.schema.json` changes, so the typed provider and Ansible are unaffected. The one to confirm is the Postgres README edit — if the website mirrors it, it needs a refresh. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: ### Release note ```release-note feat(cozystack-controller): publish a key-free per-application CA trust anchor (`<release>.tenant-ca`) that tenants read to verify managed-application TLS, declared through a new `TenantProjection` CRD. Postgres is the first consumer. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added tenant-facing CA certificate projections for PostgreSQL deployments. * Tenant secrets now provide only the CA certificate, excluding private keys. * Added support for configuring and monitoring tenant CA projections. * **Documentation** * Updated PostgreSQL TLS instructions to retrieve certificates through tenant secrets. * **Bug Fixes** * Improved protection against invalid certificates, secret collisions, and accidental exposure of private key material. * Added validation to ensure projected certificates remain synchronized with their sources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d443a092ad |
feat(api)!: require the application release prefix to be dot-free
Constrain ApplicationDefinition spec.application.release.prefix to lowercase DNS-1123 characters (^[a-z0-9-]*$), which excludes the dot. Release names are "<prefix><app name>" and the tenant CA trust anchor is projected to "<release>.tenant-ca"; the dot is the separator that makes that name unreachable by any engine, so a prefix carrying a dot would break the guarantee. BREAKING CHANGE: an ApplicationDefinition whose release prefix contains a dot, or any character outside [a-z0-9-], is now rejected at admission. Existing definitions with such a prefix must be corrected before this CRD is applied. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
37a91f98b8 |
feat(cozystack-controller): extract application CA into a key-free tenant Secret
Every catalog engine issues a per-release CA, but the object holding ca.crt usually holds a private key beside it (cert-manager's tls.key, CloudNativePG's ca.key), so granting a tenant read on it hands over key material too. Add a controller that publishes the trust anchor as one canonical, key-free Secret per release, "<release>.tenant-ca", holding only ca.crt. A chart declares the anchor by rendering a namespaced TenantProjection sentinel (internal.cozystack.io/v1alpha1) that names the source Secret and key. The controller rebuilds ca.crt from the parsed certificate DER rather than copying the source, refuses any value carrying private key material, and owns the projection by owner reference so a foreign Secret at the same name is never overwritten. No tenant RBAC grants any verb on internal.cozystack.io; a ValidatingAdmissionPolicy pins writes to helm-controller as defence in depth. The controller does not grant tenant read access — the lineage webhook does, from the ApplicationDefinition's spec.secrets. This side stamps the engine-agnostic internal.cozystack.io/tenant-ca label and forces a re-admission when those selectors change. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Ivan Okhotnikov
|
73be7032a3 |
fix(kubernetes-nodes): drop unbacked cluster option source
The cluster field referenced x-cozystack-options source 'kubernetes', which has no provider (cozystack-api returns NotFound), so the dashboard dropdown 404s. Drop the annotation so the field is a plain text input; a real provider listing Kubernetes CRs is a follow-up. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
Ivan Okhotnikov
|
f842c0a17e |
feat(kubernetes-nodes): generate API types for KubernetesNodes
Generated by cozyvalues-gen + controller-gen from values.yaml. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
myasnikovdaniil
|
960d853706 |
feat(api): typed waitStrategy/healthCheckExprs for kstatus readiness (#2642) (#3273)
## What Adds typed `waitStrategy` + `healthCheckExprs` fields so a package/app can declare the Flux helm-controller v1.5.0 kstatus health gate for the custom resource it wraps — making `HelmRelease.Ready` mean "the CR is actually healthy" instead of "helm applied the CR". This is the operator/apiserver foundation of the kstatus epic (#2642); the API surface was reviewed and approved by the API owner in #3263. Commits (per the review's "isolated step-0" request): 1. `build(deps)` — the dependency bump (see below). 2. `feat(api)` — the fields, plumbing, tests, and regenerated code. 3. `fix(api)` — match the openapi code-generator pin to the 0.35 runtime (fixes an aggregated-apiserver crash the bump surfaced). 4. `fix(api)` — declare `OpenAPIModelName` on the apps types so SSA resolves catalog kinds. 5. `docs(api)` — note that `disableWait` silently defeats `healthCheckExprs`. ## API surface Typed fields on **both** HelmRelease-building paths: - `ApplicationDefinition.spec.release` (apiserver / catalog apps → `convertApplicationToHelmRelease`) - `ComponentInstall` (operator / platform packages → `buildHelmReleaseSpec`) ```go // WaitStrategy maps to HelmReleaseSpec.WaitStrategy.Name — a deliberate scalar // simplification of the upstream {name} object. One of poller|legacy. WaitStrategy string // HealthCheckExprs maps to HelmReleaseSpec.HealthCheckExprs (the upstream // kustomize.CustomHealthCheck type verbatim). HealthCheckExprs []kustomize.CustomHealthCheck ``` `config.ResolveWaitStrategy` is shared by both builders and **couples the default**: `healthCheckExprs` are only evaluated under the `poller` strategy, so when expressions are set and no strategy is given, the generated HelmRelease defaults to `poller`. A package that sets only `healthCheckExprs` is therefore self-contained, independent of the controller's global default. ## The dependency bump (commit 1) The `WaitStrategy` / `kustomize.CustomHealthCheck` / `HelmReleaseSpec.HealthCheckExprs` types do not exist below `helm-controller/api` v1.5.0 — the runtime is already v1.5.0 (shipped by #2602), so this closes a Go-types-vs-runtime skew. It is **not** an isolated bump: v1.5.x requires **k8s 0.35 + controller-runtime 0.23**, cascading the k8s stack 0.34 → 0.35 (and the go directive to 1.26). cozystack replaces `k8s.io/apimachinery` with a fork carrying the still-open upstream `ConvertToVersion` patch (k/k#135537). That single commit rebases cleanly onto apimachinery v0.35.0; published at `cozystack/apimachinery@release-1.35-pr135537`, and `go.mod` pins its pseudo-version. **The repo builds with zero source changes under the new stack.** > Maintainer note: please confirm the `release-1.35-pr135537` fork branch naming, or fold it into the `cozystack` branch as you prefer — the pin can be updated to match. ## Not included (conscious YAGNI) `disableWait` is intentionally omitted. The only known need (the kubernetes tenant-addon deadlock) was resolved child-side, and `waitStrategy: legacy` is not equivalent (legacy still waits Helm-v3-style; `disableWait` skips waiting entirely). Add it later when a concrete leaf needs a true no-gate escape hatch. ## Tests - `config.TestResolveWaitStrategy` — the couple-the-default logic (unset+exprs→poller; explicit legacy honored; unset+no-exprs→nil). - Operator + apiserver cases mirror each other (`TestBuildHelmReleaseSpecHealthCheckExprs` / `TestConvertApplicationToHelmRelease_HealthCheckExprs`). - `go build ./...` clean, `go vet` clean, `gofmt` clean, codegen regenerated and committed. The 4 backupstrategy CRDs regenerate with benign k8s-0.35 PodSpec description drift (DynamicResourceAllocation alpha→stable, toleration Lt/Gt, resizePolicy note) — no functional change. ## Follow-up Phase 3 (per-leaf adoption — postgres/kafka/mongodb/etc. declaring their CEL via these fields) builds on this. The full CRD→CEL catalog is maintained in the epic's readiness audit. Refs #2642 ```release-note feat(api): add `waitStrategy` and `healthCheckExprs` to ApplicationDefinition and ComponentInstall, so a HelmRelease reports Ready only once the custom resource it wraps is actually healthy ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable CEL health-check expressions for application releases and package components. * Added `waitStrategy` support (`poller` and `legacy`) with automatic `poller` selection when health checks are configured and no strategy is set. * Propagated these settings into generated Helm releases, affecting readiness/wait behavior. * **Documentation** * Updated Application and PackageSource schemas to include `healthCheckExprs` and `waitStrategy`. * Extended related Kubernetes CRD documentation, including projected pod certificate annotations and workload reference fields. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey KolkovandClaude Fable 5
|
04d74bbfad |
chore(postgres): sync generated types.go docstring for bootstrap.serverName
The pre-commit run-make-generate hook regenerates the Go types from the values annotations; the serverName default note landed in values.yaml, README, schema and postgres-rd but missed types.go. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey KolkovandClaude Opus 4.8
|
5fbc90047d |
docs(apps/postgres): modernize backup annotations/README for the plugin flow
Review follow-up (cozystack#3300): the apps/postgres user-facing docs still described the removed native spec.backup.barmanObjectStore / archive_command model. Update the values.yaml @field/@typedef/@param annotations (useSystemBucket, schedule, S3CredentialsSecret, EndpointCA, endpointCA, the @param backup note, bootstrap.serverName) and the hand-written README Operations prose to the barman-cloud plugin + ObjectStore model, then regenerate values.schema.json, the README parameters table, the api types, and the postgres-rd ApplicationDefinition schema (which feed the dashboard form help + cozyrd schema). Field names, defaults, and the storageClass immutability marker are unchanged; only backup/ bootstrap description text changed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
13ff340277 |
fix(backups): migrate CNPG backups to the barman-cloud plugin
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrei Kvapil
|
fbbe4af7df |
refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass (#3218)
## What Removes the `network.cozystack.io` API group — `ExposureClass` + `ServiceExposure` (#3081) — and its reconciler, in favor of native `Service` `type: LoadBalancer` + `loadBalancerClass`. ## Why Full rationale and discussion in #3164. For **managed applications** the chart already owns the Service, so: - `type: LoadBalancer` + `loadBalancerClass` + an admin-provisioned pool covers vendor-neutral external exposure without a dedicated API group and controller; - the `externalIPs` (node-IP) default never went through `ServiceExposure` (its backend is a no-op) and is left unchanged; - the only current consumer, the host ingress, keeps rendering `type: LoadBalancer` and just drops the `ServiceExposure`; the admin provisions the address pool / announcer out of band, as the install docs already assume. `ServiceExposure`'s one unique feature — dynamic per-namespace scoping of a shared VIP pool — is a convenience the managed-app path does not need, and it does not justify a second tenant-networking API group next to `sdn.cozystack.io`. ## Changes - Delete `api/network/v1alpha1` and `internal/controller/serviceexposure` (+ CRD definitions, e2e bats, controller registration, RBAC). - Host ingress: drop the `ServiceExposure` template; add an optional `publishing.loadBalancerClass` that sets `spec.loadBalancerClass` on the host-ingress LoadBalancer Service. - The `externalIPs` path and default behavior are unchanged. ## Verification - `go build ./...` — green. - `helm unittest` — `core/platform` 72/72, `extra/ingress` 25/25 (incl. host-vs-tenant `loadBalancerClass` scoping and the immutable-field migration guard). ## Migration None required. `network.cozystack.io` (#3081) has never shipped in a release — it is on `main` only (not backported to `release-1.5`; absent from v1.4.x / v1.5.x), so no upgrading cluster has these CRDs and this removes it before it is ever released. Clusters that track `main` and explicitly set `publishing.exposureClass` (opt-in, only since 2026-06-30) should provision their MetalLB / Cilium address pool themselves and set `publishing.loadBalancerClass`; any orphaned `cozystack-<class>` pool can be deleted by hand. Default clusters (externalIPs path) are unaffected. ## Status **Draft / RFC for the #3164 discussion** — this removes an API group that landed only days ago, so nothing here should merge before the design call. cc @lexfrei (author of #3081) and @lllamnyp (#3164). Deferred: when Kubernetes actually removes `Service.spec.externalIPs`, the node-IP default migrates via a MetalLB `IPAddressPool` with no `L2Advertisement` (allocate-but-don't-announce) — a separate change. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for configuring the ingress host Service with a native `loadBalancerClass` value. * Expanded chart tests to verify class wiring and immutability behavior across install and upgrade scenarios. * **Bug Fixes** * Removed the legacy exposure-class/service-exposure flow and related controller behavior. * Updated templates and CRD packaging so the platform no longer renders or depends on those deprecated resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
f1619215ff |
docs(api): note disableWait silently defeats healthCheckExprs
The field docs already say healthCheckExprs are only evaluated under the poller wait strategy, but that is only half the precedence rule. Upstream evaluates them only when the Helm action itself has wait enabled (helm-controller/api@v1.5.1 v2/helmrelease_types.go:193-199), so the release.cozystack.io/helm-install-disable-wait annotation makes them a silent no-op whatever waitStrategy says — no error, no warning, the release just reports Ready as before. This is reachable today: packages/system/kubernetes-rd already sets that annotation and is a plausible candidate for CEL expressions in phase 3. Document it on the ApplicationDefinition field (so it reaches the CRD description) and next to HelmInstallDisableWaitAnnotation. ComponentInstall has no disableWait, so its field doc is unchanged. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
1ea4eb452c |
fix(api): regenerate OpenAPI with code-generator v0.35.0
This PR bumps the runtime k8s libraries to 0.35 (k8s.io/apiserver,
kube-openapi 20250910) but hack/update-codegen.sh still pinned the
openapi code-generator at v0.34.1.
The 0.34.1 openapi-gen keys the version.Info definition by its Go
import path ("k8s.io/apimachinery/pkg/version.Info"), whereas
kube-openapi 0.35 now resolves that model via the new OpenAPIModelNamer
interface — version.Info{}.OpenAPIModelName() ->
"io.k8s.apimachinery.pkg.version.Info". The key mismatch makes the
aggregated apiserver fatal while building its root OpenAPI spec
("cannot find model definition for io.k8s.apimachinery.pkg.version.Info"
-> klog.Fatalf, exit 255), crashlooping cozystack-api. That takes down
the apps.cozystack.io aggregated API, so the Bucket kind can no longer
be resolved and the backupstrategy-controller HelmRelease install fails,
reddening E2E.
Bump the code-generator pin to v0.35.0 (matching the runtime) and commit
the regenerated artifacts. No hand edits to generated files.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
96a9652fb1 |
feat(api): typed waitStrategy/healthCheckExprs for kstatus readiness
Add first-class fields so a package/app can declare the Flux helm-controller v1.5.0 kstatus health gate for the custom resource it wraps, so HelmRelease.Ready means "the CR is actually healthy" instead of "helm applied the CR" (#2642). API-owner approved in #3263. - api: WaitStrategy (scalar, maps to HelmReleaseSpec.WaitStrategy.Name — a deliberate simplification) + HealthCheckExprs ([]kustomize.CustomHealthCheck, the upstream type verbatim) on both ApplicationDefinition.spec.release (the apiserver path) and ComponentInstall (the operator path). - config.ResolveWaitStrategy: shared by both HelmRelease builders. Couples the default — healthCheckExprs imply the poller strategy when none is set, since exprs are only evaluated under poller. This removes the "must also set waitStrategy: poller" footgun and does not depend on the controller's global default. - thread through buildHelmReleaseSpec (operator) and convertApplicationToHelmRelease (apiserver); the apiserver loads the typed spec.release fields in start.go. - regenerate deepcopy, the ApplicationDefinition and PackageSource CRDs, and openapi. The backupstrategy CRDs also regenerate — benign k8s-0.35 PodSpec description drift, not a functional change. disableWait is intentionally not added (YAGNI): the only known need (the kubernetes tenant-addon deadlock) was resolved child-side, and waitStrategy: legacy is not equivalent (legacy still waits Helm-v3-style; disableWait skips waiting entirely). Add it later when a concrete leaf needs a true no-gate escape hatch. Refs #2642 Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniilandmattia-eleuteri
|
db4db4a6e8 |
feat(kubernetes): per-node-group nodeHealthCheck overrides
Add optional nodeGroups[name].maxUnhealthy and nodeGroups[name].nodeStartupTimeout
fields that override the cluster-wide nodeHealthCheck defaults per worker node
group. When unset, a group inherits nodeHealthCheck.{maxUnhealthy,nodeStartupTimeout},
so existing values are unaffected. The MachineHealthCheck render resolves the
per-group value first, reuses the existing int-or-string validation, and names
the offending group in the fail() message when an override is invalid.
Delivers the per-group granularity of the standalone maxUnhealthy proposal on
top of the cluster-wide nodeHealthCheck from the Talos worker bootstrap PR.
Co-authored-by: mattia-eleuteri <mattia@hidora.io>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Aleksei Sviridkin
|
b73860d4c9 |
fix(kubernetes): bump tenant worker Talos to v1.13.6
Tenant worker VMs were pinned to Talos v1.13.0, which ships Linux 6.18.24. That kernel predates both fixes for a pair of related KVM x86 shadow-paging use-after-free bugs: CVE-2026-46113, fixed in 6.18.30, and CVE-2026-53359, fixed in 6.18.38. A worker's own kernel runs KVM whenever a tenant enables nested virtualization inside the VM, and either bug lets a nested guest escape into the worker. Talos v1.13.6 ships Linux 6.18.38. Stable kernel branches are cumulative, so it is the first v1.13 patch release carrying both fixes: the pinned v1.13.0 ships 6.18.24 and carries neither, while v1.13.5 ships 6.18.36 and carries only CVE-2026-46113. Nested virtualization stays available to tenants — the kernel bump is the fix. The new pin changes the rendered worker spec, so the content hash in the KubevirtMachineTemplate and talos-reconcile Job names rotates and each worker pool rolls once on upgrade. Pinned hashes in the chart tests are recomputed accordingly. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f60fe2270b |
Merge remote-tracking branch 'origin/main' into feat/kubernetes-talos-image-factory-override
Signed-off-by: Aleksei Sviridkin <f@lex.la> # Conflicts: # packages/system/kubernetes-rd/cozyrds/kubernetes.yaml |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
3eb182ddf1 |
feat(apps/kubernetes): make Talos image factory URL and installer repo overridable
Tenant worker VMs boot from a Talos raw disk image that CDI streams over HTTP from the hardcoded public Talos Image Factory, and the in-guest talos-reconcile Job pulls its installer image from the same hardcoded host. Neither was overridable, so air-gapped / rate-limited / flaky-egress environments could not point workers at a mirror (the README flagged this as a Phase-2 gap), and CI had no way to avoid bulk-pulling the OS image from the public internet per worker. Add two chart values, both defaulting to the public factory so rendered output is byte-identical and behaviour is unchanged: - talos.imageFactoryURL (default https://factory.talos.dev) - base URL for the worker OS disk image streamed in by CDI. - talos.installerRepository (default factory.talos.dev/installer) - OCI repository prefix for the in-guest installer image. Regenerated values.schema.json, the aggregated-API Go types, the kubernetes-rd cozyrds schema and the README via `make generate`; added unittests for the override and trailing-slash trimming. Refs #3231 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
IvanHunters
|
d30a2cfffb |
docs+test(apps/kubernetes): align nodeGroups default docs with actual zero-worker topology + regression-guard the replicas fix
Follow-ups from the self-review of #3233: - `values.yaml` / helper docstring / regenerated README + schema + cozyrds openAPISchema + Go type doc: the previous "one ingress-nginx worker" contract was already misleading (helper default carried `minReplicas: 0` but the chart hardcoded `spec.replicas: 2`), and the replicas fix makes the docstring strictly false — an empty-nodeGroups install now provisions zero workers until either the cluster-autoscaler responds to an unschedulable Pod or the operator explicitly scales the group. Update the docstrings to describe the actual behaviour and to spell out the ingress-nginx-addon implication so operators are not surprised at addon-install time. - `tests/nodegroups_default_test.yaml`: two regression cases pinning `MachineDeployment.spec.replicas == $group.minReplicas` for `minReplicas: 0` (the render-side OIDC bats path — zero worker DataVolumes provisioned) and for `minReplicas: 2` (the kubernetes-latest / kubernetes-previous bats path — byte- identical to pre-PR behaviour). Without these a future refactor can silently re-hardcode `replicas: 2` and turn no tests red. helm-unittest suite grows from 171 to 173, both new cases pass. Ref: cozystack/cozystack#3231. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Andrei Kvapil
|
3dbc7d3662 |
refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass
Remove the network.cozystack.io API group (ExposureClass + ServiceExposure) and its reconciler. Managed-application charts own their Service, so native type: LoadBalancer + loadBalancerClass + an admin-provisioned pool covers external exposure without a dedicated API group and controller. The host ingress -- the only consumer -- keeps rendering type: LoadBalancer and gains an optional publishing.loadBalancerClass to pick the LB controller; the externalIPs (node-IP) default path is unchanged. Rationale in cozystack#3164. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Timofei Larkin
|
7b65cf2afd |
feat(platform): migrate to etcd-operator v1alpha2 (#2859)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for the newer etcd snapshot workflow (BackupClass/BackupJob/RestoreJob) with in-place restore. * Updated etcd chart/operator packaging and migration flow for the v1alpha2 operator release. * **Bug Fixes** * Switched readiness/condition handling to **Available**. * Improved waits to cover all etcd member pods during restore and scenario steps. * **Documentation** * Refreshed etcd backup/restore docs and example scenarios to match the v1alpha2 APIs and parameters. * **Tests** * Updated e2e coverage to validate v1alpha2 operator runtime contracts and replaced legacy backup-block tests with a snapshot round-trip test. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
cc5d688e15 |
feat(platform): migrate to etcd-operator v1alpha2
Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Ivan Okhotnikov
|
7b8a330ad4 |
docs(apps/kubernetes): update users[].email docstring — email claim, not preferred_username
Gemini flagged that the `values.yaml` @field docstring on `oidc.users[].email` still described the claim mapping as `preferred_username`, misleading operators after the claim was switched to `email` for real (see the earlier fix and its end-to-end verification on dev3). Rewrite the description to match the actual mechanic: `email` claim, requested via `--oidc-extra-scope=email` in the chart-generated kubeconfig, emitted by every conformant OIDC provider. `make generate` re-propagates the update through README.md, types.go and the cozyrds embedded schema. Note on the other finding in the same review — Gemini also flagged `audiences` as being incorrectly nested under `issuer` in the AuthenticationConfiguration Secret. That is a false positive: per the kube-apiserver v1beta1 spec (apiserver.config.k8s.io/v1beta1, `JWTAuthenticator.Issuer`), `audiences` IS a child of `issuer`, not a sibling of `jwt[i]`. The current shape is correct and verified end-to-end on dev3 (alice/bob tokens carry `aud: [tenant-test-kubernetes-test3, kubernetes]` and apiserver validates them successfully with CRB-driven RBAC). Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
9c73b6ef10 |
fix(apps/kubernetes): render-fail when spec.oidc.mode collides with legacy --oidc-* extraArgs
Reviewer flagged a real customer-upgrade regression: the previous values.yaml docstring on `controlPlane.apiServer.extraArgs` documented `--oidc-issuer-url`/`--oidc-client-id` as a valid usage. An existing customer who wired OIDC by hand through that passthrough (following the doc) would see their tenant kube-apiserver BOOT-FAIL the moment they toggle `spec.oidc.mode` to `System` or `CustomConfig`: kube-apiserver refuses to boot with both `--authentication-config` (injected by the new spec.oidc) and any legacy `--oidc-*` flag — `--authentication-config cannot be used with any of --oidc-*`. Add a render-time guard: when `spec.oidc.mode != None`, scan the user-supplied `controlPlane.apiServer.extraArgs` for any flag with prefix `--oidc-` and fail with a clear message pointing at the `spec.oidc.mode: CustomConfig` migration path (they can carry the same issuer/audience inside `customConfig.config`). When `spec.oidc.mode: None` the passthrough still works — customers who manage OIDC entirely by hand keep working until they opt into the new selector explicitly. Also update the values.yaml docstring on `extraArgs` to steer OIDC users to `spec.oidc.mode` and warn against the collision. Two new helm-unittest cases: the collision fails, and mode=None with legacy flags renders normally (backward compat). Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Ivan Okhotnikov
|
5950450758 |
feat(apps/kubernetes): add Kubernetes.spec.oidc selector API
Introduce a flat selector — `mode: System | CustomConfig | None` — plus a `users[]` list and a `customConfig` block for tenant-supplied AuthenticationConfiguration. Default `None` preserves today's behavior. The `System` mode trusts the platform `cozy` realm via a per-cluster public client; `CustomConfig` accepts an inline `AuthenticationConfiguration` or a Secret reference (mutually exclusive). The `users[]` list drives per-user ClusterRoleBindings inside the tenant cluster, with two roles: `admin` and `view`. Regenerates deepcopy, JSON schema, README, and the kubernetes-rd cozyrds embedded schema. Implements Phase 1 API of the per-tenant OIDC design proposal (cozystack/community#24). Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io> |
||
|
Aleksei Sviridkin
|
22c96d8dac |
fix(kubernetes): default worker disk storageClass to the application storageClass
Worker node VMs set evictionStrategy: External, so they live-migrate. Live migration requires the system DataVolume on a ReadWriteMany volume, and linstor-csi only grants RWX on a DRBD-backed StorageClass. When a node group leaves storageClass empty the template previously omitted the field and fell through to the management cluster default StorageClass, which may be a non-DRBD class the CSI driver rejects for RWX — breaking migration. Fall back to the application-level storageClass (the replicated/DRBD class) when the node group storageClass is empty, instead of the cluster default. When both are empty the field is still omitted, preserving the explicit cluster-default opt-out. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
myasnikovdaniil
|
3e20a0cd41 |
fix(kubernetes): make node group resources optional when instanceType is set (#3121)
## What this PR does
A tenant Kubernetes node group can be sized two ways: by `instanceType`
or by an explicit `resources` block. The generated values schema,
however, marked `resources` as required on every node group, so a node
group sized purely by `instanceType` was rejected at install time:
```text
at '/nodeGroups/<name>': missing property 'resources'
```
The same failure occurs through the typed `Kubernetes` CR: an empty
`resources: {}` is dropped during the CR-to-HelmRelease translation, so
by validation time the key is absent and the required check fails — even
though sizing was already fully specified by `instanceType`.
This marks `resources` optional in the node group definition and
regenerates the dependent artifacts (`values.schema.json`, API types,
README, and the `ApplicationDefinition`). No change is needed to emit
the VM: the chart already renders `domain.cpu`/`domain.memory` only when
`resources` is present and `instancetype` only when set, so a node group
sized purely by `instanceType` now validates and installs. Node groups
that specify `resources` are unaffected.
`instanceType` and `resources` are alternative, mutually exclusive
sizing sources — KubeVirt rejects a VirtualMachine that references an
instancetype and also sets explicit `domain.cpu`/`domain.memory`, since
instancetype CPU/memory cannot be overridden. To avoid emitting a VM
that the instancetype admission webhook would later reject, the chart
now fails fast at render time with a clear message when a node group
sets both a non-empty `instanceType` and explicit `resources`.
Tests: helm-unittest coverage proves a node group with `instanceType`
and no `resources` passes schema validation and renders without explicit
`domain.cpu`/`memory`; a complementary case confirms explicit
`resources` are still honoured; and a `failedTemplate` case pins the
render-time rejection of the conflicting both-set combination. The
omit-resources case fails against the pre-fix schema with the exact
error above, so it is a real regression guard.
Addresses #3046.
### Screenshots
N/A — schema and chart change, no UI impact.
### Release note
```release-note
fix(kubernetes): allow a node group to omit `resources` when it is sized by `instanceType` (previously failed schema validation with "missing property 'resources'"); reject the unsupported combination of `instanceType` and explicit `resources` at render time
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Worker node groups can optionally size CPU/memory via `resources`, and
can omit `resources` when using `instanceType`.
* **Bug Fixes**
* Render now validates CPU/memory rules: `instanceType` with only one of
`resources.cpu`/`resources.memory` fails, and `instancetype` is omitted
when both `resources.cpu` and `resources.memory` are set (with
`instanceType` ignored).
* **Documentation**
* Updated API/Helm docs and JSON schema to clarify `resources` vs
`instanceType` precedence and the “set both CPU and memory (or neither)”
requirement.
* **Tests**
* Added helm-unittest scenarios covering omission, precedence, and
render-time failure cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
bf14a73b39 |
feat(api): add SecurityGroup network policy resource (sdn.cozystack.io) (#2922)
## What this PR does
Introduces a tenant-facing `SecurityGroup` resource in a new
`sdn.cozystack.io/v1alpha1` API group, served by the Cozystack
aggregated API server as a 1:1 projection of a `CiliumNetworkPolicy`. A
SecurityGroup attaches to one of the tenant's managed applications by
reference (`spec.targetRef: {apiGroup, kind, name}`) and declares
allow-list ingress/egress for that application. Tenants manage network
policy for their own apps entirely through
`securitygroups.sdn.cozystack.io`, without being granted any access to
the `cilium.io` API group.
The translation is synchronous and stateless — the API server converts
each SecurityGroup into a CiliumNetworkPolicy in the same namespace on
write and back on read; there is no controller. The backing policy's
`endpointSelector` is **derived** from `targetRef` (matched on the
application's lineage labels
`apps.cozystack.io/application.{group,kind,name}`), never copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
Policies owned by this API carry an `sdn.cozystack.io/securitygroup`
marker label that the storage always re-asserts and hides from the view,
so it only ever surfaces or mutates its own policies and never touches
platform-managed network policies.
The value is self-service network policy on a Cozystack-native,
RBAC-scoped surface, without exposing Cilium's version-coupled CRDs to
tenants. Because the selector is derived from an authorized application
reference rather than free-form tenant input, the RBAC boundary is
structural, not selector-validated: a tenant cannot express network
policy against arbitrary or platform-owned pods. The full design —
selector derivation from lineage labels, the structural boundary, the
additive-policy reality (a SecurityGroup widens a managed application's
allowed traffic but cannot sever its platform-managed management plane,
and equally cannot tighten below the platform baseline), the in-tree
CiliumNetworkPolicy mirror that avoids a Cilium Go-module dependency,
RBAC, and deferred extensions (raw-pod targeting, cluster scope,
reusable groups) — is in `pkg/apis/sdn/DESIGN.md`.
Implements the attachment model from #1614.
### Release note
```release-note
feat(api): add the SecurityGroup resource (sdn.cozystack.io/v1alpha1) — a tenant-facing, namespace-scoped firewall that attaches to a managed application by reference and projects 1:1 onto a CiliumNetworkPolicy with a machine-derived endpoint selector, letting tenants manage their application's network policy without cilium.io access.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the `SecurityGroup` API resource (`sdn.cozystack.io` v1alpha1)
with a namespaced projection backed by Cilium network policies.
* Enabled the SDN `APIService` and wired SDN routing, scheme, and REST
support.
* Expanded RBAC to include SecurityGroup access and required Cilium
policy permissions.
* **Documentation**
* Added a design document describing projection behavior and rule
semantics.
* **Bug Fixes**
* Improved API rule validation coverage for missing list-type fields.
* **Tests**
* Added SDN round-trip, fuzzing, RBAC, and comprehensive REST
create/get/list/update/delete/watch/resource-version tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
a349c753ad |
fix(kubernetes): make node group resources optional and prefer them over instanceType
A node group is sized either by instanceType or by explicit resources, but the generated values schema required resources on every node group. A node group with instanceType and no resources — or an empty resources dropped during typed-CR to HelmRelease translation — failed schema validation with "missing property 'resources'" even though sizing was fully specified. Mark resources optional in the node group definition and regenerate the schema, API types, README, and ApplicationDefinition. The chart template emits domain.cpu/memory only when resources is present, so a node group sized purely by instanceType now passes validation and installs. When a node group sets both instanceType and explicit resources, explicit resources take precedence: the instancetype is omitted from the rendered VM and the per-group instancetype lookup is skipped, so KubeVirt accepts the explicit domain.cpu/memory with no instancetype to conflict with. instanceType stays required and defaulted (no schema change), so the console's required instanceType dropdown keeps working — sending the default instanceType plus explicit resources simply sizes the VM by resources. This replaces an earlier render-time hard failure on the both-set case, which was unreachable to clear from the dashboard (the required dropdown cannot be emptied). Explicit resources are all-or-nothing alongside an instanceType: a partial spec (only cpu, or only memory) on a group that carries a non-empty instanceType would emit one domain override next to the instancetype — which KubeVirt rejects — so it now fails at render time with a clear error. Partial resources on a group without an instanceType (used to tune kubelet reservations) are unaffected. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
31c15b282f |
feat(network): mechanism-agnostic LoadBalancer exposure (ExposureClass + ServiceExposure) (#3081)
## What this PR does Lays the foundation for migrating tenant LoadBalancer exposure off the deprecated `Service.spec.externalIPs` (KEP-5707) and, eventually, off MetalLB onto Cilium-native announcement — the mechanism-agnostic shape prescribed when the earlier chart-side approach (#2468) was reverted in #2603. It adds a new `network.cozystack.io/v1alpha1` API group and a controller that translates intent into whichever LoadBalancer backend the cluster admin configured: - `ExposureClass` (cluster-scoped, admin): binds a logical class name to a backend (`externalIPs` | `metallb` | `cilium` | `robotlb`) and its VIP address scope — the StorageClass/IngressClass pattern. - `ServiceExposure` (namespaced, tenant intent): "expose this Service via class X"; status reports the resolved backend and assigned IPs. - A `serviceexposure` controller renders the backend's pool + announcer once per class (MetalLB `IPAddressPool` + `L2Advertisement`; Cilium `CiliumLoadBalancerIPPool` + `CiliumL2AnnouncementPolicy`), scoped to the namespaces that opted in. Backends are pluggable behind a small Go interface, so adding BGP or another cloud LB is one implementation, not an API change. The controller never mutates the target Service — the chart still owns its shape. Platform wiring: `publishing.exposureClass.*` renders the admin `ExposureClass` and the host ingress publishes through it. The default (no class set, `externalIPs`) leaves every existing cluster's behaviour byte-for-byte unchanged; the MetalLB→Cilium announcer migration becomes a single `ExposureClass.spec.backend` change. This is the foundation only — Part of #2586. Converting per-app exposure, flipping Cilium `l2announcements` on, changing the default, and removing MetalLB are deferred to later phases. ### Release note ```release-note feat(network): add ExposureClass and ServiceExposure for mechanism-agnostic LoadBalancer exposure. Opt-in foundation; the default externalIPs publishing path is unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added new `ExposureClass` and `ServiceExposure` custom resources with support for multiple backends (externalIPs, MetalLB, Cilium, RobotLB), including optional L2 and address configuration. * Added Helm wiring and templates to enable service/ingress publishing via `ServiceExposure`, selecting a configured (or default) `ExposureClass`. * **Bug Fixes** * Improved reconciliation, finalizer-based cleanup, and status updates for readiness and assigned IP reporting, including safer handling when backends or managed resources change. * **Tests** * Added end-to-end and unit test coverage for backend rendering, readiness, and cleanup flows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |