mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-03 10:37:21 +09:00
kafka-multi-instance
5813
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alexey Artamonov
|
7ab524d97b |
feat(kafka): allow multiple KRaft Kafka clusters per namespace
Fresh KRaft clusters no longer need to be alone in a namespace. Their node pools are now release-scoped (`<release>-broker`, `<release>-controller`) instead of the fixed `kafka`/`controller`, which is unique per namespace (KafkaNodePool object names are namespace-unique) and lets any number of Kafka instances coexist — every pod/PVC/Service is already prefixed by the unique cluster name. The fixed `kafka` name is still required to adopt an existing ZooKeeper cluster's `<cluster>-kafka-N` brokers and data in place during migration, and that name is namespace-unique — so only one ZooKeeper->KRaft migration can run per namespace. The broker pool name is resolved by kafka.brokerPoolName: "kafka" for a cluster that already migrated or is a classic ZK cluster about to migrate, otherwise "<release>-broker". A second migration into a namespace that already holds a "kafka" pool owned by another cluster fails the render with a clear message; fresh KRaft clusters are never affected. This replaces the previous blanket "one Kafka per namespace" guard (kafka.assertSingleInstance), which over-restricted the fresh case. The migration hook and workload monitors follow the same naming; unit tests updated. Verified on a live cluster: two fresh single-node KRaft clusters (mi-a, mi-b) installed side by side in one namespace came up with distinct node pools, pods and PVCs. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Andrey KolkovandClaude Opus 4.8
|
6a968e0058 |
test(kafka): drop source-grepping migration-timeout drift-guard
The removed bats file grepped two static literals in two separate packages (helm-install-timeout in system/kafka-rd, activeDeadlineSeconds in apps/kafka) and asserted an ordering between them. Substring matching cannot verify the contract it targets: the real invariant is between the running migration Job and the release timeout helm-controller actually enforces on the pre-upgrade hook, not between two YAML literals in the tree. It also greened only for the minute-format literal and errored on any legitimate reformat (e.g. "1h" / "2700s"). The timeout > Job-deadline relationship is already documented inline at both values. Its machine-checkable home is an e2e that upgrades an existing release through the HelmRelease pre-upgrade hook and confirms a real migration completes inside the release timeout (the chainsaw suite drives migration via kubectl, bypassing that path). Signed-off-by: Andrey Kolkov <androndo@gmail.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alexey Artamonov
|
d737d1a6f0 |
fix(kafka): replace dead ZooKeeper panel with KRaft controller metrics
The strimzi-kafka Grafana dashboard still had a 'ZK request latency' panel (kafka_server_zookeeperclientmetrics_zookeeperrequestlatencyms), which goes permanently empty after the ZooKeeper->KRaft migration. Replace it with a KRaft controller commit-latency panel (kafka_server_raftmetrics_commit_latency_avg, scraped from the controller pods). Metric names verified against a live KRaft cluster. The PVC-exclusion filter that references zookeeper stays valid and is left as is. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
a8863ee145 |
fix(kafka): harden migration hook and isolate the migration e2e
Addresses the second review round on the ZK->KRaft migration. Migration hook: - Fully qualify the Strimzi resources as kafkas.kafka.strimzi.io / kafkanodepools.kafka.strimzi.io. The bare plural "kafkas" also exists in this chart's apps.cozystack.io group, so a bare name resolves by per-cluster discovery priority and can hit the tenant kind — which the hook Role does not grant, failing the pre-upgrade hook deterministically. - Classify NotFound structurally via `kubectl get --ignore-not-found` instead of substring-matching stderr, so an unrelated "... not found" error can no longer be mistaken for a fresh install and permanently skip a live ZooKeeper cluster. This also drops the jq dependency. - Treat a KRaft-native / already-migrated CR (annotated strimzi.io/kraft=enabled) as done, never driving it backwards into kraft=migration. - Keep only `before-hook-creation` on the Job's hook-delete-policy so a failed attempt's pod logs survive for a post-mortem until the next run. - Route the kubectl image through cozy-lib.image (cluster registry mirror) and pin it by digest (clastix/kubectl:v1.32@sha256:...), matching the other cozystack hooks, instead of an unpinned docker.io tag. Controller quorum: - Fix the controller node-pool replica count at creation instead of deriving it from kafka.replicas on every reconcile. Strimzi 0.45 cannot scale a controller pool, so a day-2 edit of kafka.replicas (e.g. 1 -> 3) must not flip the quorum; the helper now keeps the existing "controller" pool's replica count when it is present. Docs: - Document that migration to KRaft is one-way: neither a version-field downgrade nor a pre-KRaft chart rollback is supported once migrated (Strimzi has no KRaft->ZooKeeper rollback after completion). E2E: - Run the ZK->KRaft chainsaw migration in its own namespace instead of the shared tenant-test. It raw-applies Strimzi resources plus a namespace-wide LimitRange and leaves deleteClaim:false PVCs, which polluted the sibling kafka suite. Raise the seed readiness timeout for CI headroom. Validated against a live cluster: migration adopts the broker PVC in place and reaches KRaft under a 128Mi LimitRange, in isolation. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
37bfd448ca |
fix(kafka): set resources on the migration node pools
The ZK->KRaft pre-upgrade hook created the adopting broker ("kafka") and
"controller" KafkaNodePools without a resources block. Those hook-created pools
govern the Kafka pods for the entire migration window — Helm applies the
resourced steady-state kafkanodepools.yaml only after the pre-upgrade hook
finishes. In a tenant namespace the LimitRange defaults every container to
128Mi, so the migration's controller (and rolled broker) JVMs OOM, never reach
KRaftPostMigration, and the hook times out — wedging the migration and thrashing
it on every reconcile.
Give both hook pools the same resources the steady-state pools get
(cozy-lib.resources.defaultingSanitize from kafka.resourcesPreset/resources).
This also avoids a redundant second broker roll (resource-less -> resourced)
after the hook completes.
Tests: a helm-unittest assertion that both migration pools render a resources
block; the Chainsaw ZK->KRaft suite now seeds under a 128Mi LimitRange (and
gives the seed cluster explicit resources), reproducing the tenant-quota
condition, and asserts the migration still reaches KRaft with the broker PVC
adopted. Validated green against a live cluster under the LimitRange.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
|
||
|
Alexey Artamonov
|
42e0c3dd84 |
fix(kafka): enforce one Kafka per namespace; simplify node pool names
Follow-up to the KRaft-migration review. The broker KafkaNodePool must be named exactly "kafka" so Strimzi adopts an existing ZooKeeper cluster's <cluster>-kafka-N brokers and data in place during migration, but KafkaNodePool object names are unique within a namespace — so two Kafka clusters cannot both own a "kafka" pool there. This is an upstream constraint and Strimzi's own guidance is one Kafka cluster per namespace (strimzi/strimzi-kafka-operator discussions/11120). Adopt that model instead of the earlier fresh-vs-migrated conditional naming: - New kafka.assertSingleInstance helper fails the render (with a clear message and the upstream reference) if the namespace already holds another Kafka cluster. Reconciles/upgrades of the same release are allowed; lookup returns nothing during `helm template`, so unit tests never trip it. - Node pools are now named "kafka" (broker) and "controller" unconditionally, which also adopts existing brokers on migration and yields clean <cluster>-kafka-N / <cluster>-controller-N names. The fresh-vs-migrated brokerPoolName helper is removed. - WorkloadMonitor pool-name selectors and the migration hook updated to match; unit tests updated for the fixed pool names. - README documents the one-Kafka-per-namespace rule and links the Strimzi discussion. The ZK->KRaft Chainsaw migration suite (seed ZooKeeper cluster, migrate, assert the original broker PVC survives and no wrongly-named PVC is created, cluster reaches KRaft) was run green against a live cluster. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
e00c8f4a30 |
fix(kafka): address KRaft-migration review — broker pool adoption, hook safety, tests
Addresses the branch review on the ZK->KRaft migration path. CRITICAL — broker node pool adoption. Strimzi derives broker/PVC names as <cluster>-<pool>-<id>; a pre-node-pool ZooKeeper cluster has brokers <cluster>-kafka-N, so the adopting broker pool must be named exactly "kafka" or the operator provisions fresh empty brokers and orphans the tenant's data. A new kafka.brokerPoolName helper resolves the broker pool to "kafka" for an existing (migrating) cluster and keeps the release-scoped "<release>-broker" for fresh installs, so multiple Kafka instances still coexist in one namespace. The migration hook creates the pool as "kafka" and fails closed if a "kafka" pool already belongs to another cluster (node pool names are namespace-unique, so in-place migration of co-namespaced ZK clusters is unsupported). Wired through kafkanodepools.yaml, workloadmonitor.yaml and the hook. MAJOR — migration robustness. - Raise the HelmRelease timeout: the migration Job can run tens of minutes but the release timeout defaults to 10m, so helm-controller aborted and thrashed the pre-upgrade hook. Add release.cozystack.io/helm-install-timeout=45m on the kafka ApplicationDefinition, above the Job's activeDeadlineSeconds (2400s). - Idempotent migration: do not re-issue kraft=migration once the cluster is past the ZooKeeper phase (an invalid backward transition Strimzi rejects on retry). - Fail closed on the decision gates: distinguish a genuine NotFound (skip) from any other kubectl error (abort), and never treat an empty kafkaMetadataState on an existing CR as "migrated". MINOR — version downgrade documented as unsupported; integer (not lexical) compare in the version gate; deleteClaim/PVC-retention and the one-per-namespace migration limitation documented in the README; docs/storage-immutability.md updated for KRaft. Tests — helm-unittest coverage for the broker pool name, the hook guards and the idempotent/fail-closed logic; a bats check for the helm-install-timeout budget ordering; and a Chainsaw ZK->KRaft migration suite that seeds a ZooKeeper cluster, drives the migration, and asserts the original broker PVC survives (not recreated) and the cluster reaches KRaft. The migration sequence and the Chainsaw assertions were validated against a live cluster. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
457282caf8 |
fix(kafka): address branch-review — restore CA cert secrets, guard KRaft finalize
Follow-up to the KRaft migration addressing the branch review. - kafka-rd cozyrds: restore the two public CA certs in secrets.include (clients-ca-cert + cluster-ca-cert). The rebase had collapsed them to the bare clients-ca, which is Strimzi's clients-CA *private key* secret — a private-key exposure to the tenant and a TLS regression (clients need cluster-ca-cert to trust the broker listener). Covered by the existing hack/check-kafka-rd-secrets.bats. - migration-hook: guard the KRaft finalize step. The wait loop only exited via break on success and fell through to unconditionally apply strimzi.io/kraft=enabled on timeout — unsafe and irreversible if the cluster is still in ZooKeeper/KRaftMigration. Add a migration_ready flag and exit 1 when the safe state (KRaftPostMigration+) is not reached, leaving the deployed-version ConfigMap unstamped so Helm retries. New matchRegex assertions in migration-hook_test.yaml cover the guard. - migration-hook: raise activeDeadlineSeconds 1800 -> 3600 so a legitimately-progressing migration is not SIGKILLed before the in-script waits and error paths can run. - Extract the duplicated controller-quorum logic into a shared kafka.controllerReplicas helper (used by kafkanodepools, workloadmonitor and migration-hook) so the three stay in lockstep. - Reflow the README migration section to one line per paragraph per the repo prose rule. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Matthieu
|
b5b9525342 |
feat(kafka): migrate from ZooKeeper to KRaft mode with automatic migration
Replace ZooKeeper-based Kafka with KRaft (Kafka Raft) metadata mode. Strimzi 0.45 is the last line that still supports ZooKeeper, so this prepares the transition before ZooKeeper support is removed upstream. Chart changes: - The Kafka CR always renders strimzi.io/kraft=enabled with node pools and no spec.zookeeper block. Storage, replicas and resources move to separate broker + controller KafkaNodePools (a homogeneous separate-pool layout works for both fresh installs and migrations; Strimzi 0.45 does not support migration into a combined pool). - Kafka version is now user-configurable via the standard files/versions.yaml pattern (v3.9 default, v3.8 available); appVersion bumped to 3.9.0. - WorkloadMonitors track the broker and controller pools (no zookeeper monitor); the dashboard RBAC grants the controller monitor and drops the zookeeper one. Migration of existing ZooKeeper clusters is handled by a Helm pre-upgrade Job (templates/migration-hook.yaml) gated by a <release>-kafka-deployed-version ConfigMap, following the established seaweedfs/etcd hook pattern. The Job creates the broker + controller pools with Helm ownership labels, drives the Strimzi ZK->KRaft state machine (node-pools=enabled, kraft=migration -> kraft=enabled once KRaftPostMigration is reached) and is skipped once the ConfigMap records version >= "1". Testing: - Helm unit tests cover the Kafka CR, node pools, migration hook, metrics, topics, version map, dashboard RBAC and workload monitors. - The e2e suite is ported to the current Chainsaw framework (hack/e2e-chainsaw/kafka): it asserts the KRaft topology (both node pools, kafkaMetadataState=KRaft), the absence of ZooKeeper, and bootstrap endpoints. Rebased onto current main: TLS listener support, entity-operator resource guards, x-cozystack-options schema annotations and the WorkloadMonitor replica-path fix are all preserved and reconciled with KRaft. Signed-off-by: Matthieu <matthieu@hidora.com> Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Andrei Kvapil
|
eef95f456d |
docs: add CNCF General Technical Review (GTR) questionnaire (#3572)
## What this PR does Adds the CNCF General Technical Review (GTR) questionnaire for Cozystack, completed for the Sandbox → Incubation Due Diligence (tracks cncf/toc#1916). It answers the Day 0 / Day 1 / Day 2 questions from the CNCF TOC project-reviews template, with versioned links to the v1.6 documentation and to the repo's governance and security files. **Status: DRAFT / work in progress.** The questionnaire itself is a living document per the template's guidance. Everything repository evidence settles has been answered and verified against the tree; the banner at the top of the file names the seven substantive gaps that need measurement, publication or a maintainer decision — load/scale results and tested maximums, published SLOs/SLIs, measured platform baseline overhead, the upgrade→downgrade CI matrix, whether upgrades trigger Talos node reboots, the security-team rotation and diversity policy, and publication of the CVE-scanning pipeline's policy documents. Version-specific links should be pinned to the submitted tag before final submission. ## Changes beyond the new file — please read this part Review feedback found two factual errors that originate **outside** this questionnaire, so fixing them only in the GTR would have left a reviewer clicking through to the contradiction. Three existing files are therefore touched: - **`SECURITY.md` — supported-versions table. This is a policy change and needs a policy review, not a wording check.** The table was dated "As of March 17, 2026" and named `v1.1.x` as the current stable line, five minors behind `v1.6.1`. It is rewritten to be keyed to **observable patch activity on a line** — supported means a patch release within the last 6 months, limited support 6 to 18 months, end of life beyond that — so an adopter can determine their own status from the Releases page and the table cannot drift again. Two intermediate versions of this PR got the policy wrong in opposite directions and both are reverted: one narrowed support to the latest two minors, which contradicts practice since several `1.x` lines receive patch releases concurrently; the other removed the end-of-life category altogether and extended discretionary backports across all of `0.x`. What lands here restores an end-of-life category, puts the `v0.41` cutoff back with its last-patch date, and states explicitly that the six- and eighteen-month marks describe current behaviour rather than committing the project to patch any line for that long. `@kvaps`, as a CODEOWNER for this file: please review the **policy**. If the project would rather state a guaranteed backport window, or keep support entirely at maintainer discretion with no stated marks, that is a governance decision and this PR deliberately does not make it. - **`SECURITY.md` — automated security analysis.** It said "two scanners run continuously" and listed CodeQL and Scorecard. It now also describes `zizmor` (GitHub Actions SAST, gating on PRs), the organization-wide Trivy CVE-scanning pipeline, and Renovate's actual coverage. It also records that CodeQL is **not** currently among the required status checks on `main` — those are `pre-commit` and `E2E Tests` — so the CodeQL gate is review convention rather than branch protection today. - **`CONTRIBUTOR_LADDER.md`** — Reviewer nomination described adding a username to an `OWNERS` file. No `OWNERS` file exists in this repository; line 158 of the same document already refers to `CODEOWNERS`. Corrected to name the files that exist. This does not change the process. - **`docs/security/self-assessment.md`** — brought in step with the above on the same three points (CodeQL's gate status, the Trivy pipeline and its limitations, Renovate's real coverage), so the GTR, `SECURITY.md` and the self-assessment agree rather than contradict each other in front of the TOC. ## Two things for `@kvaps` to decide, not to approve as wording 1. **Which release lines are supported — and the evidence points two ways.** The six-month activity test in the rewritten table classifies every `1.x` line from `1.0` to `1.6` as supported, because each has had a patch release inside six months (`v1.1.7` as recently as 2026-06-03, `v1.0.7` on 2026-03-30). But *recent* activity looks like three lines, not seven: of the sixteen most recent backport branches, fifteen target `release-1.4`, `release-1.5` or `release-1.6`, and only `1.4`, `1.5` and `1.6` have shipped a release since 2026-07-03. Older backport branches for `release-1.2` and `release-1.3` exist but are months old — and since branches are never deleted, their existence proves nothing about the present, which is the same trap the table's own lead-in now warns about. So the concrete question is: **is the intended set `1.4`/`1.5`/`1.6` supported with `1.0`–`1.3` on limited support, or is it genuinely all seven?** If a critical fix would today be backported to `v1.1`, the wide table is right. If it would not, the six-month window is the wrong instrument and should be shortened. I have deliberately not chosen — either answer is defensible and it is a maintainer call, not a documentation one. **A third file turns on the same answer.** [`docs/security/incident-response.md`](https://github.com/cozystack/cozystack/blob/main/docs/security/incident-response.md) step 5 says a fix ships "across all affected supported release lines", pointing at the `SECURITY.md` table — so widening that table to seven lines widens this commitment with it. This PR does not touch that file and does not assert the claim, but the two should move together: if the answer is "all seven", incident response is right and `SECURITY.md` should stop hedging; if it is "1.4–1.6", the table, the window and that sentence all change at once. 2. **Whether there is a backport rule at all.** No project document, public or private, describes how a fix is chosen for backporting to a given supported line. An intermediate version of this PR asserted one (by severity, default-config reachability and adopter count); that was invented and has been removed. The file now says the decision is taken per fix and the rule is not documented. If a rule exists informally, writing it down would answer a question the seven-supported-lines table otherwise raises. ## Gaps this documents rather than fixes Recorded here because they surfaced while verifying the questionnaire's claims and each is a real finding: - CodeQL is not a required status check, though two documents described it as gating merges. - There is no top-level `NOTICE` file, for a distribution bundling on the order of fifty upstream components. - No default Kubernetes audit policy is shipped; audit logging is opt-in with the policy left to the adopter. - Release artifacts are not signed and no issue tracks adding signing; build provenance is explicitly disabled (`--provenance=false`); SBOM generation is implemented but defaulted off. - The CVE-scanning pipeline lives in a repository that returns 404 to non-members. Its aggregate reporting is now published (below); its policy documents and scanning scripts, which hold no finding data, are not, so the method behind the numbers is still not independently auditable. - `nightly.yaml` has been failing since 2026-08-02 and `e2e-tag.yaml` failed around the v1.6.1 release; current CI health is disclosed next to the workflow links. ## New: published security reporting, and a backlog audit that changes the SCA answer `docs/security/reports/` now holds monthly vulnerability-handling reports plus a [backlog reconciliation](https://github.com/cozystack/cozystack/blob/main/docs/security/reports/reconciliation-2026-08.md). The reconciliation exists because the CVE-scanning pipeline has no write-back from this repository, so it has never marked a finding as fixed and its counts were misleading in both directions. Audited against what actually shipped: - Remediation did happen — a clear majority of confirmed findings relate to a component whose in-tree version now carries the fix — but as **one wave in June 2026**, not a steady stream. The changes that did most of the work name no CVE at all: re-vendoring the dashboard console removed the vulnerable npm dependencies rather than upgrading them; refreshing the Alpine bases cleared fourteen; bringing the Go toolchain current satisfies all 62 `stdlib` findings. - **57% of the backlog was never applicable or cannot be assessed.** The largest block is packages containing no executable vulnerable code — chiefly kernel headers, since a container never provides a running kernel, which is the same reasoning the maintainers already applied 80 times to `kernel-headers` and never to `linux-libc-dev`. Reachable exposure in a stock install is an estimated 2–5% of the tracker. - **Four component bumps are available upstream and not taken**: `nats-server`, the CoreDNS image, OpenBao, `fast-uri`. - Seven defects in the pipeline itself, with a write-back specification to fix the root cause. Two issues counted as findings are actually quarterly-review tracking tasks. The disclosure rule moved in the open direction: findings with no upstream fix will be named with mitigation guidance, because under the previous rule that class — a fifth of the backlog — stayed secret permanently, and it is exactly where an adopter needs to know a mitigation is the only control. Conversely, remediation is no longer attributed to individual months, because crossing a count of still-open findings with one month's publicly enumerable component bumps narrows the search for unpatched exposure more than a bare count does. Several findings warrant maintainer attention ahead of the backlog and are being raised privately rather than named here. ### Downstream repositories - [x] No downstream repository is affected by this change (docs-only; no code, API, manifest or packaging changes). ### Release note ```release-note docs: add CNCF General Technical Review (GTR) questionnaire for the Sandbox → Incubation review ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Documentation** - Added a comprehensive technical review covering planning, installation, operations, architecture, security, and troubleshooting. - Added April–August 2026 security reports and a complete CVE backlog reconciliation. - Expanded security self-assessment, scanning coverage, reporting guidance, disclosure rules, and audit limitations. - Clarified supported-version policies, release backport decisions, and private vulnerability-reporting procedures. - **Process Updates** - Updated reviewer nomination requirements to use current ownership and maintainer records. - Clarified severity- and risk-based backport decisions for coordinated security releases. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
scooby87
|
9dd2841c74 |
feat(clickhouse): add configurable version parameter (#3476)
## What this PR does The ClickHouse server and Keeper images were hardcoded to `24.9.2.42` in the chart templates, so the chart exposed no top-level `version` values key. The Cozystack API returns a managed database's engine version by passing the HelmRelease's `spec.values.version` straight through into the Application object, so ClickHouse instances carried no `spec.version` in the API — unlike postgres, mariadb, mongodb, opensearch, rabbitmq and redis, which all ship a `version` parameter. This closes the ClickHouse half of the gap tracked in #1246. This adds the same version-selection scaffolding the other managed-database charts already use: - `files/versions.yaml` maps a `major.minor` key to the full image tag, shared by the ClickHouse server and Keeper images (both are pinned to one version). - `templates/_versions.tpl` (`clickhouse.versionMap`) resolves the tag and fails the render on an unsupported version. - `values.yaml` gains a `version` enum (`v25.8`, `v25.3`, `v24.9`), consumed by `templates/clickhouse.yaml` and `templates/chkeeper.yaml`. - `hack/update-versions.sh` + `make update` refresh the map from Docker Hub, keeping only tags published for *both* the server and Keeper images and preserving the current default so a regeneration never bumps the major of existing deployments. The default stays `v24.9` → `24.9.2.42`, so existing installations render byte-for-byte identically and are never upgraded to a new major implicitly. The generated `values.schema.json`, `README.md`, the Go API type and the `clickhouse-rd` `openAPISchema` are regenerated with `cozyvalues-gen` v1.6.0. A new `tests/version_test.yaml` helm-unittest suite covers the server/Keeper mapping for each version and the schema-level rejection of an unsupported value. ### Downstream repositories The diff changes `packages/apps/clickhouse/values.schema.json` by adding a new `version` field with an `enum`. Per the trigger map, this reaches `terraform-provider-cozystack`, whose ClickHouse resource is hand-written and would need a matching `version` attribute with a `stringvalidator.OneOf` list and its expand/flatten pair. There is no existing PR or issue there for it yet. This is left for a maintainer decision rather than a speculative cross-repo PR, so the box below is intentionally left unticked and flagged here. No other downstream repository is reached (the website reference page is regenerated from `README.md` by the release docs bot; ClickHouse is already in its app list). - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: needs a `version` attribute on the ClickHouse resource (maintainer decision — see note above) - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(clickhouse): add a configurable `version` parameter (v25.8, v25.3, v24.9; default v24.9) so the deployed ClickHouse version is selectable and returned through the API. Existing installations are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a configurable ClickHouse version setting for both server and Keeper images, defaulting to `v24.9`. * Supported versions are `v25.8`, `v25.3`, and `v24.9`, with compatible image patches selected automatically. * **Documentation** * Documented the version setting, default, and supported values. * **Validation** * Unsupported versions are rejected with an error listing allowed options. * **Tests** * Added coverage for version mapping, defaults, invalid values, and update behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
scooby87
|
94614c021a |
fix(etcd): spread members across nodes with default soft pod anti-affinity (#3962)
## What this PR does The `etcd` chart hardcoded only a **soft** `topologySpreadConstraints` (`whenUnsatisfiable: ScheduleAnyway`) and never rendered `spec.affinity`, even though the `EtcdCluster` CRD supports it. On small clusters the scheduler therefore co-locates multiple members on one node; losing that node takes out quorum (or the whole cluster). We hit this in production: a single node reboot left several 3-member etcd clusters with 2–3 members on the downed node, and one tenant's control-plane etcd lost quorum until the node came back. This renders `EtcdCluster.spec.affinity` from a new `affinity` value: - **Default (value left empty):** the chart applies a **soft** (preferred) `podAntiAffinity` on `kubernetes.io/hostname`, so members spread across nodes on a best-effort basis. Soft on purpose — a hard rule would leave a member `Pending` when replicas exceed schedulable nodes, or when node-local storage pins a member to a node, so it is safe to default on. - **Override:** set `affinity` to any rule — e.g. a hard `requiredDuringSchedulingIgnoredDuringExecution` podAntiAffinity — to forbid co-location outright on clusters with enough nodes and non-node-pinned storage. A supplied value replaces the default cleanly. The existing `topologySpreadConstraints` block (and its `_cluster.scheduling.globalAppTopologySpreadConstraints` override) is unchanged. Generated artifacts (`values.schema.json`, `README.md`, and the embedded `openAPISchema` in `etcd-rd`) were regenerated with `make generate`; `make test` passes (18/18). ### Screenshots n/a — no UI changes. ### Downstream repositories Walked the trigger map against the diff: this only adds one values key to an app chart. No package is added/renamed/removed, `kind`/`plural` are unchanged, no installer values or release assets or variants change, and `README.md` is regenerated here (the website bot regenerates reference pages from it on release). No manual downstream follow-up is required. - [x] No downstream repository is affected by this change ### Release note ```release-note fix(etcd): etcd clusters now spread their members across nodes by default via a soft podAntiAffinity, and expose an `affinity` value so operators can set a hard rule; previously a single node could host a quorum of members and take out the cluster when it failed. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable pod affinity rules for etcd deployments. * By default, etcd members use soft anti-affinity to spread across hosts without blocking scheduling. * Custom affinity settings can override the default behavior. * Exposed affinity configuration in the etcd application schema and dashboard. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
scooby87
|
6bb946fa5b |
fix(postgres-operator): bump CloudNativePG to 1.30.0 for the Cluster scale-subresource selector (#3951)
## What this PR does Bumps the vendored `cloudnative-pg` chart from 0.27.1 to 0.29.0 (operator + CRDs from 1.28.1-patched-to-1.28.2 up to 1.30.0) and drops the hand-maintained `patches/cloudnative-pg-1.28.2.patch` entirely — 0.29.0 is a published chart, so the version-alignment patch is no longer needed. **Why this is required.** Database read-replica autoscaling (community proposal `design-proposals/database-horizontal-autoscaling`) drives the CNPG `Cluster` scale subresource with a stock HPA (via KEDA). That only works when the subresource serves `status.selector`, which [cloudnative-pg#8996](https://github.com/cloudnative-pg/cloudnative-pg/pull/8996) added and shipped in CNPG 1.28.4 / 1.29.2 / 1.30.0. cozystack shipped 1.28.2, whose scale subresource has no selector, so the HPA fails with `InvalidSelector`. **Why 1.30.0 and not 1.28.4.** The floor is ≥ 1.28.4, but the upstream chart repo publishes no chart for 1.28.4 (the 1.28 line stops at chart 0.27.1 / appVersion 1.28.1), so 1.28.4 could only be reached by hand-patching the 0.27.1 chart's operator image and CRDs up to the backport. A live PoC on a dev cluster proved that path is broken: the 1.28.4 operator running under the 0.27.1 (1.28.1) chart templates crash-loops on a missing RBAC grant it needs but the older chart does not provide — `endpointslices.discovery.k8s.io is forbidden: ... cannot list resource "endpointslices"` — so `status.selector` is never populated and the HPA still cannot scale. 1.30.0 (chart 0.29.0) is the only published chart that carries the selector together with matching RBAC/CRDs/templates, and the 1.28 line is EOL (2026-06-30), so 1.30.0 also keeps us on a supported release. cozystack uses the barman-cloud plugin (not native barman, removed in 1.31) and no CNPG Poolers, so the 1.30 deprecations and the known pooler bug do not apply. **Validation on a dev cluster (CNPG 1.30.0).** Operator rolls out clean (no crashloop, the `endpointslices` RBAC error is gone); all existing `Cluster` objects return to Ready; `/scale` now returns a populated `status.selector` (`cnpg.io/cluster=<name>,cnpg.io/podRole=instance`); an HPA pointed at a `Cluster` no longer reports `InvalidSelector` (`AbleToScale=True, SucceededGetScale`); psql connectivity and streaming replication verified. The version-guard unittest is updated to 1.30.0 and now also asserts the clusters CRD scale subresource carries `labelSelectorPath: .status.selector`. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff, file by file. The change touches only the vendored `cloudnative-pg` operator chart (CRDs, RBAC, values), the package `Makefile`, and the version-guard test. The `packages/apps/postgres` app and its `version` enums are unchanged, `packages/core/platform/values.yaml` is unchanged, and the `clusters.postgresql.cnpg.io` CRD is not one the terraform provider types by hand (it travels as opaque JSON). No downstream repository restates anything this diff changes. - [x] No downstream repository is affected by this change ### Release note ```release-note fix(postgres-operator): bump CloudNativePG to 1.30.0 so the Cluster scale subresource serves status.selector, enabling HPA-driven read-replica autoscaling (previously the HPA failed with InvalidSelector on 1.28.2) ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Upgraded the CloudNativePG operator and Helm chart to 1.30.0/0.29.0. * Added expanded image catalogs, pooler settings, pod selectors, scheduling options, monitoring TLS, and LZ4 backup compression. * Improved permissions for database roles, endpoint slices, and lease monitoring. * **Bug Fixes** * Strengthened configuration validation and removed the deprecated workload reference option. * Updated compatibility requirements to Kubernetes 1.29 or newer. * **Documentation** * Expanded installation, configuration, verification, upgrade, and uninstall guidance. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
9858fb63ac |
test(etcd): cover default and overridden affinity rendering
The affinity value added in this branch introduced a two-branch conditional in templates/etcd-cluster.yaml (default soft podAntiAffinity vs. a supplied value) with no coverage. Add a helm unittest suite pinning both branches: the default preferred per-host rule, and that a supplied affinity replaces the default outright (no preferred rule leaks through). Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io> |
||
|
Timofei Larkin
|
f5b957ee27 |
feat(kubernetes)!: manage worker pools as KubernetesNodes, adopt on upgrade (Phase 2, breaking) (#3315)
## What this PR does Phase 2 of the Kubernetes-app split ([cozystack/community#8](https://github.com/cozystack/community/pull/8)) — the **breaking** half. Stacked on #3314 (review that first). Removes worker node pools from the `kubernetes` chart (they moved to `kubernetes-nodes` in #3314) and adopts existing pools on upgrade: - `kubernetes` chart is now **control-plane only**: drops the pool `range` block, the `talos-reconcile` Job + RBAC (deleted; moved to `kubernetes-nodes`), the preserve-old-KMT block, the `kubernetes.nodeGroups` helper, and the per-group dashboard RBAC. - **`spec.nodeGroups` is removed from the `Kubernetes` CR** (BREAKING). Worker pools are now separate `KubernetesNodes` resources. - **Migration 54** (bumps `targetVersion` to 55): pre-upgrade hook that, for every `kubernetes-<cluster>` release, creates one `kubernetes-nodes-<cluster>-<pool>` HelmRelease and re-annotates the existing worker objects onto it with `helm.sh/resource-policy=keep`, so the now control-plane-only parent upgrade does not prune running workers. The default `md0` (implicit when `nodeGroups` was empty) is materialised explicitly. Idempotent. Genuine infrastructure/read failures fail closed (`exit 1`, retried on the next upgrade); hostile stored shapes (a pool name over Helm's 53-char release-name cap, an RFC-1123-invalid or whitespace/slash map key, a non-object `nodeGroups`) are warned, pinned, and skipped rather than deadlocking the entire fleet's upgrade. - The `ingressNginx` addon no longer auto-provisions a node (the implicit `md0` default is gone). If enabled without a pool carrying `roles: [ingress-nginx]`, the controller DaemonSet simply schedules nothing (degraded, not failed) — the requirement is documented in the chart NOTES.txt rather than enforced at render time. Because the child chart renders the pool objects **byte-identically** to the old parent render (golden parity in #3314; the KubevirtMachineTemplate content-hash name is preserved), adoption does not churn live worker VMs. ### Migration number `54` is contiguous with main (latest is `53`) and `targetVersion` is `55`. The `52`/#3201 collision noted in earlier revisions has been resolved: this branch was rebased and merged past it, so the numbering is contiguous and `run-migrations.sh` has no gap to hard-fail on. ### Not yet done (blocking draft → ready) - [ ] **dev e2e upgrade validation** (`cozystack-pr-test`): upgrade a custom-`nodeGroups` cluster and a default-`md0` cluster; assert zero worker-VM churn (KMT hash, Machine uid/creationTimestamp stable) and that the adopted child HRs reconcile. - [ ] downstream trigger-map walk (website docs for the `nodeGroups` → `KubernetesNodes` change; terraform-provider for the removed API field). ### Downstream repositories Deliberately unticked (draft) — the breaking API change affects docs and the terraform provider; follow-ups to be filed before ready. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: ```release-note Worker node pools of a managed Kubernetes cluster are now managed as separate `KubernetesNodes` resources. `spec.nodeGroups` on the `Kubernetes` CR is removed; existing pools are adopted automatically on upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Worker pools are now separate `KubernetesNodes` resources with their own lifecycle. * Added a render-time compatibility guard to prevent worker pools from running ahead of the parent control-plane minor version. * Improved upgrade/uninstall behavior to better preserve already-adopted worker objects. * **Breaking Changes** * Removed per-pool `nodeGroups` and worker node health settings from the `Kubernetes` chart/CR; pool configuration now lives in `KubernetesNodes`. * Ingress requires at least one `KubernetesNodes` pool with `roles: [ingress-nginx]`. * `cluster` and `storageClass` are immutable after creation; pool `version` may lag but must not be ahead of the parent minor. * **Documentation** * Updated READMEs, notes, and schemas to reflect the `KubernetesNodes` split and the immutability/version rules. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Alexey Artamonov
|
e707475f96 |
fix(clickhouse): fetch tags via skopeo and fail closed on a missing pin
Address review of the version generator (hack/update-versions.sh): - Fetch tags with `skopeo list-tags` (registry v2 /tags/list) instead of the Docker Hub v2 API, whose anonymous pagination caps at 1000 entries and then returns an error page with no `.results`, silently truncating the list so the frozen 24.9.2.42 fell outside the window and the run failed. skopeo is already in build-deps and is how the sibling generators (postgres, redis, mongodb, opensearch, kubernetes) fetch. - Make a frozen default tag missing from the registry a hard error instead of silently falling back to a newer patch, which would move the default image for existing installs. - Restore 0644 on the regenerated files (mktemp's 0600 survived the mv). - Exclude /hack from the chart tarball via .helmignore. - Refresh files/versions.yaml: v25.8 -> 25.8.32.4, the true latest 25.8 patch now visible with the complete tag list. - bats: add a missing-frozen-tag hard-error case and make the atomicity test reach the resolution phase after a valid default rather than failing at the default-in-set check. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
c83a22e077 |
fix(etcd): spread members across nodes with default soft pod anti-affinity
The etcd chart only set a soft topologySpreadConstraints and never rendered spec.affinity, so the scheduler could place several members of a 3-node EtcdCluster on the same node; losing that node then took out quorum (observed in production after a single node reboot). Render EtcdCluster.spec.affinity from a new `affinity` value. When it is left empty the chart applies a soft (preferred) podAntiAffinity on kubernetes.io/hostname, so members are spread on a best-effort basis without ever leaving a member Pending on small or node-pinned-storage clusters. Operators can override the value — e.g. a hard requiredDuringScheduling rule — where node count and storage allow. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Kirill Ilin
|
212d0e1944 |
feat(keycloak): let the KMS proxy trust a private Vault CA (#3874)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The KMS-encrypting DB proxy could only reach a Vault whose certificate chains to a publicly trusted root. A Vault fronted by an internal load balancer with a self-signed certificate was therefore unreachable over HTTPS, leaving plain HTTP as the only option for the DEK wrap/unwrap traffic. `encryption.kms.vault` now accepts the CA either inline as `caBundle` (the chart renders the Secret) or as an already-seeded `caSecretName`, with `caSecretKey` naming the key to project. The CA reaches the proxy as `SSL_CERT_FILE` **and** `SSL_CERT_DIR`. `SSL_CERT_FILE` on its own would only *add* the CA to the roots the image ships: `crypto/x509` replaces its file list but then scans `SSL_CERT_DIR` unconditionally, and the pinned proxy image carries `/etc/ssl/certs/ca-certificates.crt`. Setting both collapses the pool to the mount, so the connection that guards the KEK trusts the configured CA and nothing else. The database leg is unaffected either way, because it never consults the system trust store: it is plaintext today, and will verify against an explicit `RootCAs` pool once `KKP_BACKEND_CA_FILE` is wired. Supporting guarantees: a non-PEM `caBundle` fails the render instead of being dropped silently at runtime; rotating an inline `caBundle` rolls the pods through a `checksum/vault-ca` annotation, since Go memoizes its root pool per process; both CA keys are inert unless `kms.backend=vault-transit`, matching every other `vault.*` key; and setting `caBundle` together with `caSecretName` fails the render instead of silently picking one. Existing installations are unaffected: with neither key set the proxy keeps using the image's system trust store, and no volume or env var is added. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(keycloak): the KMS-encrypting DB proxy can now trust a private CA for Vault, supplied inline as encryption.kms.vault.caBundle or as a pre-seeded Secret via encryption.kms.vault.caSecretName (encryption.kms.vault.caSecretKey names the key). The Vault connection is pinned to that CA alone, which makes a Vault behind an internal load balancer with a self-signed certificate reachable over HTTPS instead of plain HTTP. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Vault TLS CA configuration for encrypted deployments. * Supports inline certificate bundles or existing Secrets with a configurable key. * Configured secure Vault connections to use the selected CA certificate. * **Bug Fixes** * Rejects invalid certificate bundles and prevents conflicting CA sources. * Restricts custom CA settings to Vault Transit connections. * Preserves the system trust store when no custom CA is configured. * Triggers pod rollouts when inline CA bundles change. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Alexey Artamonov
|
e23fceffe7 |
docs(postgres-operator): distinguish chart version from appVersion in comment
Reword the values.yaml comment so it does not call appVersion 1.30.0 a "published chart": the published chart version is 0.29.0 and it carries appVersion 1.30.0. Keeping the two distinct stops a maintainer from running `helm pull --version 1.30.0`. Addresses CodeRabbit review feedback on #3951. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
IvanHunters
|
bf38655ab5 |
docs(computeplane): nodeGroups allowlist drops unlisted fields, not pass-through
The nodeGroups docs still described the old denylist behaviour: "unlisted NodeGroup fields pass through to the wrapped chart unvalidated". After the switch to an allowlist (pick) both halves are wrong: fields go to the kubernetes-nodes pool chart (not the wrapped kubernetes chart), and anything the pick does not name is dropped, not passed through. A tenant that sets e.g. talos.version on a nodeGroup gets no error and no effect (accepted-but-inert). Correct the source @param and the hand-written README intro, regenerate schema/README/cozyrd, and refresh the stale "omit" wording in the splice test comment to say allowlist. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Kirill Ilin
|
517e33f67b |
fix(keycloak): pin the Vault leg to the configured CA and roll on rotation
SSL_CERT_FILE alone does not narrow trust. crypto/x509 replaces only its file list and then scans SSL_CERT_DIR unconditionally, and the pinned proxy image ships /etc/ssl/certs/ca-certificates.crt, so the connection guarding the KEK trusted the private CA plus every public root — a mis-issued certificate for the Vault hostname was still accepted. Set SSL_CERT_DIR to the mount as well, which collapses the pool to the CA the operator configured. An unvalidated caBundle failed open on top of that: a non-PEM value is dropped silently by AppendCertsFromPEM, leaving the proxy on the image's roots with no error at all. Reject it at render time, the way backend and auth already are. Rotating an inline caBundle had no effect either. The pod template did not change, so no new pods, and crypto/x509 memoizes the pool per process — the upgrade reported success while the old CA stayed in use until some later restart. Derive a checksum/vault-ca annotation from the bundle so the rollout happens with the change. Also add vault.caSecretKey: the projected key was hardcoded to ca.crt, and a bundle stored under another name (a cert-manager Certificate from an ACME issuer emits no ca.crt at all) wedged the pod at mount time with no way to correct it from values. Drop the claim that the database leg verifies against its own CA pool: the chart never sets KKP_BACKEND_CA_FILE, so that leg is plaintext today, as the note 30 lines above already says. The conclusion holds — it never consults the system trust store — but for the other reason. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
Kirill Ilin
|
98ef5413f4 |
fix(keycloak): keep the Vault CA options inert on the static KMS backend
The volume and mount that consume the Vault CA sit at pod level, outside the vault-transit env block, while $vaultCASecret was seeded from caSecretName regardless of the backend. With kms.backend=static and caSecretName set, the Deployment mounted a Secret the chart never renders, so the proxy — the single choke point for all Keycloak DB traffic — would sit in ContainerCreating. Gate both CA keys, and the mutual-exclusion check between them, on vault-transit, so the whole vault.* subtree behaves the same way under static: ignored. Project only ca.crt from a pre-seeded Secret, which keeps a cert-manager tls.key out of the container and turns a bundle stored under another key into a mount failure rather than an opaque TLS error later. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
Kirill Ilin
|
3531553176 |
feat(keycloak): let the KMS proxy trust a private Vault CA
The proxy could only reach a Vault whose certificate chains to a publicly trusted root, so a Vault fronted by an internal load balancer with a self-signed certificate was unreachable over HTTPS. Allow the CA to be supplied inline (caBundle, the chart renders the Secret) or as a pre-seeded Secret (caSecretName), mounted and exposed as SSL_CERT_FILE. That redirects only the system trust store, which nothing but the Vault client consults — the database leg verifies against its own CA pool. Assisted-By: Claude AI Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
Andrei Kvapil
|
652231e0cc |
docs: add public ROADMAP.md (2026-2028) (#2666)
## Summary This PR adds a public `ROADMAP.md` covering the project's direction over the two-year window May 2026 – May 2028, aligned with the CNCF Sandbox → Incubating → Graduated trajectory. ## What A single living document organized into 11 strategic tracks: 1. Platform Core 2. Testing & Conformance 3. Security, Certifications & Compliance (largest track — covers OSPS Baseline L1–L3, OpenSSF Best Practices Passing/Silver/Gold, Scorecard, SLSA Build L1–L3, CLOMonitor, CNCF TAG Security self-assessment and third-party audit, EU CRA posture, SOC 2 evidence pack, PCI DSS v4.0.1 reference architecture, ISO/IEC 5230) 4. Storage Independence (blockstor) 5. Host OS Strategy 6. AI/ML Platform 7. AI Inside Cozystack 8. Marketplace & Application Ecosystem 9. Multi-Cluster & Federation 10. Developer Experience & Reference Architectures 11. Ecosystem Standardization & Community ## Why The CNCF Incubation review (targeted for late Q3 2026) expects a public, readable roadmap in-tree. The existing Project V2 board ([#1](https://github.com/orgs/cozystack/projects/1)) is the issue-level source of truth, but a multi-quarter narrative view is needed for adopters, downstream vendors, and TAG reviewers. ## Preview The document is a living artifact: updated quarterly via a public roadmap review, with material changes routed through CzEP (Cozystack Enhancement Proposal) in \`cozystack/community/design-proposals/\`. ## Notes for reviewers - Content is intended as a discussion starting point; expect refinements in subsequent PRs once SIGs form. - Certification timelines are aspirational targets, not commitments — refinement expected once a SIG-Security charter is ratified. - Cross-references to other Cozystack repositories (blockstor, community, cozystack-ui) reflect current state on \`main\` as of 2026-05-18. ## Test plan - [ ] Markdown renders correctly on GitHub - [ ] All internal repo links resolve (\`./GOVERNANCE.md\`, \`./MAINTAINERS.md\`, \`./SECURITY.md\`, \`./CONTRIBUTOR_LADDER.md\`) - [ ] All external references in §10 resolve - [ ] Maintainer review of certification timelines (Track 3) for feasibility - [ ] Maintainer review of SIG formation timeline (Track 11) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated the public roadmap for August 2026 with clearer commitment and direction horizons. * Refreshed current-state information, release details, in-flight initiatives, and milestone planning. * Added platform milestones for service exposure, readiness, quotas, autoscaling, and secrets management. * Strengthened end-to-end quality gates and reproducible benchmarking guidance. * Revised plans for AI/ML, self-hosted assistance, CLI development, governance, conformance, risks, and maintenance. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
32ec1b5a00 |
fix(migrations): correct unadopted-record count semantics and stale comments
- The unadopted-record WARNING said "N pool(s)" but counted SKIPPED_FILE lines, and the foreign-owner branch writes one line per object (so a single pool with MD+MHC+WM reported "3 pools"). Reword to "N record(s) ... one line per object or pool ... not a distinct-pool count" so the number does not mislead. - The inline-chart restamp comment implied the cozystack-apps source is enforced; it is not (matching by chart name only is a deliberate trade-off: gating on the historical sourceRef risks under-matching a real legacy parent, which loses its workers, whereas the loose match's worst case is a harmless mislabel). Spell out the trade-off and that the render guard is the data-loss backstop. - The overflow comment cited "up to 42 chars per rest.go"; the enforced cap is 32 (maxKubernetesClusterName = 53 - 17 - 4). Correct it and note legacy/hand-created names can still be longer, which is why the overflow branch exists. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
3630842172 |
fix(computeplane): reject non-RFC-1123 pool keys at render time
The pool key becomes the pool HelmRelease name and its CAPI object names, which must be RFC-1123 labels, but only the length was guarded. An uppercase/underscore key rendered an invalid metadata.name the apiserver rejects, and a key containing ':' rendered invalid YAML that failed the whole module render -- a non-uniform, late failure next to the guarded length case. Add a render-time RFC-1123 guard on the key alongside the length guard (mirrors migration 56's child-name guard), and pin the length guard's derived remedy budget (15) in the overflow test so a wrong computation is caught, not just the presence of the error. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Timur TukaevandAndrei Kvapil
|
2619c91f09 |
docs: add public ROADMAP.md (2026-2028)
Adds an in-tree public roadmap giving adopters, downstream vendors and CNCF TAG reviewers a multi-quarter narrative view alongside the issue-level Project V2 board. The document covers twelve tracks, the certification and compliance programme, the standardization strategy and the SIG formation plan. It marks which horizons are commitments with owners and capacity behind them and which are direction, and it states plainly that the board remains the issue-level source of truth. Co-authored-by: Andrei Kvapil <kvapss@gmail.com> Assisted-By: Claude Signed-off-by: Timur Tukaev <timur.tukaev@aenix.io> Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
IvanHunters
|
c0fcdd767c |
fix(migrations): discover pre-chartRef parents via inline spec.chart in the restamp
The restamp discovered unlabelled parent kubernetes releases by spec.chartRef only, but that misses the population it exists for. The apps.cozystack.io/ application.kind label was introduced 2025-12-19 ( |
||
|
IvanHunters
|
c6b6755eec |
fix(migrations): record foreign-owner skips, harden record_skip, scope restamp to kubernetes- names
Three hardening fixes on migration 56's new code: - The foreign-owner branch in adopt_one pins the object prune-proof but does not adopt it (the child release can then stay Failed on the ownership conflict), yet it was the one warn+pin+skip path that did not record itself in the durable unadopted ConfigMap -- so an operator relying on that record to find stuck pools would miss it. Record it like every sibling branch. - record_skip ran a bare append under set -e inside skip branches that are designed never to abort; a failed write (e.g. /tmp full) could deadlock the fleet upgrade for one tenant. Make it fail-open, matching the branches' intent. - The restamp selector matched any release by chartRef; scope it to names starting with kubernetes- so it cannot mislabel (and then have the sweep skip) a non-standard release, keeping restamp and the sweep's contract in sync. Also fix the previously-vacuous negative restamp test (its fixture lacked the chartRef, so it never reached the label guard it claimed to check). Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
c2ee58756e |
fix(computeplane): allowlist pool values instead of denylisting, derive length-guard prefix
The pool-value splice used omit (a denylist), which leaks every key it does not name. Since this module runs untrusted tenant code and the NodeGroup schema has no additionalProperties:false, a tenant could pass the platform-injected _cluster object (cluster-domain/oidc/dns01/proxy, delivered via cozystack-values and merged UNDER spec.values, so a tenant key wins) or nameOverride straight through to the pool chart -- and any future kubernetes-nodes top-level value would silently re-open the same hole. Switch to pick: pass through only the enumerated per-pool shape fields, so everything cluster-level or module-owned is dropped by construction (fail-closed for new fields). Also derive the length-guard's release-name prefix from $clusterRelease so it cannot drift from the name actually rendered. Regression-tested: _cluster and nameOverride no longer reach the child values; maxUnhealthy and the module wiring still do. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Alexey Artamonov
|
30af0e1f60 |
docs(postgres-operator): update values comment for the native 1.30.0 bump
The explanatory comment in values.yaml still described the removed patches/cloudnative-pg-1.28.2.patch mechanism and the old 0.27.1/1.28.1 versions, and pointed at an "edit the patch" workflow that no longer exists. Rewrite it to describe the native 0.29.0/1.30.0 chart bump (operator and CRDs aligned by construction, no patch step) while preserving the still-valid guidance: do not pin image.tag, keep the operator and CRDs in lockstep, since a mismatched pin prunes CRD status fields and fails backups cluster-wide. Addresses review feedback on #3951 (IvanHunters, CodeRabbit). Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
IvanHunters
|
3d1a83f419 |
fix(migrations): make the unadopted-pools record publish best-effort
The durable-record ConfigMap write sat under set -euo pipefail right before the version stamp, so a permanent failure (e.g. the record exceeding etcd's object size limit at a very large skip count) would abort the migration and deadlock the fleet upgrade with no path out -- even though every skipped pool was already pinned prune-proof earlier in the sweep, so the workers are safe regardless of whether the record lands. The record is a convenience artifact, not a safety mechanism, so swallow its failure with a warning instead of failing closed. Covered by a bats case that fails the publish and asserts the run still stamps. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
12fc8365bb |
fix(kubernetes): make the worker-adoption guard remedy actionable
The guard's refusal message told the operator to run the platform migration, but the case that reaches it is precisely one the migration already ran and stamped for (its slot was renumbered out on a backport, or the parent HelmRelease predates the application.kind label and the selector skipped it) -- re-running does nothing. Name the manual recovery instead: annotate resource-policy=keep on the listed MachineDeployments, then stamp the cozystack-version ConfigMap back below the worker-adoption migration so it re-runs. Also note in the Phase 2 README that computeplane clusters render their worker pools as separate KubernetesNodes releases too. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
952901776a |
fix(migrations): adopt unlabelled parent kubernetes releases and record unadopted pools in migration 56
The worker-adoption sweep selected parents by the apps.cozystack.io/ application.kind label, which postdates v1.2. A parent kubernetes HelmRelease older than that, or one restored by hand, lacks the label: the sweep skipped it, stamped, and the parent's control-plane-only re-render then pruned its un-pinned workers with no pin and no warning. Discover such releases by their chartRef instead and restamp the three application labels before the sweep (additive, cannot corrupt a release, and repairs it for cozystack-api too). Every warn+pin+skip branch leaves a pool pinned prune-proof but unadopted, recorded only in the Job log, which the next upgrade reaps; the migration does not re-run, so an operator cannot find those pools afterward. Accumulate the skips and publish them as configmap/cozystack-migration-56-unadopted before the stamp. Extend the fake kubectl and add bats coverage for the restamp (positive and negative) and the unadopted-ConfigMap paths. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
34e801cb9e |
fix(computeplane): stop tenant pool values leaking cluster-level keys and overflowing release names
The per-pool child HelmRelease spliced the tenant nodeGroup dict verbatim (toYaml) after setting cluster/clusterReleaseName, and the NodeGroup schema has no additionalProperties:false. A tenant could therefore (a) set a duplicate cluster/clusterReleaseName key that wins the YAML merge and re-parents the pool onto a tenant-controlled cluster, or (b) pass talos/ version/images through a shape-only knob and pick the worker OS image, installer and kubelet version. omit those five keys at the splice so only the pool's own shape flows through (tolerated extras like maxUnhealthy still do). Separately, each pool renders a 38-char fixed release-name prefix against Helm's 53-char cap, so a nodeGroup key over 15 chars created a HelmRelease the apiserver accepts but Flux never reconciles, with no other signal. Guard the length at render time, next to the ingress-role guard, mirroring the migration overflow branch and the aggregated API's validateNameLength. Also refresh the generated values doc: the module now materialises its own default md0 and pool storageClass falls back to the kubernetes-nodes chart default, not the wrapped kubernetes app. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Andrei Kvapil
|
4689b2b844 |
feat(kubeovn-webhook): add client-certificate verification, off by default (#2698)
## What this PR does Adds client-certificate verification to the kube-ovn webhook, and turns it on only when an operator supplies a CA bundle. `--client-ca-file` is new and empty by default. With it empty the server sets `tls.RequestClientCert`: it asks callers for a certificate, records the issuer of whatever arrives, and serves the request either way. Set the flag and the server switches to `tls.RequireAndVerifyClientCert`, so a caller without a certificate signed by that CA is refused at the TLS handshake. **This is step one of two, and on its own it does not close [GHSA-g883-q79m-8225](https://github.com/cozystack/cozystack/security/advisories/GHSA-g883-q79m-8225).** With the default empty value nothing is rejected — the webhook still answers an unauthenticated caller, and the only change is that the caller's issuer now appears in the logs. That is deliberate: switching straight to `RequireAndVerifyClientCert` would break every cluster whose API server sends no client certificate, and there is no way to know from here which those are. The logs are how we find out. Step two, in a follow-up: read a day of issuer logs on a Talos cluster and on `isp-full-generic`, confirm the API server presents a certificate we can pin, then set the CA bundle so verification becomes mandatory. The advisory should be published as fixed only after that lands — not with this PR. An earlier revision also carried a NetworkPolicy restricting ingress to `kube-system`. It was dropped, and the title has been corrected to match; only the TLS path is left here. ### Screenshots N/A — no user-facing or UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff: no row matches. The change touches one system package and the Go image it builds — no app package, no `values.schema.json`, no `ApplicationDefinition`, no platform variant, no node prerequisite. ### Release note ```release-note feat(kubeovn-webhook): the webhook now accepts --client-ca-file. Left empty (the default) it requests and logs client certificates without requiring them; set to a CA bundle it rejects callers that do not present a certificate signed by that CA. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional client-certificate verification for the pod mutation webhook through configurable CA settings. * Added separate rate limits for requests with and without client certificates. * Added remote-address logging and clearer verification status reporting. * **Security** * Requests continue to be forwarded while certificate observations are recorded when verification is disabled. * Serving-certificate updates remain supported; client CA changes require a webhook restart. * **Documentation** * Documented client CA configuration and current certificate rotation limitations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
35668dd27c |
feat(dashboard): rework the Marketplace entry point and the Admin portal (#3928)
## What this PR does Makes the Console the dashboard's entry point, reworks the Marketplace into an ordinary destination, and turns the Admin portal into the place where tenant-wide administration actually lives. **The Console is where the dashboard opens.** Visiting the dashboard without a path lands on the Console, which shows what the tenant already runs — what someone opening it almost always came for — and the header gives the Console the primary emphasis. The Marketplace is a place you go deliberately, when the intent is to add something new, and it is one click away from its own tab, from the Console overview and from the command palette. **Marketplace** opens the grouped application list directly. The "Choose a product to deploy" landing page and its Show-all-apps button are gone, and category links keep filtering to a single category. **Administration moves from Console to Admin.** Tenants, Modules, External IPs and the per-tenant Info page need no special permission, so the Admin tab is now always visible instead of being gated. The two cluster-wide operator areas keep their own independent gates, so Capacity and Backup Classes still appear only for users who can open them. Resource routes became portal-aware, so a detail, edit or order page opened from Admin stays under `/admin` instead of jumping back to `/console`. **Tenants** is a hierarchy tree instead of a flat table. It is built from `TenantNamespace` labels (`tenant.cozystack.io/<ancestor-ns>`), so a tenant whose parent is not accessible attaches to its nearest visible ancestor rather than disappearing, and rows are named relative to that parent — a sub-tenant reads as `crpjxhwm`, not `whmcs-crpjxhwm`. Each row carries its own actions: open that tenant's Info, or create a sub-tenant at any level of the tree. **Modules** is rebuilt around the same tree. It previously showed only the selected tenant's modules as Enabled or Disabled cards, which meant a module provided by a parent tenant looked disabled, and clicking it navigated to a Tenant CR that does not exist in the child's namespace — an endless spinner. The page now covers every tenant the user can access, with each tenant's modules as chips showing the module icon, the namespace running it and its readiness. Availability comes from the `namespace.cozystack.io/<module>` label on each `TenantNamespace`, which names the providing namespace: the tenant itself for a local module, an ancestor for an inherited one. An inherited chip is shown only when that ancestor is out of the user's reach, since otherwise its own chip already shows the module. Clicking any chip opens the module in whichever tenant runs it. Tenants whose whole subtree has no modules collapse into a "+ N tenants" branch. **External IPs** lists LoadBalancer services across every accessible tenant in one table with a Namespace column, instead of only the selected tenant. A namespace whose services the user cannot list contributes no rows rather than failing the page. Two navigation fixes come along with this. Detail pages offered a fixed tab set, so Info showed Workloads, Services and Ingresses tabs it never fills — tabs now appear only for resource groups the instance actually owns, probed through the tabs' own list refs so they share one cache entry and one watch. Back and the sidebar followed the resource plural rather than the entry point, so a module page sent the user to a one-item list and highlighted nothing; module pages now return to Modules and keep it highlighted, while Info returns to Tenants. Finally, copying a secret key no longer requires revealing it on screen first, and the button briefly turns into a check mark so it is visible that the value reached the clipboard. ### Screenshots <img width="1720" height="1374" alt="Screenshot 2026-08-20 at 21-08-30 Cozystack" src="https://github.com/user-attachments/assets/8499577c-8f9e-4df9-a6bb-df507497a4a0" /> <img width="1720" height="1374" alt="Screenshot 2026-08-20 at 21-08-40 Cozystack" src="https://github.com/user-attachments/assets/01d256e9-2585-41fd-8070-b910f03d8f3f" /> ### Downstream repositories The diff touches only `packages/system/dashboard/images/console`. The console is vendored in this repository (`cozystack-ui` is archived), and the change adds no package, no values schema, no `ApplicationDefinition` semantics and nothing under `hack/`, so no repository in the trigger map is reached. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(dashboard): the dashboard now opens on the Console rather than the Marketplace, the Marketplace opens the application list directly, and tenant administration moves into the Admin portal, which is now always available. Tenants and Modules are shown as the tenant hierarchy across every tenant the user can access: a module provided by a parent tenant is no longer reported as disabled, and opening it lands on the tenant that actually runs it instead of hanging. External IPs covers every accessible tenant in one table, detail pages only offer tabs the instance has resources for, and a secret key can be copied while it stays masked. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Administration now includes dedicated tenant, module, external IP, and information pages. - Tenants appear in a collapsible hierarchy with creation, editing, and detail navigation. - Modules and external IPs are shown across accessible tenant namespaces, including inherited modules. - Application details show tabs only for available resources. - Secret values can be copied while remaining masked. - Console is now the primary landing page, with Marketplace available separately. - **Bug Fixes** - Improved sidebar highlighting for nested pages. - Administration is accessible without operator permissions, while restricted areas remain protected. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Alexey Artamonov
|
3c78c0be56 |
feat(clickhouse): add configurable version parameter
Add a `version` value (v25.8, v25.3, v24.9; default v24.9) to the ClickHouse chart, backed by files/versions.yaml and resolved by templates/_versions.tpl for both the ClickHouse server and Keeper images, so the Cozystack API returns the engine version (spec.version) like the other managed-DB charts. Default v24.9 -> 24.9.2.42 keeps existing installations byte-for-byte identical. The maintainer-run hack/update-versions.sh generator intersects the server/keeper Docker Hub tags byte-collated (LC_ALL=C) so comm keeps tags across the X.9 -> X.10 boundary; freezes the default major to the tag it already ships (read from versions.yaml, so the freeze follows the default); errors on a dropped default or an unresolvable major; writes atomically; and runs on stock macOS bash 3.2 / BSD. Covered by hack/clickhouse-update-versions_test.bats (10 cases) and the tests/version_test.yaml helm-unittest suite. Refs: #1246 Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
b20e8ee91a |
fix(postgres-operator): bump CloudNativePG to 1.30.0 for the Cluster scale-subresource selector
Bump the vendored cloudnative-pg chart 0.27.1 -> 0.29.0 (operator + CRDs 1.28.1-patched-to-1.28.2 -> 1.30.0) and drop the hand-maintained cloudnative-pg-1.28.2.patch entirely — 0.29.0 is a published chart, so no version-alignment patch is needed. Why this is required -------------------- Database read-replica autoscaling (community proposal design-proposals/database-horizontal-autoscaling) drives the CNPG Cluster scale subresource with a stock HPA (via KEDA). That only works when the subresource serves status.selector, which cloudnative-pg#8996 added and shipped in CNPG 1.28.4 / 1.29.2 / 1.30.0. cozystack shipped 1.28.2, whose scale subresource has no selector, so the HPA fails with InvalidSelector. Why 1.30.0 and not 1.28.4 ------------------------- The floor is >= 1.28.4, but the upstream chart repo publishes no chart for 1.28.4 (the 1.28 line stops at chart 0.27.1 / appVersion 1.28.1), so 1.28.4 could only be reached by hand-patching the 0.27.1 chart's operator image and CRDs up to the backport. A live PoC on a dev cluster proved that path is broken: the 1.28.4 operator running under the 0.27.1 (1.28.1) chart templates crash-loops on a missing RBAC grant it needs but the older chart does not provide — endpointslices.discovery.k8s.io is forbidden: ... cannot list resource "endpointslices" — so status.selector is never populated and the HPA still cannot scale. The hand-patch would have to port the operator's RBAC (and any other template delta) from 1.28.1 to 1.28.4 by hand, with no published chart to diff against. 1.30.0 (chart 0.29.0) is the only published chart that carries the selector together with matching RBAC/CRDs/templates, and the 1.28 line is EOL (2026-06-30), so 1.30.0 also keeps us on a supported release. cozystack uses the barman-cloud plugin (not native barman, removed in 1.31) and no CNPG Poolers, so the 1.30 deprecations and the known pooler bug do not apply. Update the version-guard unittest to 1.30.0 and assert the clusters CRD scale subresource carries labelSelectorPath: .status.selector. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Andrei Kvapil
|
724354ffe1 |
fix(console): address the review findings on the Admin portal rework
Four of the five findings share one root cause: a node in the tenant
tree can attach to its nearest *visible* ancestor when its real parent
is inaccessible, and per-row actions then derived their target from
that bridged ancestor.
The Edit action on a bridged tenant row pointed at a Tenant CR that
does not exist. The CR lives in the real parent's namespace, so the
target now comes from realParentNamespace — and Edit is offered only
when that namespace is readable, because otherwise there is no CR the
user could open and the button could only ever dead-end. The test that
asserted two Edit buttons had frozen the broken target as expected
behaviour; it now names the rows it expects, and the Edit button gained
a title so that assertion means something.
An inherited module chip is rendered only when the providing tenant is
NOT visible, yet it was a live button: clicking it denied the instance
GET and switched the active tenant to a namespace outside the visible
set, which the selector then bounced to an arbitrary fallback. It is
now an inert badge that states where the module comes from.
InfoRedirect's missing-definition branch hardcoded /console, throwing an
admin out of the portal its own docstring promises to preserve.
The sidebar's alsoMatch used a bare startsWith, so /admin/apps would
light up for /admin/apps-v2. It now matches whole path segments.
And SecretsTab let a rejected clipboard write escape as an unhandled
rejection, leaving the button silent — a denied permission or an
insecure context was indistinguishable from a no-op. It now reports the
failure.
Declined: CodeRabbit asked to split `import { useK8sList, type K8sResource }`
into a separate `import type`. The inline modifier is valid under
verbatimModuleSyntax and is what 26 other imports in this console use,
so the change would make the file the odd one out.
Reported-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
|
||
|
IvanHunters
|
5b08d48632 |
docs(migrations): pin DEFAULT_MD0 provenance to the pre-removal helper commit
The kubernetes.nodeGroups helper that DEFAULT_MD0 reproduces was deleted in this line of work, so nothing in-tree diffs against it and a future edit to the constant (instanceType/diskSize/...) would silently rename the content-hashed KubevirtMachineTemplate and roll every implicit-md0 pool. Reference the pre-removal definition (git show 92b9785da^:.../kubernetes/templates/_helpers.tpl) so the canonical default stays diffable. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
8cf3af7309 |
test(kubernetes-api): cover the removed-fields warning at its Create/Update call sites
TestWarnRemovedKubernetesFields pinned the helper in isolation, so deleting either call site (rest.go Create/Update) left it green while an operator editing a Phase 2 removed field would silently get no warning. These drive Create and Update end to end and assert the warning surfaces; removing a call site now fails loudly (verified by mutation). Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
d93e136e23 |
fix(computeplane): provision worker pools via kubernetes-nodes after Phase 2 split
Phase 2 moved worker nodeGroups out of apps/kubernetes into apps/kubernetes-nodes, but the ComputePlane module still passed nodeGroups to the wrapped kubernetes chart, which now ignores it: a fresh ComputePlane cluster came up with zero workers (and the pinned ingress-nginx addon stuck Pending), while existing ones were unreachable by migration 56 (it only adopts kubernetes-* releases, not computeplane-cluster). The module now emits one kubernetes-nodes child HelmRelease per nodeGroup, shaped like a natively created pool, and materialises the pre-split default md0 (roles: [ingress-nginx]) when nodeGroups is empty. The pool chart's clusterName gains an internal clusterReleaseName override so pools attach to the module's fixed computeplane-cluster release, which is off the aggregated API's kubernetes-<cluster> convention; the default keeps every API-created pool byte-identical. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
myasnikovdaniil
|
334e2d8bf9 |
test(e2e): record the runner's SVM feature flags in the identity capture (#3945)
Adds one line to the runner identity record, taken from the `lscpu` read it already does: ``` svm features: svm npt nrip_save vgif ``` `vgif` decides whether a guest that is itself running a hypervisor pays a VM exit on every `CLGI`/`STGI`. Here that is the sandbox Talos nodes hosting the tenant KubeVirt VMs, so it only bites at double nesting, and the sandbox nodes' own boot is unaffected. Across the archived reports no run missing the flag has passed a tenant-Kubernetes suite: 0 of 62, against 11 of 12 with it. Exposure varies between hosts of the same shape, so it is not a property of the lane and cannot be recovered once the runner is recycled. Recording it on every run, red and green, is what makes the correlation countable rather than reconstructed by hand. `apply` runs `hack/e2e-runner-identity.sh` before `docker run`, and every workflow that runs the suite reaches it through `prepare-env`, so this is captured once per run on both paths. `lscpu` is already answering there: the `cpu model` line in today's records is parsed out of the same table. No new bounded read, no compiler on the runner, no sysfs. Gated on `svm` so an Intel host's `vnmi` is not reported as an SVM feature. Five tests for the reader. Identity suite 62 assertions, `hack/` sweep 732. No behaviour change and no gating on the flag. |
||
|
Myasnikov Daniil
|
a94abe0971 |
test(e2e): record the runner's SVM feature flags in the identity capture
Adds one line to the runner identity record, taken from the lscpu read it
already does:
svm features: svm npt nrip_save vgif
vgif decides whether a guest that is itself running a hypervisor pays a VM exit
on every CLGI/STGI. Here that is the sandbox Talos nodes hosting the tenant
KubeVirt VMs, so it only bites at double nesting, and the sandbox nodes' own
boot is unaffected.
Across the archived reports no run missing the flag has passed a
tenant-Kubernetes suite: 0 of 62, against 11 of 12 with it. Exposure varies
between hosts of the same shape, so it is not a property of the lane and cannot
be recovered once the runner is recycled. Recording it on every run, red and
green, is what makes the correlation countable rather than reconstructed by hand.
No new bounded read, no compiler on the runner, no sysfs. Gated on svm so an
Intel host's vnmi is not reported as an SVM feature.
Five tests for the reader. Identity suite 62 assertions, hack/ sweep 732.
No behaviour change and no gating on the flag.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Aleksei Sviridkin
|
3c780acd41 |
test(e2e): stop blocking merges on the tenant node-join deadline (#3932)
## What this PR does The tenant node-join wait in the two kubernetes e2e suites has stopped discriminating. Nested virtualisation on the shared runners degrades far enough that a worker which registers in minutes elsewhere can miss any deadline this test can afford, so a red on that one wait says as much about the machine the run landed on as about the product. What it produces in practice is a re-run, and a signal that teaches people to re-run is a signal that gets the meaningful reds re-run too. The suite still fails on it. That is the part worth reading twice, because it is what separates this from hiding a flake: the script exits non-zero, Chainsaw records a failure, and its whole catch runs (previous-instance container logs, the host snapshot, the data-plane capture, the event dump), every one of which is keyed on the test failing. The JUnit report says a test failed. What changes is one layer up. The wait writes a `SOFT-RED-node-join.txt` marker beside its diagnostics, and a new lane-side gate, `hack/e2e-node-join-soft-red.sh`, reads that marker to decide whether the job blocks. Softening inside the suite would have bought a green check by throwing all of that evidence away. The gate opens on one shape only, every failed test in the run carrying that marker, and blocks on everything else, including every way it can fail to answer the question: a report it cannot read, a failed test the report does not name, a failure with no marker, a run whose report records no failure at all. A marker is written only when `timeout` reports 124 on that one wait, so an assertion after the join, or the wait failing to run, stays an ordinary red. What the marker attributes is the test rather than the individual failure: Chainsaw writes one testcase per Test and combines that Test's step errors into its single failure element, so a second step error rides soft alongside the deadline, while a failed `catch`, `finally` or step-level `cleanup` rides soft for a different reason worth keeping apart, since at the pinned version those run with no step report and never reach the failure element at all. The convention doc states both rather than working around the granularity, since the suite stops at the wait and nothing it exists to prove runs after the marker. Only the lanes that block a merge ask the gate. `pull-requests` and `e2e-fork` do; `nightly` and `e2e-tag` run the same suites and stay hard. `e2e-tag` is the release candidate's own e2e check, where a soft pass would ship a release whose tenant clusters brought up no worker nodes. `nightly` is the only place an e2e failure reaches anyone who did not open a pull request, since nothing in this tree sends a notification anywhere, so the red job is the notification. A unit guard holds that split in both directions and derives the lane set from the workflows that run the suite, so a new lane has to be placed on one side or the other before it passes. One consequence of a softened red is worth naming rather than discovering later: the job finishes green, so its result stops answering whether the suite passed, and it has two kinds of consumer that each break their own way. A step gated on the job failing silently stops running. The SSH breakpoint in `pull-requests` is that case, and it is the only interactive way into a sandbox whose node-join never finished, so the E2E step records what the gate decided and the breakpoint's condition reads it back; the step is label-gated, so the session opens for a PR someone deliberately marked for debugging rather than on every softened run. A job that reports the result silently starts reporting a pass. That is the required `E2E Tests` commit status in both pull-request lanes, and on a softened run it now says the node-join deadline was missed, that the suite failed, and that the merge is not blocked on it, instead of saying the suite passed. The handover is four links long, and a guard reads each of them inside the block it has to live in rather than anywhere in the lane: the step carrying the gate's id writes the decision to the output sink, the job that runs the suite publishes it naming that step, the job that reports the result reads it off that job, and it branches on the value ahead of the branch that would otherwise call the run a pass. A link matched file-wide passes while it sits on a neighbour, and the value is then empty for good with nothing to say so. The first link is read tighter still, and in both directions: the write has to be an unconditional statement in the same branch as the exit that makes the job green, that branch has to fall through to a written non-zero exit, no other green exit may leave the step, the step may not be marked unable to fail, and the write may not appear anywhere else. A write under a condition of its own records the decision on some runs and not others; an exit that escapes the branch turns a red the gate REFUSED to soften into a green job with nothing recorded; and a write on the passing path would report a suite that passed as one that missed the deadline. What the guard cannot do is stated with it: it cannot tell one reporter from two, it reads the branch as text rather than evaluating it, and every block it reads is delimited by indentation. What is given up is real. A regression whose only symptom is that workers never reach Ready no longer blocks a merge, and these two suites are the only ones in the tree that boot tenant worker nodes. The mechanism sees a timer, not a cause, so a deterministic node-join regression rides soft as readily as a slow runner does, and the failed job stops being the thing that opens a debugging path on its own. That is a debt rather than a design, and the way out is written down beside the convention it bends: calibrate the runner canary against real lane data, then move the tolerance under the canary's alert so the lane softens a run measured as degraded and blocks one measured as healthy. Until that calibration exists the canary cannot carry the decision, which is why it does not carry it today. The deadline goes from 18m to 29m, and the reason is bounded on purpose. 29m is this test's own budget and nothing else: its clock starts where the wait starts, after the machinedeployment wait, the LoadBalancer assignment and the healthz probe, each with a ceiling of its own, while the chart's timers have been running since the Machine was created. The autoscaler's `maxNodeProvisionTime` measures from that creation and the MachineHealthCheck's `nodeStartupTimeout` from the later of creation and `InfrastructureReady`, restarting at that transition, so the offset is not fixed and no ordering against either timer is claimed. What the chart's 20m and 30m give is a scale to pick against: a budget over the first leaves room for a remediation cycle to fit inside the window, which is a possibility rather than a guarantee, since a run can spend the whole window on one worker that never comes back. The operation ceiling both suites give the script rises with the deadline (50m to 67m), and the diagnostics phase budget rises inside it (7m to 8m), because the collectors gated ahead of the serial console cost more between them than the old budget covered. Both budget guards read their terms from the sources that set them rather than restating them. The e2e job cap rises from 180 to 215 minutes in all four lanes, and its guard is honest about what it can hold: it derives the pair's operation and catch ceilings from source and requires one figure across every lane that runs the suite, and it records that the suites' full summed ceilings (226 minutes counting the `finally` legs and the storageclass-fallback Test) have never fit any cap in that file, including at 180. The margin above the derivable bound is empirical instead: three nightlies that failed both bringup Tests ran their full-suite e2e job in 149, 158 and 160 minutes under the previous ceilings, a run of that shape gains at most 12 minutes per suite here, and 215 clears the resulting projection by about 30 minutes where 180 cleared its measurement by 20. The prose that restates the cap outside the workflows moves with it, in the Chainsaw config and in the two report collectors that derive their own headroom from it. Test coverage: a new unit suite for the gate pins the direction it fails in (an unreadable report, a failed testcase the report leaves unnamed or names with the empty string or with blanks only, a failure without a marker, and a report recording no failure all block), the lane split in both directions, the four-link handover of the softening decision, and the job cap. The fixtures put the unnameable case ahead of a named suite on purpose, because an awk record runs to the next testcase and a case placed last cannot exercise the leak the tally exists to catch. The existing node-join suite gains guards that the marker is written only under the status meaning the deadline expired, that it is written ahead of the diagnostics phase rather than behind it, and that the deadline quoted in prose across the tree is the one the wait gives, in words as well as in minutes. Every one of those guards was checked by mutating the thing it protects and confirming it, and only it, goes red. ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note test(e2e): the tenant node-join deadline still fails its suite and still collects its diagnostics, but no longer blocks a merge in the pull-request lanes; nightly and the release-tag lane stay hard ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - E2E checks can now report a successful, non-blocking status when the only failure is the recognized tenant node-join deadline. - Failure diagnostics and suite results remain available for these tolerated failures. - **Bug Fixes** - Improved detection and attribution of node-join deadline failures, avoiding false-positive soft passes. - **Documentation** - Updated E2E timing, diagnostic budgets, and failure-handling guidance to reflect the longer test windows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
ad62a8d62b |
test(e2e): stop blocking merges on the tenant node-join deadline
The tenant node-join wait no longer discriminates. Nested virtualisation on the shared runners degrades far enough that a guest which registers in minutes on real hardware can miss any deadline this test can afford, so a red on that one wait says as much about the machine the run landed on as about the product, and it teaches a reader to re-run the job. The suite still fails on it. That is what keeps Chainsaw's catch running: the previous-instance container logs, the host snapshot, the data-plane capture and the event dump are all keyed on the test failing, and the report stays honest. What changes is one layer up. The wait writes a marker beside its diagnostics, and a new lane-side gate reads that marker to decide whether the job blocks. It opens only when every failed test in the run carries one, and blocks on anything it cannot attribute: a report it could not read, a failed test the report does not name, a failure with no marker. Only the lanes that block a merge ask it. The two pull-request lanes do; nightly and the tag lane run the same suites and stay hard. A soft pass on the tag lane would ship a release candidate whose tenant clusters brought up no worker nodes, and nightly is the only place an e2e failure reaches anyone who did not open a pull request, since no lane in this tree notifies anywhere. A softened red finishes the job green, so the job's result stops answering whether the suite passed, and both kinds of consumer are handed the decision instead of inferring it. A step gated on the job failing would silently stop running: the SSH breakpoint is that case, and it still opens for a pull request marked for debugging. A job that reports the result would silently start reporting a pass: the required commit status now says the deadline was missed and that the merge is not blocked on it, rather than that the suite passed. The deadline goes to 29m. That is the test's own budget and nothing more: its clock starts after the machinedeployment wait, the LoadBalancer assignment and the healthz probe, while the chart's autoscaler and health-check timers have been running since the Machine was created, so no ordering against them is claimed. The chart's 20m and 30m give a scale to pick against, and a budget over the first leaves room for a remediation cycle without promising that one fits. The operation ceiling both suites give the script rises with the deadline, and the diagnostics phase budget rises inside it, because the collectors gated ahead of the serial console cost more between them than the old budget covered. Both budget guards read their terms from the sources that set them. The e2e job cap rises with them, and its guard holds what a guard can hold: the pair's operation and catch ceilings, and one figure across every lane that runs the suite. It also records that the suites' full summed ceilings have never fit any cap in that file, so the margin above the bound is taken from a measured run instead. The prose that restates the cap elsewhere moves with it, and the sweep that holds the deadline's restatements now reads the figure spelled in words as well as in minutes. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrei Kvapil
|
bfd4ea0643 |
feat(dashboard): land on the Console instead of the Marketplace
Opening the dashboard without a path sent the user to the catalog, and the header gave Marketplace the primary emphasis. The Console is the better entry point: it shows what the tenant already runs, which is what someone opening the dashboard almost always came for. The Marketplace stays one click away for when the intent is to add something new. The landing path becomes a named constant beside the portal helper, so the redirect and the tab emphasis cannot drift apart, and the library's own default tab set is reordered to match rather than contradict its only consumer. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
IvanHunters
|
a00421aa17 |
feat(kubernetes-api): warn on Phase 2 removed fields and document them
The Kubernetes CR still accepts and stores spec.nodeGroups, spec.nodeHealthCheck and spec.maxNodeProvisionTime after the Phase 2 split (the adoption migration leaves them on an upgraded CR rather than rewriting parent values mid-upgrade), but they no longer render anything, so an operator scaling via spec.nodeGroups.<x>.minReplicas silently got nothing. Emit a non-blocking client-facing admission warning on Create/Update when any of these fields is present, pointing the operator at the KubernetesNodes resources. A hard rejection is intentionally avoided: it would break edits to already-upgraded clusters that still carry the fields. Also enumerate all the Phase 2 breaking changes (three removed fields, two KubernetesNodes immutability rules, the 32-char cluster-name cap) in the chart README. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
69da032103 |
docs(kubernetes): point references at migration 56 after the renumber
The merge renumbered the worker-pool adoption migration from 54 to 56, but six references still named 54 (README, pre-delete-unpin hook, cluster.yaml, worker_adoption_guard test), which now points at main's redis-operator adoption migration. Repoint them at 56. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |