Fresh KRaft clusters no longer need to be alone in a namespace. Their node pools
are now release-scoped (`<release>-broker`, `<release>-controller`) instead of the
fixed `kafka`/`controller`, which is unique per namespace (KafkaNodePool object
names are namespace-unique) and lets any number of Kafka instances coexist —
every pod/PVC/Service is already prefixed by the unique cluster name.
The fixed `kafka` name is still required to adopt an existing ZooKeeper cluster's
`<cluster>-kafka-N` brokers and data in place during migration, and that name is
namespace-unique — so only one ZooKeeper->KRaft migration can run per namespace.
The broker pool name is resolved by kafka.brokerPoolName: "kafka" for a cluster
that already migrated or is a classic ZK cluster about to migrate, otherwise
"<release>-broker". A second migration into a namespace that already holds a
"kafka" pool owned by another cluster fails the render with a clear message;
fresh KRaft clusters are never affected.
This replaces the previous blanket "one Kafka per namespace" guard
(kafka.assertSingleInstance), which over-restricted the fresh case. The migration
hook and workload monitors follow the same naming; unit tests updated. Verified
on a live cluster: two fresh single-node KRaft clusters (mi-a, mi-b) installed
side by side in one namespace came up with distinct node pools, pods and PVCs.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Addresses the second review round on the ZK->KRaft migration.
Migration hook:
- Fully qualify the Strimzi resources as kafkas.kafka.strimzi.io /
kafkanodepools.kafka.strimzi.io. The bare plural "kafkas" also exists in this
chart's apps.cozystack.io group, so a bare name resolves by per-cluster
discovery priority and can hit the tenant kind — which the hook Role does not
grant, failing the pre-upgrade hook deterministically.
- Classify NotFound structurally via `kubectl get --ignore-not-found` instead of
substring-matching stderr, so an unrelated "... not found" error can no longer
be mistaken for a fresh install and permanently skip a live ZooKeeper cluster.
This also drops the jq dependency.
- Treat a KRaft-native / already-migrated CR (annotated strimzi.io/kraft=enabled)
as done, never driving it backwards into kraft=migration.
- Keep only `before-hook-creation` on the Job's hook-delete-policy so a failed
attempt's pod logs survive for a post-mortem until the next run.
- Route the kubectl image through cozy-lib.image (cluster registry mirror) and
pin it by digest (clastix/kubectl:v1.32@sha256:...), matching the other
cozystack hooks, instead of an unpinned docker.io tag.
Controller quorum:
- Fix the controller node-pool replica count at creation instead of deriving it
from kafka.replicas on every reconcile. Strimzi 0.45 cannot scale a controller
pool, so a day-2 edit of kafka.replicas (e.g. 1 -> 3) must not flip the quorum;
the helper now keeps the existing "controller" pool's replica count when it is
present.
Docs:
- Document that migration to KRaft is one-way: neither a version-field downgrade
nor a pre-KRaft chart rollback is supported once migrated (Strimzi has no
KRaft->ZooKeeper rollback after completion).
E2E:
- Run the ZK->KRaft chainsaw migration in its own namespace instead of the
shared tenant-test. It raw-applies Strimzi resources plus a namespace-wide
LimitRange and leaves deleteClaim:false PVCs, which polluted the sibling kafka
suite. Raise the seed readiness timeout for CI headroom. Validated against a
live cluster: migration adopts the broker PVC in place and reaches KRaft under
a 128Mi LimitRange, in isolation.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The ZK->KRaft pre-upgrade hook created the adopting broker ("kafka") and
"controller" KafkaNodePools without a resources block. Those hook-created pools
govern the Kafka pods for the entire migration window — Helm applies the
resourced steady-state kafkanodepools.yaml only after the pre-upgrade hook
finishes. In a tenant namespace the LimitRange defaults every container to
128Mi, so the migration's controller (and rolled broker) JVMs OOM, never reach
KRaftPostMigration, and the hook times out — wedging the migration and thrashing
it on every reconcile.
Give both hook pools the same resources the steady-state pools get
(cozy-lib.resources.defaultingSanitize from kafka.resourcesPreset/resources).
This also avoids a redundant second broker roll (resource-less -> resourced)
after the hook completes.
Tests: a helm-unittest assertion that both migration pools render a resources
block; the Chainsaw ZK->KRaft suite now seeds under a 128Mi LimitRange (and
gives the seed cluster explicit resources), reproducing the tenant-quota
condition, and asserts the migration still reaches KRaft with the broker PVC
adopted. Validated green against a live cluster under the LimitRange.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Follow-up to the KRaft-migration review. The broker KafkaNodePool must be named
exactly "kafka" so Strimzi adopts an existing ZooKeeper cluster's
<cluster>-kafka-N brokers and data in place during migration, but KafkaNodePool
object names are unique within a namespace — so two Kafka clusters cannot both
own a "kafka" pool there. This is an upstream constraint and Strimzi's own
guidance is one Kafka cluster per namespace
(strimzi/strimzi-kafka-operator discussions/11120).
Adopt that model instead of the earlier fresh-vs-migrated conditional naming:
- New kafka.assertSingleInstance helper fails the render (with a clear message
and the upstream reference) if the namespace already holds another Kafka
cluster. Reconciles/upgrades of the same release are allowed; lookup returns
nothing during `helm template`, so unit tests never trip it.
- Node pools are now named "kafka" (broker) and "controller" unconditionally,
which also adopts existing brokers on migration and yields clean
<cluster>-kafka-N / <cluster>-controller-N names. The fresh-vs-migrated
brokerPoolName helper is removed.
- WorkloadMonitor pool-name selectors and the migration hook updated to match;
unit tests updated for the fixed pool names.
- README documents the one-Kafka-per-namespace rule and links the Strimzi
discussion.
The ZK->KRaft Chainsaw migration suite (seed ZooKeeper cluster, migrate, assert
the original broker PVC survives and no wrongly-named PVC is created, cluster
reaches KRaft) was run green against a live cluster.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Addresses the branch review on the ZK->KRaft migration path.
CRITICAL — broker node pool adoption. Strimzi derives broker/PVC names as
<cluster>-<pool>-<id>; a pre-node-pool ZooKeeper cluster has brokers
<cluster>-kafka-N, so the adopting broker pool must be named exactly "kafka"
or the operator provisions fresh empty brokers and orphans the tenant's data.
A new kafka.brokerPoolName helper resolves the broker pool to "kafka" for an
existing (migrating) cluster and keeps the release-scoped "<release>-broker"
for fresh installs, so multiple Kafka instances still coexist in one namespace.
The migration hook creates the pool as "kafka" and fails closed if a "kafka"
pool already belongs to another cluster (node pool names are namespace-unique,
so in-place migration of co-namespaced ZK clusters is unsupported). Wired
through kafkanodepools.yaml, workloadmonitor.yaml and the hook.
MAJOR — migration robustness.
- Raise the HelmRelease timeout: the migration Job can run tens of minutes but
the release timeout defaults to 10m, so helm-controller aborted and thrashed
the pre-upgrade hook. Add release.cozystack.io/helm-install-timeout=45m on the
kafka ApplicationDefinition, above the Job's activeDeadlineSeconds (2400s).
- Idempotent migration: do not re-issue kraft=migration once the cluster is past
the ZooKeeper phase (an invalid backward transition Strimzi rejects on retry).
- Fail closed on the decision gates: distinguish a genuine NotFound (skip) from
any other kubectl error (abort), and never treat an empty kafkaMetadataState on
an existing CR as "migrated".
MINOR — version downgrade documented as unsupported; integer (not lexical)
compare in the version gate; deleteClaim/PVC-retention and the one-per-namespace
migration limitation documented in the README; docs/storage-immutability.md
updated for KRaft.
Tests — helm-unittest coverage for the broker pool name, the hook guards and the
idempotent/fail-closed logic; a bats check for the helm-install-timeout budget
ordering; and a Chainsaw ZK->KRaft migration suite that seeds a ZooKeeper
cluster, drives the migration, and asserts the original broker PVC survives (not
recreated) and the cluster reaches KRaft. The migration sequence and the
Chainsaw assertions were validated against a live cluster.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Follow-up to the KRaft migration addressing the branch review.
- kafka-rd cozyrds: restore the two public CA certs in secrets.include
(clients-ca-cert + cluster-ca-cert). The rebase had collapsed them to
the bare clients-ca, which is Strimzi's clients-CA *private key* secret
— a private-key exposure to the tenant and a TLS regression (clients
need cluster-ca-cert to trust the broker listener). Covered by the
existing hack/check-kafka-rd-secrets.bats.
- migration-hook: guard the KRaft finalize step. The wait loop only exited
via break on success and fell through to unconditionally apply
strimzi.io/kraft=enabled on timeout — unsafe and irreversible if the
cluster is still in ZooKeeper/KRaftMigration. Add a migration_ready flag
and exit 1 when the safe state (KRaftPostMigration+) is not reached,
leaving the deployed-version ConfigMap unstamped so Helm retries. New
matchRegex assertions in migration-hook_test.yaml cover the guard.
- migration-hook: raise activeDeadlineSeconds 1800 -> 3600 so a
legitimately-progressing migration is not SIGKILLed before the in-script
waits and error paths can run.
- Extract the duplicated controller-quorum logic into a shared
kafka.controllerReplicas helper (used by kafkanodepools, workloadmonitor
and migration-hook) so the three stay in lockstep.
- Reflow the README migration section to one line per paragraph per the
repo prose rule.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Replace ZooKeeper-based Kafka with KRaft (Kafka Raft) metadata mode.
Strimzi 0.45 is the last line that still supports ZooKeeper, so this
prepares the transition before ZooKeeper support is removed upstream.
Chart changes:
- The Kafka CR always renders strimzi.io/kraft=enabled with node pools
and no spec.zookeeper block. Storage, replicas and resources move to
separate broker + controller KafkaNodePools (a homogeneous separate-pool
layout works for both fresh installs and migrations; Strimzi 0.45 does
not support migration into a combined pool).
- Kafka version is now user-configurable via the standard files/versions.yaml
pattern (v3.9 default, v3.8 available); appVersion bumped to 3.9.0.
- WorkloadMonitors track the broker and controller pools (no zookeeper
monitor); the dashboard RBAC grants the controller monitor and drops the
zookeeper one.
Migration of existing ZooKeeper clusters is handled by a Helm pre-upgrade
Job (templates/migration-hook.yaml) gated by a <release>-kafka-deployed-version
ConfigMap, following the established seaweedfs/etcd hook pattern. The Job
creates the broker + controller pools with Helm ownership labels, drives the
Strimzi ZK->KRaft state machine (node-pools=enabled, kraft=migration ->
kraft=enabled once KRaftPostMigration is reached) and is skipped once the
ConfigMap records version >= "1".
Testing:
- Helm unit tests cover the Kafka CR, node pools, migration hook, metrics,
topics, version map, dashboard RBAC and workload monitors.
- The e2e suite is ported to the current Chainsaw framework
(hack/e2e-chainsaw/kafka): it asserts the KRaft topology (both node pools,
kafkaMetadataState=KRaft), the absence of ZooKeeper, and bootstrap
endpoints.
Rebased onto current main: TLS listener support, entity-operator resource
guards, x-cozystack-options schema annotations and the WorkloadMonitor
replica-path fix are all preserved and reconciled with KRaft.
Signed-off-by: Matthieu <matthieu@hidora.com>
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
## What this PR does
The ClickHouse server and Keeper images were hardcoded to `24.9.2.42` in
the chart templates, so the chart exposed no top-level `version` values
key. The Cozystack API returns a managed database's engine version by
passing the HelmRelease's `spec.values.version` straight through into
the Application object, so ClickHouse instances carried no
`spec.version` in the API — unlike postgres, mariadb, mongodb,
opensearch, rabbitmq and redis, which all ship a `version` parameter.
This closes the ClickHouse half of the gap tracked in #1246.
This adds the same version-selection scaffolding the other
managed-database charts already use:
- `files/versions.yaml` maps a `major.minor` key to the full image tag,
shared by the ClickHouse server and Keeper images (both are pinned to
one version).
- `templates/_versions.tpl` (`clickhouse.versionMap`) resolves the tag
and fails the render on an unsupported version.
- `values.yaml` gains a `version` enum (`v25.8`, `v25.3`, `v24.9`),
consumed by `templates/clickhouse.yaml` and `templates/chkeeper.yaml`.
- `hack/update-versions.sh` + `make update` refresh the map from Docker
Hub, keeping only tags published for *both* the server and Keeper images
and preserving the current default so a regeneration never bumps the
major of existing deployments.
The default stays `v24.9` → `24.9.2.42`, so existing installations
render byte-for-byte identically and are never upgraded to a new major
implicitly. The generated `values.schema.json`, `README.md`, the Go API
type and the `clickhouse-rd` `openAPISchema` are regenerated with
`cozyvalues-gen` v1.6.0. A new `tests/version_test.yaml` helm-unittest
suite covers the server/Keeper mapping for each version and the
schema-level rejection of an unsupported value.
### Downstream repositories
The diff changes `packages/apps/clickhouse/values.schema.json` by adding
a new `version` field with an `enum`. Per the trigger map, this reaches
`terraform-provider-cozystack`, whose ClickHouse resource is
hand-written and would need a matching `version` attribute with a
`stringvalidator.OneOf` list and its expand/flatten pair. There is no
existing PR or issue there for it yet. This is left for a maintainer
decision rather than a speculative cross-repo PR, so the box below is
intentionally left unticked and flagged here. No other downstream
repository is reached (the website reference page is regenerated from
`README.md` by the release docs bot; ClickHouse is already in its app
list).
- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: needs a `version` attribute on the ClickHouse resource
(maintainer decision — see note above)
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
feat(clickhouse): add a configurable `version` parameter (v25.8, v25.3, v24.9; default v24.9) so the deployed ClickHouse version is selectable and returned through the API. Existing installations are unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configurable ClickHouse version setting for both server and
Keeper images, defaulting to `v24.9`.
* Supported versions are `v25.8`, `v25.3`, and `v24.9`, with compatible
image patches selected automatically.
* **Documentation**
* Documented the version setting, default, and supported values.
* **Validation**
* Unsupported versions are rejected with an error listing allowed
options.
* **Tests**
* Added coverage for version mapping, defaults, invalid values, and
update behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The `etcd` chart hardcoded only a **soft** `topologySpreadConstraints`
(`whenUnsatisfiable: ScheduleAnyway`) and never rendered
`spec.affinity`, even though the `EtcdCluster` CRD supports it. On small
clusters the scheduler therefore co-locates multiple members on one
node; losing that node takes out quorum (or the whole cluster).
We hit this in production: a single node reboot left several 3-member
etcd clusters with 2–3 members on the downed node, and one tenant's
control-plane etcd lost quorum until the node came back.
This renders `EtcdCluster.spec.affinity` from a new `affinity` value:
- **Default (value left empty):** the chart applies a **soft**
(preferred) `podAntiAffinity` on `kubernetes.io/hostname`, so members
spread across nodes on a best-effort basis. Soft on purpose — a hard
rule would leave a member `Pending` when replicas exceed schedulable
nodes, or when node-local storage pins a member to a node, so it is safe
to default on.
- **Override:** set `affinity` to any rule — e.g. a hard
`requiredDuringSchedulingIgnoredDuringExecution` podAntiAffinity — to
forbid co-location outright on clusters with enough nodes and
non-node-pinned storage. A supplied value replaces the default cleanly.
The existing `topologySpreadConstraints` block (and its
`_cluster.scheduling.globalAppTopologySpreadConstraints` override) is
unchanged.
Generated artifacts (`values.schema.json`, `README.md`, and the embedded
`openAPISchema` in `etcd-rd`) were regenerated with `make generate`;
`make test` passes (18/18).
### Screenshots
n/a — no UI changes.
### Downstream repositories
Walked the trigger map against the diff: this only adds one values key
to an app chart. No package is added/renamed/removed, `kind`/`plural`
are unchanged, no installer values or release assets or variants change,
and `README.md` is regenerated here (the website bot regenerates
reference pages from it on release). No manual downstream follow-up is
required.
- [x] No downstream repository is affected by this change
### Release note
```release-note
fix(etcd): etcd clusters now spread their members across nodes by default via a soft podAntiAffinity, and expose an `affinity` value so operators can set a hard rule; previously a single node could host a quorum of members and take out the cluster when it failed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable pod affinity rules for etcd deployments.
* By default, etcd members use soft anti-affinity to spread across hosts
without blocking scheduling.
* Custom affinity settings can override the default behavior.
* Exposed affinity configuration in the etcd application schema and
dashboard.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Bumps the vendored `cloudnative-pg` chart from 0.27.1 to 0.29.0
(operator + CRDs from 1.28.1-patched-to-1.28.2 up to 1.30.0) and drops
the hand-maintained `patches/cloudnative-pg-1.28.2.patch` entirely —
0.29.0 is a published chart, so the version-alignment patch is no longer
needed.
**Why this is required.** Database read-replica autoscaling (community
proposal `design-proposals/database-horizontal-autoscaling`) drives the
CNPG `Cluster` scale subresource with a stock HPA (via KEDA). That only
works when the subresource serves `status.selector`, which
[cloudnative-pg#8996](https://github.com/cloudnative-pg/cloudnative-pg/pull/8996)
added and shipped in CNPG 1.28.4 / 1.29.2 / 1.30.0. cozystack shipped
1.28.2, whose scale subresource has no selector, so the HPA fails with
`InvalidSelector`.
**Why 1.30.0 and not 1.28.4.** The floor is ≥ 1.28.4, but the upstream
chart repo publishes no chart for 1.28.4 (the 1.28 line stops at chart
0.27.1 / appVersion 1.28.1), so 1.28.4 could only be reached by
hand-patching the 0.27.1 chart's operator image and CRDs up to the
backport. A live PoC on a dev cluster proved that path is broken: the
1.28.4 operator running under the 0.27.1 (1.28.1) chart templates
crash-loops on a missing RBAC grant it needs but the older chart does
not provide — `endpointslices.discovery.k8s.io is forbidden: ... cannot
list resource "endpointslices"` — so `status.selector` is never
populated and the HPA still cannot scale. 1.30.0 (chart 0.29.0) is the
only published chart that carries the selector together with matching
RBAC/CRDs/templates, and the 1.28 line is EOL (2026-06-30), so 1.30.0
also keeps us on a supported release. cozystack uses the barman-cloud
plugin (not native barman, removed in 1.31) and no CNPG Poolers, so the
1.30 deprecations and the known pooler bug do not apply.
**Validation on a dev cluster (CNPG 1.30.0).** Operator rolls out clean
(no crashloop, the `endpointslices` RBAC error is gone); all existing
`Cluster` objects return to Ready; `/scale` now returns a populated
`status.selector` (`cnpg.io/cluster=<name>,cnpg.io/podRole=instance`);
an HPA pointed at a `Cluster` no longer reports `InvalidSelector`
(`AbleToScale=True, SucceededGetScale`); psql connectivity and streaming
replication verified.
The version-guard unittest is updated to 1.30.0 and now also asserts the
clusters CRD scale subresource carries `labelSelectorPath:
.status.selector`.
### Downstream repositories
Walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file. The change touches only the vendored
`cloudnative-pg` operator chart (CRDs, RBAC, values), the package
`Makefile`, and the version-guard test. The `packages/apps/postgres` app
and its `version` enums are unchanged,
`packages/core/platform/values.yaml` is unchanged, and the
`clusters.postgresql.cnpg.io` CRD is not one the terraform provider
types by hand (it travels as opaque JSON). No downstream repository
restates anything this diff changes.
- [x] No downstream repository is affected by this change
### Release note
```release-note
fix(postgres-operator): bump CloudNativePG to 1.30.0 so the Cluster scale subresource serves status.selector, enabling HPA-driven read-replica autoscaling (previously the HPA failed with InvalidSelector on 1.28.2)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Upgraded the CloudNativePG operator and Helm chart to 1.30.0/0.29.0.
* Added expanded image catalogs, pooler settings, pod selectors,
scheduling options, monitoring TLS, and LZ4 backup compression.
* Improved permissions for database roles, endpoint slices, and lease
monitoring.
* **Bug Fixes**
* Strengthened configuration validation and removed the deprecated
workload reference option.
* Updated compatibility requirements to Kubernetes 1.29 or newer.
* **Documentation**
* Expanded installation, configuration, verification, upgrade, and
uninstall guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The affinity value added in this branch introduced a two-branch conditional
in templates/etcd-cluster.yaml (default soft podAntiAffinity vs. a supplied
value) with no coverage. Add a helm unittest suite pinning both branches:
the default preferred per-host rule, and that a supplied affinity replaces
the default outright (no preferred rule leaks through).
Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
## What this PR does
Phase 2 of the Kubernetes-app split
([cozystack/community#8](https://github.com/cozystack/community/pull/8))
— the **breaking** half. Stacked on #3314 (review that first).
Removes worker node pools from the `kubernetes` chart (they moved to
`kubernetes-nodes` in #3314) and adopts existing pools on upgrade:
- `kubernetes` chart is now **control-plane only**: drops the pool
`range` block, the `talos-reconcile` Job + RBAC (deleted; moved to
`kubernetes-nodes`), the preserve-old-KMT block, the
`kubernetes.nodeGroups` helper, and the per-group dashboard RBAC.
- **`spec.nodeGroups` is removed from the `Kubernetes` CR** (BREAKING).
Worker pools are now separate `KubernetesNodes` resources.
- **Migration 54** (bumps `targetVersion` to 55): pre-upgrade hook that,
for every `kubernetes-<cluster>` release, creates one
`kubernetes-nodes-<cluster>-<pool>` HelmRelease and re-annotates the
existing worker objects onto it with `helm.sh/resource-policy=keep`, so
the now control-plane-only parent upgrade does not prune running
workers. The default `md0` (implicit when `nodeGroups` was empty) is
materialised explicitly. Idempotent. Genuine infrastructure/read
failures fail closed (`exit 1`, retried on the next upgrade); hostile
stored shapes (a pool name over Helm's 53-char release-name cap, an
RFC-1123-invalid or whitespace/slash map key, a non-object `nodeGroups`)
are warned, pinned, and skipped rather than deadlocking the entire
fleet's upgrade.
- The `ingressNginx` addon no longer auto-provisions a node (the
implicit `md0` default is gone). If enabled without a pool carrying
`roles: [ingress-nginx]`, the controller DaemonSet simply schedules
nothing (degraded, not failed) — the requirement is documented in the
chart NOTES.txt rather than enforced at render time.
Because the child chart renders the pool objects **byte-identically** to
the old parent render (golden parity in #3314; the
KubevirtMachineTemplate content-hash name is preserved), adoption does
not churn live worker VMs.
### Migration number
`54` is contiguous with main (latest is `53`) and `targetVersion` is
`55`. The `52`/#3201 collision noted in earlier revisions has been
resolved: this branch was rebased and merged past it, so the numbering
is contiguous and `run-migrations.sh` has no gap to hard-fail on.
### Not yet done (blocking draft → ready)
- [ ] **dev e2e upgrade validation** (`cozystack-pr-test`): upgrade a
custom-`nodeGroups` cluster and a default-`md0` cluster; assert zero
worker-VM churn (KMT hash, Machine uid/creationTimestamp stable) and
that the adopted child HRs reconcile.
- [ ] downstream trigger-map walk (website docs for the `nodeGroups` →
`KubernetesNodes` change; terraform-provider for the removed API field).
### Downstream repositories
Deliberately unticked (draft) — the breaking API change affects docs and
the terraform provider; follow-ups to be filed before ready.
- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
```release-note
Worker node pools of a managed Kubernetes cluster are now managed as separate `KubernetesNodes` resources. `spec.nodeGroups` on the `Kubernetes` CR is removed; existing pools are adopted automatically on upgrade.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Worker pools are now separate `KubernetesNodes` resources with their
own lifecycle.
* Added a render-time compatibility guard to prevent worker pools from
running ahead of the parent control-plane minor version.
* Improved upgrade/uninstall behavior to better preserve already-adopted
worker objects.
* **Breaking Changes**
* Removed per-pool `nodeGroups` and worker node health settings from the
`Kubernetes` chart/CR; pool configuration now lives in
`KubernetesNodes`.
* Ingress requires at least one `KubernetesNodes` pool with `roles:
[ingress-nginx]`.
* `cluster` and `storageClass` are immutable after creation; pool
`version` may lag but must not be ahead of the parent minor.
* **Documentation**
* Updated READMEs, notes, and schemas to reflect the `KubernetesNodes`
split and the immutability/version rules.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review of the version generator (hack/update-versions.sh):
- Fetch tags with `skopeo list-tags` (registry v2 /tags/list) instead of
the Docker Hub v2 API, whose anonymous pagination caps at 1000 entries
and then returns an error page with no `.results`, silently truncating
the list so the frozen 24.9.2.42 fell outside the window and the run
failed. skopeo is already in build-deps and is how the sibling
generators (postgres, redis, mongodb, opensearch, kubernetes) fetch.
- Make a frozen default tag missing from the registry a hard error
instead of silently falling back to a newer patch, which would move
the default image for existing installs.
- Restore 0644 on the regenerated files (mktemp's 0600 survived the mv).
- Exclude /hack from the chart tarball via .helmignore.
- Refresh files/versions.yaml: v25.8 -> 25.8.32.4, the true latest 25.8
patch now visible with the complete tag list.
- bats: add a missing-frozen-tag hard-error case and make the atomicity
test reach the resolution phase after a valid default rather than
failing at the default-in-set check.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The etcd chart only set a soft topologySpreadConstraints and never rendered
spec.affinity, so the scheduler could place several members of a 3-node
EtcdCluster on the same node; losing that node then took out quorum (observed
in production after a single node reboot).
Render EtcdCluster.spec.affinity from a new `affinity` value. When it is left
empty the chart applies a soft (preferred) podAntiAffinity on
kubernetes.io/hostname, so members are spread on a best-effort basis without
ever leaving a member Pending on small or node-pinned-storage clusters.
Operators can override the value — e.g. a hard requiredDuringScheduling rule —
where node count and storage allow.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Reword the values.yaml comment so it does not call appVersion 1.30.0 a
"published chart": the published chart version is 0.29.0 and it carries
appVersion 1.30.0. Keeping the two distinct stops a maintainer from running
`helm pull --version 1.30.0`.
Addresses CodeRabbit review feedback on #3951.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The nodeGroups docs still described the old denylist behaviour: "unlisted
NodeGroup fields pass through to the wrapped chart unvalidated". After the switch
to an allowlist (pick) both halves are wrong: fields go to the kubernetes-nodes
pool chart (not the wrapped kubernetes chart), and anything the pick does not name
is dropped, not passed through. A tenant that sets e.g. talos.version on a
nodeGroup gets no error and no effect (accepted-but-inert). Correct the source
@param and the hand-written README intro, regenerate schema/README/cozyrd, and
refresh the stale "omit" wording in the splice test comment to say allowlist.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
SSL_CERT_FILE alone does not narrow trust. crypto/x509 replaces only its file
list and then scans SSL_CERT_DIR unconditionally, and the pinned proxy image
ships /etc/ssl/certs/ca-certificates.crt, so the connection guarding the KEK
trusted the private CA plus every public root — a mis-issued certificate for the
Vault hostname was still accepted. Set SSL_CERT_DIR to the mount as well, which
collapses the pool to the CA the operator configured.
An unvalidated caBundle failed open on top of that: a non-PEM value is dropped
silently by AppendCertsFromPEM, leaving the proxy on the image's roots with no
error at all. Reject it at render time, the way backend and auth already are.
Rotating an inline caBundle had no effect either. The pod template did not
change, so no new pods, and crypto/x509 memoizes the pool per process — the
upgrade reported success while the old CA stayed in use until some later
restart. Derive a checksum/vault-ca annotation from the bundle so the rollout
happens with the change.
Also add vault.caSecretKey: the projected key was hardcoded to ca.crt, and a
bundle stored under another name (a cert-manager Certificate from an ACME issuer
emits no ca.crt at all) wedged the pod at mount time with no way to correct it
from values.
Drop the claim that the database leg verifies against its own CA pool: the chart
never sets KKP_BACKEND_CA_FILE, so that leg is plaintext today, as the note 30
lines above already says. The conclusion holds — it never consults the system
trust store — but for the other reason.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The volume and mount that consume the Vault CA sit at pod level, outside the
vault-transit env block, while $vaultCASecret was seeded from caSecretName
regardless of the backend. With kms.backend=static and caSecretName set, the
Deployment mounted a Secret the chart never renders, so the proxy — the single
choke point for all Keycloak DB traffic — would sit in ContainerCreating.
Gate both CA keys, and the mutual-exclusion check between them, on
vault-transit, so the whole vault.* subtree behaves the same way under static:
ignored. Project only ca.crt from a pre-seeded Secret, which keeps a
cert-manager tls.key out of the container and turns a bundle stored under
another key into a mount failure rather than an opaque TLS error later.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The proxy could only reach a Vault whose certificate chains to a publicly
trusted root, so a Vault fronted by an internal load balancer with a
self-signed certificate was unreachable over HTTPS.
Allow the CA to be supplied inline (caBundle, the chart renders the Secret) or
as a pre-seeded Secret (caSecretName), mounted and exposed as SSL_CERT_FILE.
That redirects only the system trust store, which nothing but the Vault client
consults — the database leg verifies against its own CA pool.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
- The unadopted-record WARNING said "N pool(s)" but counted SKIPPED_FILE lines,
and the foreign-owner branch writes one line per object (so a single pool with
MD+MHC+WM reported "3 pools"). Reword to "N record(s) ... one line per object
or pool ... not a distinct-pool count" so the number does not mislead.
- The inline-chart restamp comment implied the cozystack-apps source is enforced;
it is not (matching by chart name only is a deliberate trade-off: gating on the
historical sourceRef risks under-matching a real legacy parent, which loses its
workers, whereas the loose match's worst case is a harmless mislabel). Spell out
the trade-off and that the render guard is the data-loss backstop.
- The overflow comment cited "up to 42 chars per rest.go"; the enforced cap is 32
(maxKubernetesClusterName = 53 - 17 - 4). Correct it and note legacy/hand-created
names can still be longer, which is why the overflow branch exists.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The pool key becomes the pool HelmRelease name and its CAPI object names, which
must be RFC-1123 labels, but only the length was guarded. An uppercase/underscore
key rendered an invalid metadata.name the apiserver rejects, and a key containing
':' rendered invalid YAML that failed the whole module render -- a non-uniform,
late failure next to the guarded length case. Add a render-time RFC-1123 guard on
the key alongside the length guard (mirrors migration 56's child-name guard), and
pin the length guard's derived remedy budget (15) in the overflow test so a wrong
computation is caught, not just the presence of the error.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The restamp discovered unlabelled parent kubernetes releases by spec.chartRef
only, but that misses the population it exists for. The apps.cozystack.io/
application.kind label was introduced 2025-12-19 (fdca49838); spec.chartRef only
2026-01-14 (43da779ee). So any parent new enough to carry chartRef already
carries the label and is handled by the label-selected sweep, while every
genuinely unlabelled parent predates chartRef and instead carries an inline
spec.chart (chart name "kubernetes" from the cozystack-apps HelmRepository).
The chartRef-only selector therefore matched none of the real targets. Match both
shapes. This is a seamless-adoption improvement, not a data-loss fix: the chart's
own render-time guard already fail-closes (blocks the upgrade rather than pruning)
for any parent the restamp misses. Exact chart-name match plus the kubernetes-
name-prefix guard keep it from touching other apps or kubernetes-nodes children;
covered by an inline-shape restamp test and a non-kubernetes decoy test.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Three hardening fixes on migration 56's new code:
- The foreign-owner branch in adopt_one pins the object prune-proof but does not
adopt it (the child release can then stay Failed on the ownership conflict),
yet it was the one warn+pin+skip path that did not record itself in the durable
unadopted ConfigMap -- so an operator relying on that record to find stuck
pools would miss it. Record it like every sibling branch.
- record_skip ran a bare append under set -e inside skip branches that are
designed never to abort; a failed write (e.g. /tmp full) could deadlock the
fleet upgrade for one tenant. Make it fail-open, matching the branches' intent.
- The restamp selector matched any release by chartRef; scope it to names
starting with kubernetes- so it cannot mislabel (and then have the sweep skip)
a non-standard release, keeping restamp and the sweep's contract in sync.
Also fix the previously-vacuous negative restamp test (its fixture lacked the
chartRef, so it never reached the label guard it claimed to check).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The pool-value splice used omit (a denylist), which leaks every key it does not
name. Since this module runs untrusted tenant code and the NodeGroup schema has
no additionalProperties:false, a tenant could pass the platform-injected
_cluster object (cluster-domain/oidc/dns01/proxy, delivered via cozystack-values
and merged UNDER spec.values, so a tenant key wins) or nameOverride straight
through to the pool chart -- and any future kubernetes-nodes top-level value
would silently re-open the same hole. Switch to pick: pass through only the
enumerated per-pool shape fields, so everything cluster-level or module-owned is
dropped by construction (fail-closed for new fields). Also derive the
length-guard's release-name prefix from $clusterRelease so it cannot drift from
the name actually rendered. Regression-tested: _cluster and nameOverride no
longer reach the child values; maxUnhealthy and the module wiring still do.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The explanatory comment in values.yaml still described the removed
patches/cloudnative-pg-1.28.2.patch mechanism and the old 0.27.1/1.28.1
versions, and pointed at an "edit the patch" workflow that no longer exists.
Rewrite it to describe the native 0.29.0/1.30.0 chart bump (operator and CRDs
aligned by construction, no patch step) while preserving the still-valid
guidance: do not pin image.tag, keep the operator and CRDs in lockstep, since a
mismatched pin prunes CRD status fields and fails backups cluster-wide.
Addresses review feedback on #3951 (IvanHunters, CodeRabbit).
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The durable-record ConfigMap write sat under set -euo pipefail right before the
version stamp, so a permanent failure (e.g. the record exceeding etcd's object
size limit at a very large skip count) would abort the migration and deadlock
the fleet upgrade with no path out -- even though every skipped pool was already
pinned prune-proof earlier in the sweep, so the workers are safe regardless of
whether the record lands. The record is a convenience artifact, not a safety
mechanism, so swallow its failure with a warning instead of failing closed.
Covered by a bats case that fails the publish and asserts the run still stamps.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The guard's refusal message told the operator to run the platform migration,
but the case that reaches it is precisely one the migration already ran and
stamped for (its slot was renumbered out on a backport, or the parent
HelmRelease predates the application.kind label and the selector skipped it) --
re-running does nothing. Name the manual recovery instead: annotate
resource-policy=keep on the listed MachineDeployments, then stamp the
cozystack-version ConfigMap back below the worker-adoption migration so it
re-runs. Also note in the Phase 2 README that computeplane clusters render
their worker pools as separate KubernetesNodes releases too.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The worker-adoption sweep selected parents by the apps.cozystack.io/
application.kind label, which postdates v1.2. A parent kubernetes HelmRelease
older than that, or one restored by hand, lacks the label: the sweep skipped
it, stamped, and the parent's control-plane-only re-render then pruned its
un-pinned workers with no pin and no warning. Discover such releases by their
chartRef instead and restamp the three application labels before the sweep
(additive, cannot corrupt a release, and repairs it for cozystack-api too).
Every warn+pin+skip branch leaves a pool pinned prune-proof but unadopted,
recorded only in the Job log, which the next upgrade reaps; the migration does
not re-run, so an operator cannot find those pools afterward. Accumulate the
skips and publish them as configmap/cozystack-migration-56-unadopted before
the stamp.
Extend the fake kubectl and add bats coverage for the restamp (positive and
negative) and the unadopted-ConfigMap paths.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The per-pool child HelmRelease spliced the tenant nodeGroup dict verbatim
(toYaml) after setting cluster/clusterReleaseName, and the NodeGroup schema
has no additionalProperties:false. A tenant could therefore (a) set a
duplicate cluster/clusterReleaseName key that wins the YAML merge and
re-parents the pool onto a tenant-controlled cluster, or (b) pass talos/
version/images through a shape-only knob and pick the worker OS image,
installer and kubelet version. omit those five keys at the splice so only the
pool's own shape flows through (tolerated extras like maxUnhealthy still do).
Separately, each pool renders a 38-char fixed release-name prefix against
Helm's 53-char cap, so a nodeGroup key over 15 chars created a HelmRelease the
apiserver accepts but Flux never reconciles, with no other signal. Guard the
length at render time, next to the ingress-role guard, mirroring the migration
overflow branch and the aggregated API's validateNameLength.
Also refresh the generated values doc: the module now materialises its own
default md0 and pool storageClass falls back to the kubernetes-nodes chart
default, not the wrapped kubernetes app.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Adds client-certificate verification to the kube-ovn webhook, and turns
it on only when an operator supplies a CA bundle.
`--client-ca-file` is new and empty by default. With it empty the server
sets `tls.RequestClientCert`: it asks callers for a certificate, records
the issuer of whatever arrives, and serves the request either way. Set
the flag and the server switches to `tls.RequireAndVerifyClientCert`, so
a caller without a certificate signed by that CA is refused at the TLS
handshake.
**This is step one of two, and on its own it does not close
[GHSA-g883-q79m-8225](https://github.com/cozystack/cozystack/security/advisories/GHSA-g883-q79m-8225).**
With the default empty value nothing is rejected — the webhook still
answers an unauthenticated caller, and the only change is that the
caller's issuer now appears in the logs. That is deliberate: switching
straight to `RequireAndVerifyClientCert` would break every cluster whose
API server sends no client certificate, and there is no way to know from
here which those are. The logs are how we find out.
Step two, in a follow-up: read a day of issuer logs on a Talos cluster
and on `isp-full-generic`, confirm the API server presents a certificate
we can pin, then set the CA bundle so verification becomes mandatory.
The advisory should be published as fixed only after that lands — not
with this PR.
An earlier revision also carried a NetworkPolicy restricting ingress to
`kube-system`. It was dropped, and the title has been corrected to
match; only the TLS path is left here.
### Screenshots
N/A — no user-facing or UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
Walked the trigger map against the diff: no row matches. The change
touches one system package and the Go image it builds — no app package,
no `values.schema.json`, no `ApplicationDefinition`, no platform
variant, no node prerequisite.
### Release note
```release-note
feat(kubeovn-webhook): the webhook now accepts --client-ca-file. Left empty (the default) it requests and logs client certificates without requiring them; set to a CA bundle it rejects callers that do not present a certificate signed by that CA.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added optional client-certificate verification for the pod mutation
webhook through configurable CA settings.
* Added separate rate limits for requests with and without client
certificates.
* Added remote-address logging and clearer verification status
reporting.
* **Security**
* Requests continue to be forwarded while certificate observations are
recorded when verification is disabled.
* Serving-certificate updates remain supported; client CA changes
require a webhook restart.
* **Documentation**
* Documented client CA configuration and current certificate rotation
limitations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add a `version` value (v25.8, v25.3, v24.9; default v24.9) to the
ClickHouse chart, backed by files/versions.yaml and resolved by
templates/_versions.tpl for both the ClickHouse server and Keeper
images, so the Cozystack API returns the engine version (spec.version)
like the other managed-DB charts. Default v24.9 -> 24.9.2.42 keeps
existing installations byte-for-byte identical.
The maintainer-run hack/update-versions.sh generator intersects the
server/keeper Docker Hub tags byte-collated (LC_ALL=C) so comm keeps
tags across the X.9 -> X.10 boundary; freezes the default major to the
tag it already ships (read from versions.yaml, so the freeze follows the
default); errors on a dropped default or an unresolvable major; writes
atomically; and runs on stock macOS bash 3.2 / BSD. Covered by
hack/clickhouse-update-versions_test.bats (10 cases) and the
tests/version_test.yaml helm-unittest suite.
Refs: #1246
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Bump the vendored cloudnative-pg chart 0.27.1 -> 0.29.0 (operator + CRDs
1.28.1-patched-to-1.28.2 -> 1.30.0) and drop the hand-maintained
cloudnative-pg-1.28.2.patch entirely — 0.29.0 is a published chart, so no
version-alignment patch is needed.
Why this is required
--------------------
Database read-replica autoscaling (community proposal
design-proposals/database-horizontal-autoscaling) drives the CNPG Cluster
scale subresource with a stock HPA (via KEDA). That only works when the
subresource serves status.selector, which cloudnative-pg#8996 added and
shipped in CNPG 1.28.4 / 1.29.2 / 1.30.0. cozystack shipped 1.28.2, whose
scale subresource has no selector, so the HPA fails with InvalidSelector.
Why 1.30.0 and not 1.28.4
-------------------------
The floor is >= 1.28.4, but the upstream chart repo publishes no chart for
1.28.4 (the 1.28 line stops at chart 0.27.1 / appVersion 1.28.1), so 1.28.4
could only be reached by hand-patching the 0.27.1 chart's operator image and
CRDs up to the backport. A live PoC on a dev cluster proved that path is
broken: the 1.28.4 operator running under the 0.27.1 (1.28.1) chart
templates crash-loops on a missing RBAC grant it needs but the older chart
does not provide —
endpointslices.discovery.k8s.io is forbidden: ... cannot list resource
"endpointslices"
— so status.selector is never populated and the HPA still cannot scale. The
hand-patch would have to port the operator's RBAC (and any other template
delta) from 1.28.1 to 1.28.4 by hand, with no published chart to diff
against. 1.30.0 (chart 0.29.0) is the only published chart that carries the
selector together with matching RBAC/CRDs/templates, and the 1.28 line is
EOL (2026-06-30), so 1.30.0 also keeps us on a supported release. cozystack
uses the barman-cloud plugin (not native barman, removed in 1.31) and no
CNPG Poolers, so the 1.30 deprecations and the known pooler bug do not
apply.
Update the version-guard unittest to 1.30.0 and assert the clusters CRD
scale subresource carries labelSelectorPath: .status.selector.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Four of the five findings share one root cause: a node in the tenant
tree can attach to its nearest *visible* ancestor when its real parent
is inaccessible, and per-row actions then derived their target from
that bridged ancestor.
The Edit action on a bridged tenant row pointed at a Tenant CR that
does not exist. The CR lives in the real parent's namespace, so the
target now comes from realParentNamespace — and Edit is offered only
when that namespace is readable, because otherwise there is no CR the
user could open and the button could only ever dead-end. The test that
asserted two Edit buttons had frozen the broken target as expected
behaviour; it now names the rows it expects, and the Edit button gained
a title so that assertion means something.
An inherited module chip is rendered only when the providing tenant is
NOT visible, yet it was a live button: clicking it denied the instance
GET and switched the active tenant to a namespace outside the visible
set, which the selector then bounced to an arbitrary fallback. It is
now an inert badge that states where the module comes from.
InfoRedirect's missing-definition branch hardcoded /console, throwing an
admin out of the portal its own docstring promises to preserve.
The sidebar's alsoMatch used a bare startsWith, so /admin/apps would
light up for /admin/apps-v2. It now matches whole path segments.
And SecretsTab let a rejected clipboard write escape as an unhandled
rejection, leaving the button silent — a denied permission or an
insecure context was indistinguishable from a no-op. It now reports the
failure.
Declined: CodeRabbit asked to split `import { useK8sList, type K8sResource }`
into a separate `import type`. The inline modifier is valid under
verbatimModuleSyntax and is what 26 other imports in this console use,
so the change would make the file the odd one out.
Reported-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The kubernetes.nodeGroups helper that DEFAULT_MD0 reproduces was deleted in this
line of work, so nothing in-tree diffs against it and a future edit to the
constant (instanceType/diskSize/...) would silently rename the content-hashed
KubevirtMachineTemplate and roll every implicit-md0 pool. Reference the
pre-removal definition (git show 92b9785da^:.../kubernetes/templates/_helpers.tpl)
so the canonical default stays diffable.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Phase 2 moved worker nodeGroups out of apps/kubernetes into apps/kubernetes-nodes,
but the ComputePlane module still passed nodeGroups to the wrapped kubernetes
chart, which now ignores it: a fresh ComputePlane cluster came up with zero
workers (and the pinned ingress-nginx addon stuck Pending), while existing ones
were unreachable by migration 56 (it only adopts kubernetes-* releases, not
computeplane-cluster).
The module now emits one kubernetes-nodes child HelmRelease per nodeGroup,
shaped like a natively created pool, and materialises the pre-split default md0
(roles: [ingress-nginx]) when nodeGroups is empty. The pool chart's clusterName
gains an internal clusterReleaseName override so pools attach to the module's
fixed computeplane-cluster release, which is off the aggregated API's
kubernetes-<cluster> convention; the default keeps every API-created pool
byte-identical.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Opening the dashboard without a path sent the user to the catalog, and
the header gave Marketplace the primary emphasis. The Console is the
better entry point: it shows what the tenant already runs, which is
what someone opening the dashboard almost always came for. The
Marketplace stays one click away for when the intent is to add
something new.
The landing path becomes a named constant beside the portal helper, so
the redirect and the tab emphasis cannot drift apart, and the library's
own default tab set is reordered to match rather than contradict its
only consumer.
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The Kubernetes CR still accepts and stores spec.nodeGroups, spec.nodeHealthCheck
and spec.maxNodeProvisionTime after the Phase 2 split (the adoption migration
leaves them on an upgraded CR rather than rewriting parent values mid-upgrade),
but they no longer render anything, so an operator scaling via
spec.nodeGroups.<x>.minReplicas silently got nothing.
Emit a non-blocking client-facing admission warning on Create/Update when any
of these fields is present, pointing the operator at the KubernetesNodes
resources. A hard rejection is intentionally avoided: it would break edits to
already-upgraded clusters that still carry the fields. Also enumerate all the
Phase 2 breaking changes (three removed fields, two KubernetesNodes
immutability rules, the 32-char cluster-name cap) in the chart README.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The merge renumbered the worker-pool adoption migration from 54 to 56, but
six references still named 54 (README, pre-delete-unpin hook, cluster.yaml,
worker_adoption_guard test), which now points at main's redis-operator
adoption migration. Repoint them at 56.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The 'emits no worker-pool objects' assertions used 'not: true' with
'documentIndex: -1', which scopes the check to the last rendered document
only, so a stray MachineDeployment/KubevirtMachineTemplate/MachineHealthCheck/
TalosConfigTemplate anywhere else passed silently. Move 'any: true' inside
the containsDocument matcher and drop documentIndex so the negation covers
every document. Verified the asserts now redden when a worker object is
present.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Copying a secret key required revealing it on screen first, and the
button gave no sign that anything reached the clipboard. The value the
list already carries is enough to copy, so the button works while the
key stays masked, and it briefly turns into a check mark to confirm.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Detail pages offered a fixed tab set, so Info showed Workloads, Services
and Ingresses tabs it never fills; tabs now appear only for resource
groups the instance actually owns. The probes reuse the tabs' own list
refs and label selector, so they share one cache entry and one watch.
Back and the sidebar followed the plural instead of the entry point: a
module page sent the user to a one-item list and lit nothing in the
sidebar. Module pages now return to Modules and keep it highlighted,
while Info returns to Tenants and highlights that, matching the trees
they are reached from.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The page showed LoadBalancer services of the selected tenant only, so an
operator had to switch tenants one by one to find an address. It now
lists them for every visible TenantNamespace in a single table with a
Namespace column. A namespace whose services the user cannot list
contributes no rows instead of failing the page.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The Modules page listed the selected tenant's modules as Enabled or
Disabled cards, so a module a parent tenant provides looked disabled and
clicking it led to a Tenant CR that does not exist in the child's
namespace — an endless spinner.
It now covers every tenant the user can access, as the same hierarchy
tree the Tenants page draws, with each tenant's modules as chips
carrying the module icon, the namespace running it and its readiness.
Availability comes from the `namespace.cozystack.io/<module>` label on
each TenantNamespace, which names the providing namespace: the tenant
itself for a local module, an ancestor for an inherited one. An
inherited chip appears only when that ancestor is out of the user's
reach, since otherwise its own chip already shows the module. Clicking
any chip opens the module in whichever tenant runs it.
Tenants whose whole subtree has no modules collapse into a
"+ N tenants" branch, and the Info module is left out entirely as every
tenant carries it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The flat table becomes a tree built from TenantNamespaces, whose
`tenant.cozystack.io/<ancestor-ns>` labels carry the full ancestry. A
tenant whose parent is not accessible attaches to its nearest visible
ancestor instead of disappearing, and rows are labelled relative to that
parent, so a sub-tenant reads as `crpjxhwm` rather than
`whmcs-crpjxhwm`. Subtrees collapse from the row itself.
Each row also carries its own actions: Info opens that tenant's Info
page, and Create Tenant orders a sub-tenant at any level of the tree.
Editing a tenant switches to the namespace holding its Tenant CR — the
parent's, or its own for the hierarchy root.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Administration (Tenants, Modules, External IPs, per-tenant Info) leaves
the Console sidebar for Admin, which is now always visible: those pages
need no special permission. The two cluster-wide operator areas keep
their own gates, so Capacity and Backup Classes still appear only for
users who can open them.
Resource routes become portal-aware through a shared base-path helper,
so a detail, edit or order page opened from Admin stays under /admin
instead of jumping back to /console.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Open the grouped application list right away instead of the
"Choose a product to deploy" page and its Show-all-apps button.
Category links keep filtering to a single category.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Review found the observation could be silenced and the feature could not
be turned on. Both are fixed here; enforcement itself stays out of scope
for this step.
Rate-limit the observation instead of remembering issuers. Keying on the
issuer was the mistake: with RequestClientCert that string comes from the
caller, so a bounded set of first-seen values can be filled with 32
fabricated issuers before the API server's first call, permanently
suppressing the signal the feature exists to produce, while an unbounded
one grows without limit. A rate limit cannot be starved -- a burst costs
the current interval and the next real handshake is logged in the one
after. The two outcomes are limited separately so a flood of one cannot
hide the other, and both lines now carry the peer address, since 'none
presented' fires for any certless caller and was not attributable.
Say 'verified' rather than 'UNVERIFIED' once enforcement is on, where the
handshake really has checked the chain.
Stop telling operators to point --client-ca-file at an in-pod path. No
such path exists: the pod mounts only its own serving secret, whose
ca.crt is this webhook's serving CA, so every value either aborts startup
or rejects the API server -- and with failurePolicy: Fail that stops pod
creation cluster-wide. Say what is actually true and defer enabling to
the step that adds the volume.
Correct the claim in cert.go that this webhook configures no ClientCAs.
It can now, that pool is read once at startup and never reloaded, and the
comment justified skipping reload on the old premise. Record that
reloading has to land together with the mount, since neither is usable
alone.
Cover the observation with tests: rate limiting, the burst it exists to
survive, concurrency, both branches, the enforcing wording, and that
admission is never blocked by logging.
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Both points came out of review of this branch, and both live in the
observation path added for the staged rollout.
The set of seen issuers was unbounded. With tls.RequestClientCert the
server accepts whatever certificate a caller offers, so the issuer string
is supplied by that caller: anyone able to open a connection could mint
fresh issuers and grow the map and the log without limit. Cap it, and say
once when the cap is reached -- a cap hit on a quiet cluster is itself the
signal that the issuers are fabricated.
The log also invited the operator to pin '--client-ca-file' to whichever
CA it had just printed. That CA is unverified by construction, so an
attacker who can reach the port can propose one. Say plainly in the log
and in the comment that the value is a claim rather than an identity, and
that it has to be confirmed out of band. Enforcement still depends only
on --client-ca-file, which is trusted control-plane configuration.
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Review of the original commits found both hardening mechanisms inert, and
that was correct: the NetworkPolicy could never be enforced because
packages/system/kubeovn/values.yaml sets ENABLE_NP: false, so
kube-ovn-controller runs with --enable-np=false and programs no ACLs; and
--client-ca-file was never passed by the chart, so the
RequireAndVerifyClientCert branch was unreachable. Only the TLS 1.2 floor
did anything.
The NetworkPolicy is dropped rather than fixed. Enforcing it would mean
turning on ENABLE_NP for the whole cluster, which is a far larger change
than this disclosure warrants, and shipping an object that looks like a
control but enforces nothing is worse than not shipping it.
Client-cert verification is now reachable: clientCAFile is a chart value and
the flag is passed when set. It stays off by default, because turning it on
blind is dangerous — this webhook is registered failurePolicy: Fail, so if
the API server does not present a certificate (it only does when the cluster
runs an AdmissionConfiguration naming a kubeConfigFile for this webhook),
requiring one stops pod creation cluster-wide.
So the default is tls.RequestClientCert: certificates are requested and
logged, never required. Nothing is rejected and admission cannot break,
while the log states per issuer whether enforcement is possible on this
cluster. Setting clientCAFile then flips it to RequireAndVerifyClientCert.
Addresses GHSA-g883-q79m-8225. This commit makes the fix deployable and
observable; enforcement is a follow-up once the logs confirm the API server
presents a certificate on each supported variant.
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Alexey Artamonov