Add a @section Version parameters marker in values.yaml so that the version
parameter is grouped separately from TLS parameters in the generated README
and schema. Explicitly set tls.required: true in values.yaml so UI clients
and schema consumers see the correct default (true, matching the template
behaviour documented in the description field). Also add
[\"spec\",\"tls\",\"enabled\"] and [\"spec\",\"tls\",\"required\"] to keysOrder in
mariadb-rd so TLS child fields render in a stable order in the dashboard.
Regenerate codegen artifacts accordingly.
Assisted-by: LLM
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Run root and per-package make generate to bring deepcopy, types, README, and CRD
descriptor in sync with values.yaml annotations after the TLS work landed.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- Add tls.enabled (*bool) and tls.required fields to API types and values
- Update JSON schema with nullable tls.enabled tri-state and required flag
- Extend Makefile with test target wiring helm-unittest into CI
- Update cozyrds descriptor to expose new TLS fields
- Update README to document tls.required semantics and nullable type
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Fixes#2985. Editing a `VMDisk`'s `storageClass` or `source` after its
DataVolume already exists was silently ignored:
`packages/apps/vm-disk/templates/dv.yaml` looks up the existing
DataVolume and, when found, reused its `spec` verbatim (because
`DataVolume.spec` is immutable in CDI). The edit therefore rendered as a
no-op — the typed CR accepted it and the HelmRelease reported success,
while the DataVolume/PVC kept the original spec and the source of truth
silently diverged from the cluster.
The template now builds the desired `source` into a shared
`$desiredSource` variable (used both for rendering and comparison) and,
when a DataVolume already exists, calls `fail()` with an explicit
message if the requested `storageClass` or `source` differs from the
immutable existing spec — mirroring the explicit size handling already
done in `pvc-resize-hook.yaml`. Storage growth is unaffected and still
flows through the resize hook.
`source` is also marked `@immutable` in `values.yaml`, so the generated
schema now carries the same `x-kubernetes-validations` transition rule
(`self == oldSelf`) that `storageClass` already had; `types.go`,
`values.schema.json` and the generated `vm-disk-rd` CRD are regenerated
to match.
### Verification
- `helm unittest` covers both the immutability guard (via a mocked
existing DataVolume through `kubernetesProvider`, since `lookup` needs a
cluster) and initial-creation source rendering for every source type —
12/12 pass.
- Verified live on a dev cluster with `helm install --dry-run=server`
against a real DataVolume: an unchanged edit renders, a `storageClass`
change fails, a `source` change fails, and a storage grow still renders
the resize hook without tripping the guard.
### Scope note
This PR only makes the drift visible and marks `source` immutable. Two
deeper platform findings sit outside its scope: (1) #2657 (pre-existing)
— the `apps.cozystack.io` aggregated apiserver does not evaluate the
`x-kubernetes-validations` CEL rules on update, so `@immutable` is a
no-op at admission for every Application kind, which is exactly why the
chart-level `fail()` — not the CEL rule — is the substantive guard here;
(2) #3971 — that same apiserver does not honour `--dry-run=server` and
persists the change (surfaced while testing this fix).
### Downstream repositories
Walked the trigger map in `docs/agents/contributing.md` against the diff
file by file. The diff touches only `packages/apps/vm-disk/` (template,
`values.yaml`, regenerated `values.schema.json`, tests), the regenerated
`packages/system/vm-disk-rd/cozyrds/vm-disk.yaml`, and
`api/apps/v1alpha1/vmdisk/types.go`. No package was added, renamed or
removed; no `values.schema.json` field was added, renamed or removed
(only a validation rule on the existing `source` field); no default
changed; `README.md` is unchanged; the `cozyrds` file was regenerated,
not renamed. The terraform provider models fields, defaults and enums
but not CEL immutability, so it is unaffected. No downstream repository
is reached by this change.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
fix(vm-disk): editing storageClass or source on an existing VMDisk now fails with a clear message instead of being silently ignored; delete and recreate the disk to change these immutable fields.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for blank, HTTP, image, disk, and upload VM disk
sources.
* VM disk sources and storage classes are now protected from unintended
changes, while storage expansion remains supported.
* Existing disks preserve their configured source and storage settings.
* **Bug Fixes**
* Invalid or conflicting source configurations are rejected.
* Prevented silent no-op updates and incorrect immutability errors for
existing disks.
* **Documentation**
* Documented source and storage class immutability, including the need
to recreate disks when changing either setting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add a managed Valkey service mirroring the managed Redis app. Valkey is the
BSD-3-Clause, Linux Foundation-governed fork of Redis 7.2 and a drop-in
replacement for it, offered as a governance-clean alternative to the now
source-available (AGPLv3/SSPL/RSALv2) Redis 8.x.
Valkey runs on the same Spotahome redis-operator: the operator hardcodes the
redis-server command and redis-cli in its (non-overridable) probe scripts, so
the chart deploys a thin compat image (FROM valkey/valkey:<ver>) that adds the
redis-* symlinks over the official upstream binaries — no Valkey code rebuilt.
Versions offered: v8 (8.1.8, default) and v7 (7.2.13, the BSD Redis 7.2 line).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Editing a VMDisk's storageClass or source after its DataVolume already
exists was silently ignored: dv.yaml reused the existing DataVolume.spec
verbatim (DataVolume.spec is immutable in CDI), so the edit rendered as a
no-op while the HelmRelease reported success and the CR diverged from the
cluster.
When a DataVolume already exists, fail with a clear message if the
requested storageClass or source differs from what the disk was created
with. Both compare the operator's input recorded in an annotation at
creation (vm-disk.cozystack.io/storage-class and .../source), written only
at creation and preserved verbatim afterwards — never overwritten with an
unapplied edit. Recording the input rather than reading the live object
matters on both axes: the image-clone PVC name carries a chart-version
prefix (vm-image- was renamed to vm-default-images-, commit bec35e3a) that
drifts across upgrades with no edit, and the live PVC storageClassName of an
adopted or foreign DataVolume may legitimately differ from the CR default
without being a change. The annotation key presence (not its value) marks a recorded class, so a disk
created with the cluster default "" is recorded and still caught if later
changed to a concrete class.
Disks with no annotation (created earlier, or out-of-band) are handled
conservatively so a working release never deadlocks: source is compared
against the stored spec.source only when that stored source is a shape the
chart itself produces, so an adopted DataVolume using spec.sourceRef or a
non-chart source (registry, snapshot, http+secretRef, ...) is left alone
rather than deadlocked. The image-clone exemption fires only when the stored
source is a cozy-public pvc AND the request is itself an image, so a change
away from an image source to another type is still caught. storageClass has
no safe live fallback, so an un-annotated disk is not guarded here (apiserver
CEL, #2657, is the eventual guard). An unrecognized source type
now fails explicitly. Storage growth is unaffected and still handled by
pvc-resize-hook.yaml.
Mark source as immutable in values.yaml so the generated schema carries
the same x-kubernetes-validations rule already present on storageClass,
and document the contract, the exemption, and the source-annotation schema
coupling in docs/storage-immutability.md and the chart README.
Add helm-unittest coverage for the immutability guard (rejection incl.
clearing storageClass to empty and changing a non-image disk to image,
no-op reuse with annotations preserved, legacy image-disk drift, annotation
non-poisoning, un-annotated storageClass skip, adopted default-class disk,
unknown source) and initial-creation source rendering.
Fixes#2985
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
## What this PR does
An in-place `RestoreJob` (one whose `targetApplicationRef` is omitted,
so the app is restored back into itself) re-bootstraps the Postgres app
under its own name. On stock v1.6.2 it never comes back up — the cluster
is deleted and then hangs in `Setting up primary` with
`*-full-recovery-*` pods crash-looping, or the HelmRelease upgrade fails
outright. There are two independent causes, both in the same restore
patch, and both surface only in place because a to-copy restore targets
a different app.
**1. Archive-vs-recovery `serverName` collision.** The driver set the
restored cluster's WAL-archive `serverName` equal to the recovery source
(`bootstrap.serverName = bootstrap.oldName = sourceServerName`), while
the chart pinned the archiver `serverName` to the release name. In place
those coincide, so the cluster archived onto the source's own WAL prefix
and `barman-cloud-check-wal-archive` refused to start with `Expected
empty archive`. Fixed by giving the restored cluster a fresh
`bootstrap.newServerName`, keyed by the `RestoreJob` UID (unique per
restore, stable across reconciles); the chart archives there while
`externalClusters` still recovers from `bootstrap.serverName`. This is
what manual `CNPG` strategies hit (their `serverName` defaults to the
cluster name, including the shipped
`examples/backups/postgres/10-cnpg-strategy.yaml`); the `cozy-default`
flow dodged it because its `serverName` is `<namespace>-<app>`.
**2. `useSystemBucket` guard.** The restore populates explicit S3
coordinates (`destinationPath` / `s3CredentialsSecret` / `endpointCA`)
but left `backup.useSystemBucket=true`, so the chart's
`bootstrap.enabled is incompatible with useSystemBucket` guard failed
the HelmRelease upgrade and the cluster never re-rendered. This is what
the default `cozy-default` (system-bucket) flow hits on an in-place
restore. Fixed by clearing `useSystemBucket` in the restore patch, since
the app is now on the explicit-creds path. Verified live on a v1.6.2
cluster: with `useSystemBucket` cleared, the same in-place restore
re-bootstraps to a healthy cluster and the sentinel row survives.
Both are exercised only by an in-place restore; the postgres backup e2e
drove only to-copy, so neither surfaced in CI. A step 50 in-place
restore now guards cause 1 (manual-strategy flow); cause 2 is covered by
a Go unit assertion (a system-bucket in-place e2e needs the platform
bucket wired in CI and is left as a follow-up).
This is a bug in released v1.6.2 (chart and driver are byte-identical on
`main`), so it is a backport candidate.
### Screenshots
Not applicable (no UI changes).
### Downstream repositories
The only externally visible schema surface is one new field,
`spec.bootstrap.newServerName`, in the postgres `values.schema.json` /
CRD; it is set by the CNPG backup driver on restore and never authored
by tenants, exactly like the existing `bootstrap.serverName` /
`bootstrap.oldName`. (The `useSystemBucket` clearing is internal to the
Go driver — `backup.useSystemBucket` already exists in the schema.) The
chart `README.md` is regenerated from `values.yaml` automatically by the
website docs bot on release, and `terraform-provider-cozystack` models
user-authored spec (its guard tracks top-level fields, not driver-set
nested ones), so neither needs a change in practice. Leaving every box
unticked per the trigger-map guidance so a maintainer confirms rather
than acting on a speculative follow-up.
- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
fix(postgres): in-place restore no longer wedges — the restored cluster archives WAL under a fresh serverName distinct from the recovery source, and a system-bucket app is switched to explicit credentials so the bootstrap/useSystemBucket guard no longer fails its HelmRelease.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for specifying a distinct WAL archive destination when
restoring PostgreSQL clusters.
- In-place restores now automatically use a fresh archive location while
continuing to recover from the original source.
- Added configuration and schema support for the optional
`bootstrap.newServerName` setting.
- **Bug Fixes**
- Prevented archive conflicts that could cause in-place restores to
fail.
- **Documentation**
- Documented the new restore archive configuration and behavior.
- **Tests**
- Added coverage for unique restore destinations and in-place restore
validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The ClickHouse server and Keeper images were hardcoded to `24.9.2.42` in
the chart templates, so the chart exposed no top-level `version` values
key. The Cozystack API returns a managed database's engine version by
passing the HelmRelease's `spec.values.version` straight through into
the Application object, so ClickHouse instances carried no
`spec.version` in the API — unlike postgres, mariadb, mongodb,
opensearch, rabbitmq and redis, which all ship a `version` parameter.
This closes the ClickHouse half of the gap tracked in #1246.
This adds the same version-selection scaffolding the other
managed-database charts already use:
- `files/versions.yaml` maps a `major.minor` key to the full image tag,
shared by the ClickHouse server and Keeper images (both are pinned to
one version).
- `templates/_versions.tpl` (`clickhouse.versionMap`) resolves the tag
and fails the render on an unsupported version.
- `values.yaml` gains a `version` enum (`v25.8`, `v25.3`, `v24.9`),
consumed by `templates/clickhouse.yaml` and `templates/chkeeper.yaml`.
- `hack/update-versions.sh` + `make update` refresh the map from Docker
Hub, keeping only tags published for *both* the server and Keeper images
and preserving the current default so a regeneration never bumps the
major of existing deployments.
The default stays `v24.9` → `24.9.2.42`, so existing installations
render byte-for-byte identically and are never upgraded to a new major
implicitly. The generated `values.schema.json`, `README.md`, the Go API
type and the `clickhouse-rd` `openAPISchema` are regenerated with
`cozyvalues-gen` v1.6.0. A new `tests/version_test.yaml` helm-unittest
suite covers the server/Keeper mapping for each version and the
schema-level rejection of an unsupported value.
### Downstream repositories
The diff changes `packages/apps/clickhouse/values.schema.json` by adding
a new `version` field with an `enum`. Per the trigger map, this reaches
`terraform-provider-cozystack`, whose ClickHouse resource is
hand-written and would need a matching `version` attribute with a
`stringvalidator.OneOf` list and its expand/flatten pair. There is no
existing PR or issue there for it yet. This is left for a maintainer
decision rather than a speculative cross-repo PR, so the box below is
intentionally left unticked and flagged here. No other downstream
repository is reached (the website reference page is regenerated from
`README.md` by the release docs bot; ClickHouse is already in its app
list).
- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: needs a `version` attribute on the ClickHouse resource
(maintainer decision — see note above)
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
feat(clickhouse): add a configurable `version` parameter (v25.8, v25.3, v24.9; default v24.9) so the deployed ClickHouse version is selectable and returned through the API. Existing installations are unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configurable ClickHouse version setting for both server and
Keeper images, defaulting to `v24.9`.
* Supported versions are `v25.8`, `v25.3`, and `v24.9`, with compatible
image patches selected automatically.
* **Documentation**
* Documented the version setting, default, and supported values.
* **Validation**
* Unsupported versions are rejected with an error listing allowed
options.
* **Tests**
* Added coverage for version mapping, defaults, invalid values, and
update behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
An in-place RestoreJob (targetApplicationRef omitted) re-bootstraps the
Postgres app under its own name. On stock v1.6.2 it never comes back up,
for two independent reasons in the same restore patch - both surface only
in place, because a to-copy restore targets a different app.
1. Archive-vs-recovery serverName collision. The driver set the restored
cluster's WAL-archive serverName equal to the recovery source
(bootstrap.serverName = bootstrap.oldName = sourceServerName) while the
chart pinned the archiver serverName to the release name. In place those
coincide, so the cluster archived onto the source's own WAL prefix and
barman-cloud-check-wal-archive refused to start ("Expected empty
archive"). Fixed by giving the restored cluster a fresh
bootstrap.newServerName, keyed by the RestoreJob UID (unique per
restore, stable across reconciles); the chart archives there while
externalClusters still recovers from bootstrap.serverName. This is what
the manual CNPG strategies hit (serverName defaults to the cluster
name); the cozy-default flow dodged it (serverName is <ns>-<app>).
2. useSystemBucket guard. The restore populates explicit S3 coordinates
(destinationPath / s3CredentialsSecret / endpointCA) but left
backup.useSystemBucket=true, so the chart's "bootstrap.enabled is
incompatible with useSystemBucket" guard failed the HelmRelease upgrade
and the cluster never re-rendered. This is what the cozy-default
(system-bucket) flow hits on an in-place restore. Fixed by clearing
useSystemBucket in the restore patch, since the app is now on the
explicit-creds path. The driver's app view (postgresapp.Backup) gains
the field, without omitempty so a false survives the merge patch.
Both are exercised only by an in-place restore; the postgres backup e2e
drove only to-copy, so neither surfaced in CI. A step 50 in-place restore
now guards reason 1 (manual-strategy flow); reason 2 is covered by a Go
unit assertion (a system-bucket in-place e2e needs the platform bucket in
CI and is left as a follow-up).
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a `version` value (v25.8, v25.3, v24.9; default v24.9) to the
ClickHouse chart, backed by files/versions.yaml and resolved by
templates/_versions.tpl for both the ClickHouse server and Keeper
images, so the Cozystack API returns the engine version (spec.version)
like the other managed-DB charts. Default v24.9 -> 24.9.2.42 keeps
existing installations byte-for-byte identical.
The maintainer-run hack/update-versions.sh generator intersects the
server/keeper Docker Hub tags byte-collated (LC_ALL=C) so comm keeps
tags across the X.9 -> X.10 boundary; freezes the default major to the
tag it already ships (read from versions.yaml, so the freeze follows the
default); errors on a dropped default or an unresolvable major; writes
atomically; and runs on stock macOS bash 3.2 / BSD. Covered by
hack/clickhouse-update-versions_test.bats (10 cases) and the
tests/version_test.yaml helm-unittest suite.
Refs: #1246
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The main merge kept main's GPU DeepCopy funcs, but the Phase 2 chart split
removed the worker nodeGroups (and their gpus field) from the kubernetes
chart values, so cozyvalues-gen no longer emits `type GPU` in types.go.
The orphaned DeepCopy referenced an undefined type and broke the Go build
and the generated-code drift check. Regenerate: no GPU deepcopy.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Resolve conflicts introduced by main:
- Drop the FluxCD tenant addon (API type, deepcopy, values schema, RD) in
favour of the GPU type main added; main removed the fluxcd addon wholesale.
- Renumber the Phase 2 worker-pool adoption migration 54 -> 56, behind main's
redis-operator (54) and fluxcd-removal (55) migrations; targetVersion 55 -> 57.
- Accept the branch's removal of the in-chart talos-reconcile Job; it moved to
the kubernetes-nodes chart, main only touched its comment.
- Regenerate kubernetes values.schema.json / README and the kubernetes-rd
ApplicationDefinition schema.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The optional `addons.fluxcd` toggle of the Kubernetes app deployed a
Flux Operator + FluxInstance into a tenant cluster. It was served by two
vendored charts, packages/system/fluxcd-operator and packages/system/
fluxcd, which were the last AGPL-3.0 content in this Apache-2.0
repository and a governance blocker for CNCF Incubation.
The management cluster does not use these charts (it runs classic Flux
via internal/fluxinstall; migration 21 already removed the operator from
it), and the addon defaulted to disabled. Rather than maintain an AGPL
payload for an unused feature, drop it entirely:
- delete both system packages and their platform source entries
- remove the addons.fluxcd HelmReleases, values and generated schema
from the Kubernetes app
- drop the addons.fluxcd block still submitted by the e2e harness
- drop the removed fluxcd package entries from CODEOWNERS
BREAKING CHANGE: the addons.fluxcd option of the Kubernetes app is
removed. Tenant clusters that had it enabled keep their running Flux
release; see the companion migration for the upgrade path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Reconcile the phase-2b split with main's configurable worker-health
feature (MHC condition timeouts, maxNodeProvisionTime, podCpuLimit/
podCpuRequest), which landed on both charts after this branch forked:
- kubernetes (parent): pool rendering stays removed; main's parent
worker-pool additions do not apply post-split.
- kubernetes-nodes (pool): absorb the feature; keep the directional
parent-version skew guard and the immutable cluster/storageClass rules.
KubevirtMachineTemplate content hashes are unchanged (render snapshot).
- migration 54: carry podCpuLimit/podCpuRequest (KMT-hash inputs) and the
cluster-wide readyUnknownTimeout/readyFalseTimeout/maxNodeProvisionTime
into adopted pools, so adoption neither rolls nor resets a tuned pool.
Covered by a new adopt-values bats case.
- drop the parent chart's obsolete MHC-timeout suites; the
kubernetes-nodes twins cover the feature.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The CAPI webhook validates nodeStartupTimeout with a 30s floor but does
not validate unhealthyConditions timeouts at all, so a zero or negative
readyUnknownTimeout or readyFalseTimeout applied and remediated a
Machine the moment its Ready condition flipped. Route both fields
through the same render-time duration guard as maxNodeProvisionTime,
extracted into a kubernetes.positiveDuration helper in each chart.
The guard now also accepts the canonical form a duration serializes
back to ("20m0s", "1h0m0s"): metav1.Duration prints that shape, so a
value copied from kubectl output previously failed the render. The
first segment must still be positive and later segments may be zero,
which keeps every accepted string a positive duration.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
nodeStartupTimeout is the window a Machine has to produce a Node at
all, measured from the later of the Machine's creation and its
InfrastructureReady transition. What has to fit inside it is the guest
boot and the kubelet registering with the tenant apiserver, and on a
host under load neither is quick. At 10m a worker that was still
booting was deleted before it ever joined, and its replacement started
the same race from the beginning.
Also record what a percentage maxUnhealthy does at small group sizes,
because the interaction is not what the field name suggests: the value
is scaled against the machines the check covers and rounded down, and
remediation proceeds only while the unhealthy count is at or below the
result. At the default "50%" a group of one never remediates, and a
group of two remediates one unhealthy machine but not two.
The golden-parity gate pinned the child chart's MachineHealthCheck
values to the parent's defaults as inputs, which made a default moved
on both sides fail there for the wrong reason. Drop the pins so the
comparison is between the two charts' defaults, which is the contract
it exists to hold, and pin the defaults themselves in each chart's own
unit tests instead.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
The two unhealthyConditions timeouts were fixed in the template at 30s
for `Ready=Unknown` and 300s for `Ready=False`, and both are too tight
for a worker joining on a loaded host.
`Ready=Unknown` is what the node lifecycle controller writes when a
kubelet stops reporting, on a grace period of its own, so 30s on top
of that deleted the VM of a guest that had merely stopped being
scheduled for a stretch, before it had a chance to answer again. And
`Ready=False` is the condition a registered Node holds until its CNI
is up, so 300s was the entire budget a first boot got for the CNI
rollout to reach it -- a worker deleted mid-rollout takes the rollout
with it and its replacement starts over.
Move both to values, defaulting to 5m and 10m. The remediation of a
worker that is genuinely dead is correspondingly later, which costs
less than it appears to: Pods leave an unreachable Node on the node
lifecycle controller's taint-based eviction and the toleration that
carries, not on either of these timers.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
The cluster-autoscaler treats a machine as long-unregistered once its
own max-node-provision-time elapses, and a worker that has not
registered by then stops counting towards its node group's upcoming
capacity. On a host slow enough that the Talos image import, the guest
boot and the CNI rollout do not fit inside that window, the timer
fires on machines that were still joining: the group reads as short of
capacity, so the autoscaler asks for replacements, and the extra
provisioning load makes the join it was waiting for slower still.
Render the per-node-group override the autoscaler reads off the
MachineDeployment, sourced from a new maxNodeProvisionTime value that
defaults to 30m. The duration is checked at render time because the
autoscaler discards one it cannot parse and falls back to its own
default without failing, and because the values schema types the field
as a string and so accepts every malformed duration.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
Both podCpuLimit and podCpuRequest were rendered under `with`, which
skips a zero as readily as an absent field, and their guards were keyed
the same way. The generated schema behind them admits a leading minus
and a zero as readily as a sane value, so rather than patch whichever
edge turned up, the whole value domain is settled in one place.
Presence, not truth, is the predicate everywhere either field is read.
A negative value is refused: it never reaches a running Pod. A zero
request on its own is allowed and reserves nothing. A zero request
beside a non-zero limit is refused: KubeVirt's defaulter tests the
request with IsZero, which an explicit zero satisfies exactly as an
absent key does, so it overwrites the zero with the limit and the
worker asks the scheduler for its whole ceiling. That is the outcome
podCpuRequest exists to prevent, reached silently and reported nowhere,
which is why it is refused rather than written down.
A zero limit removes the ceiling rather than being the tightest one --
the kubelet turns a zero CPU limit into no CFS quota -- but it reaches
a Pod only paired with a zero request. KubeVirt keeps deriving a
request from the vCPU count when the VM declares none, the defaulter
that would have copied the limit across does not fire because the limit
is itself zero, and API server validation refuses a request above a
limit that exists. Zero is a limit that exists. So a lone zero limit is
refused too.
Both fields document the same rules and each class has a test on both
charts. A group that sets neither field renders the bytes it rendered
before, so the content-hash name of the worker VM template does not
move.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
Adds nodeGroups.<name>.podCpuRequest, rendering into
domain.resources.requests.cpu on the worker VM, and the same field on
the kubernetes-nodes chart, which is the other half of a byte-identity
contract rather than a separate feature surface.
podCpuLimit on its own cannot raise the ceiling of a worker without
raising what that worker asks the scheduler for, because KubeVirt
copies a declared CPU limit into the request when the VM declares none.
The request is the half that has to hold still, so the pair can now be
moved apart.
Both fields conflict with instancetype sizing for the same reason and
in the same upstream check, so the request gets the guard the limit
already had. A request above its own limit fails the render too, rather
than reaching API server validation on a Pod two objects away from the
CR that named the values; the quantities are compared as numbers, so
500m under a ceiling of 1 is accepted where a lexical guard would
refuse it.
A node group that sets neither field renders the bytes it rendered
before, which the content-hash name of the KubevirtMachineTemplate
depends on, and the render-parity gate gains a case that compares the
pair across the two charts.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
The field was documented as moving the ceiling alone and leaving the CPU
request at the value KubeVirt derives from the guest vCPU count and the
cluster CPU allocation ratio. KubeVirt does the opposite: its VMI
defaulter copies a declared CPU limit into the CPU request whenever the
VM declares no request of its own, so a node group that sets only the
limit asks the scheduler for its whole ceiling, and a ceiling the nodes
cannot spare leaves the workers Pending on Insufficient cpu.
The same paragraph promised a floor that does not exist either. A limit
below the derived request was said to leave the container with a request
above its limit; the copy makes the two equal instead, whatever the
limit is.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
The split-out pool chart renders the same worker VM as the monolithic
one, and tests/render-parity.sh exists to prove it renders it
byte-identically, because the KubevirtMachineTemplate is content-hash
named and any divergence rolls every live worker VM. A field that only
one of the two charts knows breaks that quietly rather than loudly: a
pool adopted into a kubernetes-nodes release would render without the
CPU limit it was given, the hash would change, and the ceiling would
fall back to the vCPU count with nothing reporting it.
So the field, its render-time guard against combining it with an
instancetype, and its tests arrive here in the shape the parent chart
has them, and the parity gate gains a case that compares the two
renders with the limit set.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
The chart had no way to give a worker VM a CPU ceiling different from
its vCPU count. `resources.cpu` sizes the guest, and the pod side is
filled in by KubeVirt: with a ResourceQuota limiting CPU in the
namespace, which every tenant carrying resourceQuotas has, it derives
limits.cpu from the guest vCPU count. The QEMU emulator thread and the
IO threads then come out of the same allowance as the vCPU threads,
which is tight for a nested guest that pays emulation overhead.
Declaring the limit on the VM is what stops KubeVirt deriving one, and
it is the only way out that does not make things worse: dropping cpu
from the tenant resourceQuotas would let the LimitRange apply its 250m
default instead, four times tighter than what it replaced.
The limit is usable only on a group sized by resources, because
KubeVirt's instancetype applier reports domain.resources.limits.cpu as
a conflict rather than resolving it, and a VM carrying both is refused
by the admission webhook with nothing on the CR to say why. Sizing by
resources is what drops the instancetype from the VM, so the chart
fails the render for a group that kept its instancetype, beside the
existing guard for the same conflict class one field over.
The field renders no CPU request, so the request stays derived from the
vCPU count and the cluster CPU allocation ratio, and the scheduler
weight of a worker does not move with its ceiling. It renders nothing
at all when unset, byte for byte: the enclosing KubevirtMachineTemplate
is named by a hash of its content, so anything else there would roll
every live worker VM on upgrade.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
Adds a MongoDB (Percona Server for MongoDB / psmdb) backup-strategy driver so
`apps.cozystack.io/MongoDB` reaches feature parity with the CNPG, MariaDB and
ClickHouse native drivers — logical backups, restore-to-differently-named
instances, and point-in-time recovery through the unified
BackupJob/RestoreJob/BackupClass/Plan interface (closes#3545).
The driver orchestrates the operator's PerconaServerMongoDBBackup /
PerconaServerMongoDBRestore CRDs:
- BackupJob: creates a one-shot PerconaServerMongoDBBackup, polls
status.state, and persists the S3 destination + storage descriptor
(by-reference, no credentials) onto the Cozystack Backup artifact.
- RestoreJob: drives PerconaServerMongoDBRestore via backupSource (read back
from the source backup status, snapshot fallback) so the same artifact
restores in-place or into a differently-named instance; PITR via
spec.options.pitr (date/latest).
Because psmdb resolves the backup storage by name against the cluster's
spec.backup.storages, the driver names the chart's storage (s3-storage) via the
cozy-default-mongodb Strategy rather than injecting it, and surfaces a clear
Ready=False precondition when a target has backups disabled.
Wiring:
- api/backups/strategy/v1alpha1: MongoDB strategy type + generated CRD/deepcopy.
- internal/backupcontroller: psmdbstrategy_controller + psmdbtypes/mongodbapp
partial-schema mirrors; dispatch, supported-kinds and scheme registration.
- backupstrategy-controller: cozy-default MongoDB route, cozy-default-mongodb
Strategy CR, and psmdb.percona.com RBAC.
- apps/mongodb: backup.insecureSkipTLSVerify knob (psmdb s3 storage has no
CA-bundle field) so backups can target Cozystack's self-signed seaweedfs.
Tests + e2e:
- unit tests for the driver (validation, ensure-idempotency, precondition,
backupSource resolution, snapshot round-trip, PITR option parsing).
- examples/backups/mongodb + a mongodb-2-backup-roundtrip Chainsaw suite that
drives run-all.sh end-to-end (sentinel document round-trips through S3 into a
restored copy while the source stays untouched), per docs/agents/e2e-testing.md.
- docs/operations/backup-classes.md updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
## What this PR does
Gives the ClickHouse `clickhouse-backup` sidecar a way to trust an S3
endpoint whose certificate is signed by a private CA, and uses that to
make the ClickHouse backup/restore e2e round-trip actually run in CI.
This actualizes #2600, whose feature code and example scripts have since
landed on `main` independently, and whose remaining contribution — the
e2e test — was still in the retired BATS format. #2600 can be closed
once this merges.
### 1. `backup.endpointCA` on the ClickHouse chart
The sidecar had **no TLS/CA surface at all**: bucket coordinates and
credentials arrive as `S3_*` env, and nothing let it verify an endpoint
signed by a private CA. Cozystack's own in-cluster SeaweedFS is exactly
that case — it serves `:8333` behind the self-signed "SeaweedFS CA" — so
a ClickHouse release could only ever back up to a publicly-trusted
endpoint. MariaDB (`storage.s3.tls.caSecretKeyRef`) and Postgres (barman
`endpointCA`) have had a CA field all along; this closes the gap for
ClickHouse.
`backup.endpointCA` takes `{name, key}`. When set, the chart mounts that
Secret into the sidecar and points `SSL_CERT_DIR` at it. Empty — the
default — renders no env, no mount and no volume, so existing releases
are byte-for-byte unchanged.
**Why `SSL_CERT_DIR` and not `AWS_CA_BUNDLE`.** Both are honoured by
`altinity/clickhouse-backup:2.7.4`; I measured them against a
self-signed endpoint and public AWS S3 in turn. Go reads `SSL_CERT_DIR`
*in addition to* the system bundle, so both endpoints verify.
`AWS_CA_BUNDLE` *replaces* the SDK's trust pool: the private endpoint
verifies, and public S3 then fails `x509: certificate signed by unknown
authority` — it would silently break a release that also talks to a
public endpoint. `S3_DISABLE_CERT_VERIFICATION` was rejected outright,
since it drops verification rather than trusting one more CA.
### 2. The e2e suite, and what it was really testing
Two Tests added to the existing `hack/e2e-chainsaw/clickhouse/` suite
(in name order, mirroring the `etcd`/`mariadb`/`postgres` shape — see
the layout rationale below for why they live here rather than in a new
directory):
- **`clickhouse-1-backup-contracts`** (CI, `tenant-test`, ~98s): a
backup-enabled ClickHouse asserted against the *running* cluster — the
`clickhouse-backup` sidecar on the `ch-backup-api` `:7171` port, and the
`<release>-backup-api-auth` Secret whose keys the sidecar's
`API_USERNAME`/`API_PASSWORD` actually resolve `valueFrom`. Each is a
silent break if a chart bump renames it. Needs no object store.
- **`clickhouse-2-backup-roundtrip`** (CI, `tenant-root`): `Bucket` →
`Altinity` strategy + `BackupClass` → source ClickHouse with a sentinel
row → `BackupJob` → in-place `RestoreJob` → to-copy `RestoreJob` into a
second instance, driving `examples/backups/clickhouse/run-all.sh` so
test and docs cannot drift.
The round-trip was originally gated behind `CLICKHOUSE_E2E_S3_ROUNDTRIP`
and **skipped in CI** — its `PASS (1.01s)` meant the gate fired, not
that a backup happened. `backup.endpointCA` removes the reason for the
gate, so the suite now adopts the same three preconditions
mariadb/postgres rely on: run in `tenant-root` (an isolated tenant's
Cilium egress allowlist blocks its Pods from reaching `tenant-root`'s
SeaweedFS), target the in-cluster endpoint instead of the `Bucket`'s
external ingress URL (an unroutable placeholder in CI), and trust its
certificate via the copied CA. It also pre-cleans with `cleanup.sh`,
because `tenant-root` is shared and a leftover `Succeeded` `BackupJob`
would satisfy the harness's wait without a backup ever running.
Both backup Tests live in the existing `hack/e2e-chainsaw/clickhouse/`
suite rather than a new suite directory, so no selector mapping is
added: `hack/select-e2e.sh` already derives `clickhouse` both from
`packages/apps/clickhouse/**` (the `*-application` rule) and from
`examples/backups/clickhouse/**` (the harness path), and both paths
therefore select the suite that actually runs the harness. That
alignment is the reason for the layout — a separate `clickhouse-backup/`
directory would have made an edit to the harness select a suite that
cannot exercise it. `hack/select-e2e.sh` and `hack/select-install.sh`
are otherwise untouched (the only change is a comment recording that
invariant), and the existing `backup example harness edit selects its
app suite` case in `hack/select-e2e_test.bats` covers the mapping.
### Verification
- `helm unittest` — 17 pass, including 5 new `endpointCA` cases: the
unset default renders nothing, the env/mount/volume wiring, a custom key
still projected as `ca.crt`, composition with `useSystemBucket`, and
`backup.enabled=false` renders nothing.
- The CA-trust behaviour was measured against the real image, not
assumed: without a CA knob the sidecar reports `x509: certificate signed
by unknown authority`; with `SSL_CERT_DIR` the handshake succeeds.
- `hack/select-e2e_test.bats` (16) and `hack/select-install_test.bats`
(17) pass; `select-install.sh --validate` reports graph OK.
- `make generate` is idempotent, so the pre-commit job's `git diff
--exit-code` stays clean.
- The contracts Test passed in CI on earlier runs of this branch. The
round-trip only became CI-live with this work, so it still needs a green
E2E run before merge — the branch has since been rebased onto current
`main` to get one.
### Screenshots
Not applicable — no UI changes.
### Downstream repositories
Walked the trigger map against the diff, file by file. The load-bearing
change is the new `backup.endpointCA` field in
`packages/apps/clickhouse/values.yaml` / `values.schema.json`;
everything else is `hack/e2e-chainsaw/`, `examples/backups/`, one docs
paragraph, and regenerated artifacts, none of which appear in the map.
- [ ] No downstream repository is affected by this change
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
https://github.com/cozystack/terraform-provider-cozystack/issues/18 —
"add, remove or rename a field in an app's `values.schema.json`" reaches
the provider's hand-maintained schema, model and expand/flatten pair, so
the new nested object is not surfaced until it is added there. Opened as
an issue rather than a PR because the natural moment is alongside that
repo's next `api/apps/v1alpha1` pin bump, which is a maintainer call.
Not ticked, with reasons rather than silence: **cozystack/website** —
the trigger is *adding, renaming or removing a package*, and this adds a
field to an existing one. `clickhouse` is already in the website
`Makefile`'s app list and its reference page is generated from this
package's `README.md`, so the regenerated `backup.endpointCA` rows are
carried by the existing release-tag automation with no manual PR.
**cozystack/external-apps-example** — vendors `hack/package.mk` and
`update-crd.sh`, neither of which this PR touches.
**cozystack/ansible-cozystack**, **ccp**, **talm**, **cozyhr**,
**cozy-proxy**, **cozystack-telemetry-server**, **examples** — no
installer values, `hack/` layout, namespace, variant, node-prerequisite,
annotation, label or metric in their trigger rows is touched here.
### Release note
```release-note
feat(clickhouse): add `backup.endpointCA` so the clickhouse-backup sidecar can verify an S3 endpoint whose certificate is signed by a private CA, such as Cozystack's in-cluster SeaweedFS. Leave it unset for publicly-trusted endpoints; existing releases are unaffected.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added optional private S3 endpoint CA support for ClickHouse
backup/restore sidecars via `backup.endpointCA`, including configurable
Secret name and CA bundle key (default `ca.crt`).
* Sidecars mount the CA and extend trust while leaving public endpoints
unchanged.
* **Documentation**
* Updated ClickHouse chart docs, operational backup guidance, and
backup/restore examples to cover CA setup, overrides, and cleanup.
* **Tests**
* Added helm-unittest and end-to-end coverage validating CA
rendering/mounting behavior and ClickHouse backup/restore workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
State in the README that in-guest registry pulls are routable again via
talos.registryMirrors, which knob covers which artifact (the mirror
routes registry pulls; the worker OS disk image arrives as a raw HTTP
artifact under talos.imageFactoryURL and no registry mirror touches
it), and the limits: skipFallback is needed for air-gap,
machine.registries.config is not passed through, and existing nodes
keep their boot-time config until replaced. Trim the field doc comment.
The regenerated schema, README tables and cozyrds openAPISchema follow
from those sources.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add a talos.registryMirrors passthrough to the kubernetes and kubernetes-nodes
charts that renders into the worker TalosConfigTemplate machine.registries.mirrors,
so an operator can point a registry such as ghcr.io at an in-cluster pull-through
mirror and a worker boot no longer depends on live public egress. Default {}
keeps the rendered worker reconcile Job byte-identical to before, content-hash
Job name included. Free-form mirror values are shell-escaped for the unquoted
apply heredoc, covered by an execution-level bats test.
Also add the e2e ghcr-mirror (registry:2 pull-through plus a Cilium egress
policy) and wire it into the kubernetes chainsaw suites, so CI exercises the
mirror path and takes the flaky public ghcr.io kubelet-image pull off live
egress.
Refs #3513 (addresses the egress variant of the tenant-worker node-join
failures; the dominant variant is CI-runner CPU starvation, tracked in the
issue).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
A worker VM that stalls during boot before Talos apid answers is
invisible to every diagnostic the platform has: no Node registers,
no certificate request arrives, and talosctl needs the apid that
never came up. The guest serial console does show that window, but
KubeVirt attaches its guest-console-log container only when the VMI
asks for it, and the platform turns console logging off cluster-wide.
Add an opt-in per-node-group logSerialConsole that sets the KubeVirt
field, which takes precedence over the cluster-wide setting. The key
renders nothing unless true -- including when set to false -- because
the KubevirtMachineTemplate is content-hash named, so a byte emitted
for a group that did not ask for it renames the template and rolls
every live worker VM.
The description carries the two costs the field name does not show.
Changing the value rolls the group it is set on, so the stuck VM
whose console was wanted is replaced along with the rest and the
setting buys visibility into the next boot rather than the current
one. And the cluster-wide default it overrides exists because that
same container has been seen holding virt-launcher in
PodInitializing, which would take a group down rather than explain
it.
Worker VMs are emitted by two charts. Both carry the field, and the
golden-parity check gains a case that keeps their output identical.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bring the worker-pool split branch up to date with main (111 commits),
picking up the node-join and CI fixes it was missing: the 18m tenant
node-join deadline, guest-fsync-off on ephemeral CI disks (#3455),
kubevirt-csi client rate-limit raise (#3428), the Cilium ingress-IP
race guard (#3430), and etcd-operator v0.5.4.
Conflict resolution: main's #3535 (render the talos-reconcile Job for
the default md0 group) modified packages/apps/kubernetes/templates/talos/
talos-reconcile-job.yaml, which this branch deletes because the worker
split moves that Job into the kubernetes-nodes chart. Kept the parent
template deleted and dropped the parent-scoped test
talos_reconcile_nodegroups_test.yaml: the empty-nodeGroups md0-default
gap it guards cannot occur in the per-pool child chart, and migration 54
materialises the implicit md0 into an explicit child HelmRelease.
targetVersion resolves to 55 (migration 54 on top of main's 53).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The images.kubectl override now feeds both the talos-reconcile Job and the
pre-delete unpin Job, but its @field description named only the former.
Update it and regenerate the derived schema, README, Go types and the
kubernetes-nodes-rd cozyrd via cozyvalues-gen + update-crd.sh.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Review notes from IvanHunters on #3385 (LGTM with non-blocking findings).
1. The backup-s3-ca volume carries no `optional: true` while the adjacent
useSystemBucket credential refs deliberately do, and the asymmetry was
undocumented — an operator naming a Secret that does not exist gets the whole
ClickHouse Pod stuck in ContainerCreating, not just the sidecar. Kept the
fail-closed behaviour and wrote down why the two cases differ: credentials are
projected by the platform after install, so a strict ref would wedge every
fresh release until the first BackupJob, whereas this CA is named by the
tenant, must already exist, and is trust material — optional would mount an
empty dir, the Pod would look healthy, and the first backup would fail deep
inside clickhouse-backup with an x509 error naming neither the Secret nor the
chart. The comment also states the blast radius plainly.
2. The CA probe swallowed stderr, so an RBAC denial on the cross-namespace read
was reported as "CA not found" followed by advice (set S3_CA_SECRET) that
cannot fix a permissions problem. It now separates present / absent /
not-allowed-to-look and says so, and the README no longer labels step 03
simply "tenant": copying the CA reads a Secret in tenant-root, which is a
platform capability. Added a section spelling out the three ways to satisfy
that (admin credentials, a one-off admin copy, or skipping the CA entirely on
a publicly-trusted endpoint).
3. Caveat, and a usability regression this branch introduced: on a cluster with
a publicly-trusted endpoint and no seaweedfs, step 03 hard-exited on a default
the user never chose. It now distinguishes an explicitly named Secret (still
an error when missing — a typo should not be guessed past) from the unset
default, where it warns and continues with backup.endpointCA unset. Verified
all four paths against a fake kubectl: Forbidden, absent+default,
absent+explicit, and present.
4. Caveat: endpointCA reaches the clickhouse-backup sidecar only. The legacy
`schedule` CronJob writes to S3 through restic and does not consume it, so a
private-CA endpoint combined with `schedule` still fails TLS. Documented on
the field itself, so it lands in values.schema.json, the README and the
cozyrds openAPISchema.
The PR body's claim about select-e2e.sh/select-install.sh mappings was stale
after the restructure that removed them; corrected there rather than in code.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
The clickhouse-backup sidecar had no TLS/CA surface at all: bucket coordinates
and credentials arrive as S3_* env, and nothing let it verify an endpoint whose
certificate is signed by a private CA. Cozystack's own in-cluster SeaweedFS is
exactly that case — it serves :8333 behind the self-signed "SeaweedFS CA" — so a
ClickHouse release could only ever back up to a publicly-trusted endpoint, while
MariaDB (storage.s3.tls.caSecretKeyRef) and Postgres (barman endpointCA) have had
a CA field all along.
Add backup.endpointCA {name, key}. When set, the chart mounts that Secret into
the sidecar and points SSL_CERT_DIR at it. Empty (the default) renders no env, no
mount and no volume, so existing releases are untouched.
SSL_CERT_DIR rather than AWS_CA_BUNDLE, measured against
altinity/clickhouse-backup:2.7.4 with a self-signed endpoint and public AWS S3 in
turn: Go reads SSL_CERT_DIR in addition to the system bundle, so both verify;
AWS_CA_BUNDLE replaces the SDK trust pool, so the private endpoint verifies and
public S3 starts failing `x509: certificate signed by unknown authority`. That
would silently break a release that also talks to a public endpoint.
S3_DISABLE_CERT_VERIFICATION was rejected outright — it drops verification
instead of trusting one more CA.
Covered by helm-unittest: the unset default, the mount/env/volume wiring, a
custom key still projected as ca.crt, composition with useSystemBucket, and that
backup.enabled=false renders nothing.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Gate the Proxied ingress Service on _namespace.etcd like the addon
HelmRelease it fronts, so an etcd-less tenant with the addon pinned on does
not get a selector-less Service (+ test). Correct the secrets framing: the
tenant-visible surface is an allowlist (include: [] withholds everything),
so drop the never-load-bearing exclude entry, the bats cross-check and the
rename-coupling comments, and state the real mechanism. Fix the etcd
prerequisite docs (an ancestor's etcd satisfies the wrapped chart; the
awaiting-etcd beacon is the diagnostic), reword the tenant toggle to stop
advertising the not-yet-shipped placement routing, require maxReplicas >= 1
on the ingress-role node group, and fix comment drift (Service name after
the release rename, placement-hook label, package-guard lookup semantics).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Wire the computeplane module into the tenant chart as a bool toggle, matching
the other tenant modules. The module's own Helm release takes the -module
suffix so the canonical release name computeplane (and with it the
computeplane-admin-kubeconfig Secret) stays reserved for the cluster
HelmRelease the module renders.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
The version field's description said the pool "must match the parent
cluster's version", but assertParentVersion deliberately enforces a
different contract: a pool may lag the parent -- the normal state
during a rolling upgrade, parent Kubernetes CR first, then each pool --
and only a pool AHEAD of the parent fails the render (a symmetric
equality requirement would flip every pool to render-failure the
moment the parent is bumped). A user following the description would
believe the intended upgrade sequence leaves their pools unsupported.
Reword the @param to state the actual contract and regenerate with
cozyvalues-gen v1.6.0 (the CI-pinned version; 1.3.0 silently drops the
@immutable rules) so schema, README, Go types and the RD all follow.
Description-only: no defaults, enums or validations change, pinned by
the generate-drift CI check and the existing parent_version_skew_test
lag case the words now agree with.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Address review on #3315:
- Mark cluster and storageClass immutable (self == oldSelf): both feed object
names / the KubevirtMachineTemplate content hash, so editing them on a live
pool would rename everything and delete the running worker VMs.
- Validate the pool's Kubernetes minor against the parent cluster's
KamajiControlPlane at render (skipped offline / when the parent is absent),
so a pool created after the split cannot join workers ahead of the apiserver.
- Note in the snapshot suite header that the instancetype-sized default is not
pinned offline (lookup nil); the e2e exercises it.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
types.go dropped NodeGroups/NodeGroup/GPU/Kubelet/NodeHealthCheck but
the generated deepcopy still referenced them, so the api module failed
to build. Regenerated with controller-gen.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
Worker pools are now managed via KubernetesNodes resources; existing
pools are adopted by migration 52.
BREAKING CHANGE: the Kubernetes CR no longer accepts spec.nodeGroups.
Create KubernetesNodes resources for worker pools instead.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
The cluster field referenced x-cozystack-options source 'kubernetes',
which has no provider (cozystack-api returns NotFound), so the dashboard
dropdown 404s. Drop the annotation so the field is a plain text input;
a real provider listing Kubernetes CRs is a follow-up.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
The pre-commit run-make-generate hook regenerates the Go types from the
values annotations; the serverName default note landed in values.yaml,
README, schema and postgres-rd but missed types.go.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Review follow-up (cozystack#3300): the apps/postgres user-facing docs still
described the removed native spec.backup.barmanObjectStore / archive_command
model. Update the values.yaml @field/@typedef/@param annotations (useSystemBucket,
schedule, S3CredentialsSecret, EndpointCA, endpointCA, the @param backup note,
bootstrap.serverName) and the hand-written README Operations prose to the
barman-cloud plugin + ObjectStore model, then regenerate values.schema.json, the
README parameters table, the api types, and the postgres-rd ApplicationDefinition
schema (which feed the dashboard form help + cozyrd schema). Field names,
defaults, and the storageClass immutability marker are unchanged; only backup/
bootstrap description text changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Add optional nodeGroups[name].maxUnhealthy and nodeGroups[name].nodeStartupTimeout
fields that override the cluster-wide nodeHealthCheck defaults per worker node
group. When unset, a group inherits nodeHealthCheck.{maxUnhealthy,nodeStartupTimeout},
so existing values are unaffected. The MachineHealthCheck render resolves the
per-group value first, reuses the existing int-or-string validation, and names
the offending group in the fail() message when an override is invalid.
Delivers the per-group granularity of the standalone maxUnhealthy proposal on
top of the cluster-wide nodeHealthCheck from the Talos worker bootstrap PR.
Co-authored-by: mattia-eleuteri <mattia@hidora.io>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Tenant worker VMs were pinned to Talos v1.13.0, which ships Linux 6.18.24.
That kernel predates both fixes for a pair of related KVM x86 shadow-paging
use-after-free bugs: CVE-2026-46113, fixed in 6.18.30, and CVE-2026-53359,
fixed in 6.18.38. A worker's own kernel runs KVM whenever a tenant enables
nested virtualization inside the VM, and either bug lets a nested guest
escape into the worker.
Talos v1.13.6 ships Linux 6.18.38. Stable kernel branches are cumulative, so
it is the first v1.13 patch release carrying both fixes: the pinned v1.13.0
ships 6.18.24 and carries neither, while v1.13.5 ships 6.18.36 and carries
only CVE-2026-46113. Nested virtualization stays available to tenants — the
kernel bump is the fix.
The new pin changes the rendered worker spec, so the content hash in the
KubevirtMachineTemplate and talos-reconcile Job names rotates and each
worker pool rolls once on upgrade. Pinned hashes in the chart tests are
recomputed accordingly.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Tenant worker VMs boot from a Talos raw disk image that CDI streams over
HTTP from the hardcoded public Talos Image Factory, and the in-guest
talos-reconcile Job pulls its installer image from the same hardcoded host.
Neither was overridable, so air-gapped / rate-limited / flaky-egress
environments could not point workers at a mirror (the README flagged this
as a Phase-2 gap), and CI had no way to avoid bulk-pulling the OS image
from the public internet per worker.
Add two chart values, both defaulting to the public factory so rendered
output is byte-identical and behaviour is unchanged:
- talos.imageFactoryURL (default https://factory.talos.dev) - base URL for
the worker OS disk image streamed in by CDI.
- talos.installerRepository (default factory.talos.dev/installer) - OCI
repository prefix for the in-guest installer image.
Regenerated values.schema.json, the aggregated-API Go types, the
kubernetes-rd cozyrds schema and the README via `make generate`; added
unittests for the override and trailing-slash trimming.
Refs #3231
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Follow-ups from the self-review of #3233:
- `values.yaml` / helper docstring / regenerated README + schema +
cozyrds openAPISchema + Go type doc: the previous "one
ingress-nginx worker" contract was already misleading (helper
default carried `minReplicas: 0` but the chart hardcoded
`spec.replicas: 2`), and the replicas fix makes the docstring
strictly false — an empty-nodeGroups install now provisions zero
workers until either the cluster-autoscaler responds to an
unschedulable Pod or the operator explicitly scales the group.
Update the docstrings to describe the actual behaviour and to
spell out the ingress-nginx-addon implication so operators are
not surprised at addon-install time.
- `tests/nodegroups_default_test.yaml`: two regression cases pinning
`MachineDeployment.spec.replicas == $group.minReplicas` for
`minReplicas: 0` (the render-side OIDC bats path — zero worker
DataVolumes provisioned) and for `minReplicas: 2` (the
kubernetes-latest / kubernetes-previous bats path — byte-
identical to pre-PR behaviour). Without these a future refactor
can silently re-hardcode `replicas: 2` and turn no tests red.
helm-unittest suite grows from 171 to 173, both new cases pass.
Ref: cozystack/cozystack#3231.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Gemini flagged that the `values.yaml` @field docstring on
`oidc.users[].email` still described the claim mapping as
`preferred_username`, misleading operators after the claim was
switched to `email` for real (see the earlier fix and its
end-to-end verification on dev3). Rewrite the description to
match the actual mechanic: `email` claim, requested via
`--oidc-extra-scope=email` in the chart-generated kubeconfig,
emitted by every conformant OIDC provider.
`make generate` re-propagates the update through README.md,
types.go and the cozyrds embedded schema.
Note on the other finding in the same review — Gemini also
flagged `audiences` as being incorrectly nested under `issuer`
in the AuthenticationConfiguration Secret. That is a
false positive: per the kube-apiserver v1beta1 spec
(apiserver.config.k8s.io/v1beta1, `JWTAuthenticator.Issuer`),
`audiences` IS a child of `issuer`, not a sibling of `jwt[i]`.
The current shape is correct and verified end-to-end on dev3
(alice/bob tokens carry `aud: [tenant-test-kubernetes-test3,
kubernetes]` and apiserver validates them successfully with
CRB-driven RBAC).
Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
Arsolitt