Commit Graph
7777 Commits
Author SHA1 Message Date
Andrey KolkovandClaude Opus 4.8 2829f7a300 docs(mariadb): address review notes on backup example
- 00-bucket.yaml: correct the Secret-name comment to include the
  "bucket-" release prefix (bucket-<bucket>-<user>), matching the
  README and 00-helpers.sh.
- README: note that S3_CA_NAMESPACE defaults to tenant-root
  independently of NAMESPACE, so a non-root tenant run must override
  it (and S3_CA_SECRET) too.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-07-22 10:44:42 +04:00
Andrey KolkovandClaude Opus 4.8 d1f2822018 test(mariadb): add backup/restore round-trip e2e (chainsaw) via example harness
Reworks the never-merged backup-mariadb-e2e branch onto the modern
Chainsaw approach (docs/agents/e2e-testing.md). The bats port that
branch carried (hack/e2e-apps/backup-mariadb.bats) is dropped — the
whole hack/e2e-apps/*.bats app suite has been replaced by
hack/e2e-chainsaw/, and the mariadb strategy controller + example
manifests it depended on have since landed on main independently. Only
the e2e coverage was still missing.

Convert examples/backups/mariadb/ from static numbered YAML into the
same executable harness the other backup demos use (00-helpers.sh,
run-all.sh, cleanup.sh), keeping the numbered manifests as the
human-readable flow that run-all.sh drives — so the documented flow and
the automated test can never drift. run-all.sh provisions a Bucket,
derives the two per-app Secrets the strategy references from BucketInfo
+ the seaweedfs CA, deploys a source MariaDB with a sentinel row, fires
an ad-hoc BackupJob, restores to a copy with a to-copy RestoreJob, and
asserts the sentinel round-tripped through S3 into the copy while the
source is left untouched.

Add a second Chainsaw Test (mariadb-2-backup-roundtrip) to
hack/e2e-chainsaw/mariadb/ that drives run-all.sh + cleanup.sh, mirroring
postgres-2-backup-roundtrip: runs in tenant-root (the isolated e2e tenant
cannot reach tenant-root's seaweedfs across the Cilium egress policy),
skipDelete since the harness (not Chainsaw) owns the resources, fail-fast
on terminal BackupJob/RestoreJob phases, and scoped catch diagnostics.
Cleanup also prunes the operator-side k8s.mariadb.com Backup/Restore CRs
the Cozystack jobs own only by label.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-07-22 10:44:42 +04:00
Andrey KolkovandClaude Opus 4.8 6a1e2d0d06 feat(backups): support PostgreSQL point-in-time recovery (PITR)
Makes RestoreJob.spec.options.recoveryTime a genuinely supported, tested
capability for the new backups.cozystack.io API (issue #2774). The
recoveryTime -> CNPG bootstrap.recovery.recoveryTarget.targetTime plumbing
already existed on the barman-cloud-plugin branch; this adds the reliability
and verification layer the issue asked for:

- Fail-fast for an unreachable target: when a RestoreJob has recoveryTime set
  and the recovery Cluster is not healthy, the driver reads the CNPG
  full-recovery pod logs and, once several distinct attempts have exited with
  PostgreSQL's "recovery ended before configured recovery target was reached"
  FATAL, fails the RestoreJob with conditions RecoveryConverged=False /
  Ready=False (reason RecoveryTargetUnreachable) instead of hanging to the
  restore deadline. Health is checked first, so a converged recovery always
  wins; counting distinct FATAL-ing attempts (not pod failures) tolerates a
  transient archive-lag miss on a near-now target and ignores transient
  recovery-pod crashes. Adds the pods/log RBAC grant the log read requires.

- e2e (examples/backups/postgres/run-all.sh, wired into the Chainsaw
  postgres-2-backup-roundtrip suite): step 45 writes a before marker, captures
  the server timestamp, writes an after marker, restores to that timestamp and
  asserts the before row survived and the after row did not; step 46 submits a
  recoveryTime an hour ahead and asserts it fails fast with reason
  RecoveryTargetUnreachable.

- Working, uncommented example (45-restorejob-pitr.yaml) and a docs section
  (docs/operations/backup-classes.md) covering the recoverable window, how to
  discover the earliest/latest restorable time from the backup catalog, the
  archive-lag tolerance, and idempotency under GitOps.

Refs #2774

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-07-22 10:40:18 +04:00
Andrey KolkovandClaude Opus 4.8 ae946c7143 fix(etcd-operator): make CRD vendoring fail loudly on fetch error
Review of #3403 noted the `curl … | awk > file` recipe swallowed curl failures:
the pipe's exit status is awk's, and neither the Makefile nor hack/package.mk sets
pipefail, so a 404 / network error / upstream path rename on a future bump would
write an empty/truncated CRD template and still exit 0.

Fetch to a temp file under `set -e` (curl -f already exits non-zero on HTTP error),
then awk from it — shell-agnostic (no pipefail, which dash lacks). Verified: the
regenerated templates are byte-identical to the committed ones, and a bad ref now
exits non-zero leaving no template files instead of silently writing empties.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
2026-07-22 10:37:20 +04:00
Aleksei Sviridkin 480c6d8a01 chore: ignore root-level Go binaries built from cmd/
`go build ./cmd/<name>` without -o drops an ~80 MB executable in the repo
root under the package name. Ignore each cmd/ binary so a stray `git add`
cannot commit one.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-22 00:08:14 +03:00
Aleksei Sviridkin d443a092ad feat(api)!: require the application release prefix to be dot-free
Constrain ApplicationDefinition spec.application.release.prefix to
lowercase DNS-1123 characters (^[a-z0-9-]*$), which excludes the dot.
Release names are "<prefix><app name>" and the tenant CA trust anchor is
projected to "<release>.tenant-ca"; the dot is the separator that makes
that name unreachable by any engine, so a prefix carrying a dot would
break the guarantee.

BREAKING CHANGE: an ApplicationDefinition whose release prefix contains a
dot, or any character outside [a-z0-9-], is now rejected at admission.
Existing definitions with such a prefix must be corrected before this CRD
is applied.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-22 00:07:55 +03:00
Aleksei Sviridkin 99f04d99bd test(postgres): verify the tenant CA projection end to end
Assert the live controller publishes <release>.tenant-ca carrying only
ca.crt, that its certificate matches CloudNativePG's CA while the private
key never crosses over, and that the anchor is reachable through the
tenantsecrets API while the key-bearing source Secret is not.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-21 23:59:48 +03:00
Aleksei Sviridkin 858c1c04fe feat(postgres): declare the tenant CA trust anchor
CloudNativePG stores its self-signed CA in <release>-ca alongside the
private key and offers no way to label it, so a tenant cannot be handed
the trust anchor without also being handed the key. Render a
TenantProjection that lifts ca.crt alone into the key-free
<release>.tenant-ca Secret, and select that label from the postgres
tenant-resource definition so the projection is served through the
tenantsecrets API. Document the new retrieval path in the chart README.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-21 23:59:48 +03:00
Aleksei Sviridkin 37a91f98b8 feat(cozystack-controller): extract application CA into a key-free tenant Secret
Every catalog engine issues a per-release CA, but the object holding
ca.crt usually holds a private key beside it (cert-manager's tls.key,
CloudNativePG's ca.key), so granting a tenant read on it hands over key
material too. Add a controller that publishes the trust anchor as one
canonical, key-free Secret per release, "<release>.tenant-ca", holding
only ca.crt.

A chart declares the anchor by rendering a namespaced TenantProjection
sentinel (internal.cozystack.io/v1alpha1) that names the source Secret
and key. The controller rebuilds ca.crt from the parsed certificate DER
rather than copying the source, refuses any value carrying private key
material, and owns the projection by owner reference so a foreign Secret
at the same name is never overwritten. No tenant RBAC grants any verb on
internal.cozystack.io; a ValidatingAdmissionPolicy pins writes to
helm-controller as defence in depth.

The controller does not grant tenant read access — the lineage webhook
does, from the ApplicationDefinition's spec.secrets. This side stamps the
engine-agnostic internal.cozystack.io/tenant-ca label and forces a
re-admission when those selectors change.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-21 23:59:48 +03:00
IvanHunters 8f24cfb56c fix(platform): point migration fixtures and docs at renumbered migration 54
The Phase-2 worker-pool adoption migration was renumbered 52 -> 54 during
the rebase onto main (main added migrations 52 linstor-scheduler and 53
seaweedfs). Rename the adoption bats tests and their testdata directory,
and update the kubernetes-nodes pre-delete hook comment and the
kubernetes README so they reference migration 54. The bats MIG path
pointed at migrations/52 (now the linstor migration), so without this the
adoption unit tests would exercise the wrong migration.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-21 23:01:00 +03:00
Ivan Okhotnikov 49ef26fb80 docs(kubernetes): document Phase 2 split breaking changes and certSAN scope
Address review on #3315:
- README breaking-changes: worker pools move to KubernetesNodes (nodeGroups
  removed, adopted by migration 52); the parent Kubernetes CR's WorkloadsReady
  is now control-plane-only; ingressNginx needs an explicit ingress pool; do
  not edit Kubernetes CRs mid platform-upgrade.
- Note that the apiserver ClusterIP certSAN is a best-effort extra — workers
  dial by DNS, so a pool-less control plane is fine without it; the merge-patch
  array-replace predates the split.
- Confirm the pool VM pod is intentionally attributed to the parent Kubernetes
  application (byte-parity + lineage/quota under the owning cluster).

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:57:22 +03:00
Ivan Okhotnikov 3c204ca251 fix(kubernetes-nodes): immutable cluster/storageClass, validate pool version vs parent
Address review on #3315:
- Mark cluster and storageClass immutable (self == oldSelf): both feed object
  names / the KubevirtMachineTemplate content hash, so editing them on a live
  pool would rename everything and delete the running worker VMs.
- Validate the pool's Kubernetes minor against the parent cluster's
  KamajiControlPlane at render (skipped offline / when the parent is absent),
  so a pool created after the split cannot join workers ahead of the apiserver.
- Note in the snapshot suite header that the instancetype-sized default is not
  pinned offline (lookup nil); the e2e exercises it.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:57:22 +03:00
Ivan Okhotnikov 386053ba8f fix(platform): harden migration 52 (name overflow, images, adoption tests)
Address review on #3315:
- Name-length overflow no longer exits 1 (which would deadlock the platform
  pre-upgrade hook for every tenant). Instead pin the pool's worker objects
  with resource-policy=keep, skip the child HR, warn loudly, and continue.
- Copy only images.kubectl into the child values, not the parent's whole
  images map (waitForKubeconfig/talosCsrSigner are undeclared in the schema).
- Note that the pre-upgrade window is not quiet (a parent HR edit mid-upgrade
  renders the old chart and self-heals once the new artifact lands).
- Cover the adoption path directly: annotate keep + release-name, idempotent
  skip, unexpected-owner refusal, and the 6-hex KMT anchor rejecting a sibling
  pool's template.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:57:22 +03:00
Ivan Okhotnikov c11e3d6682 test(platform): guard migration 52 copies version/talos/images into child HR
Drives migration 52 against a fake kubectl and asserts the adopted
child HelmRelease carries the tenant's non-default talos.version,
version and images (plus group fields and storageClass fallback). Fails
against the pick(o;...) filter-parameter bug that dropped those keys and
rolled live worker VMs; passes against the pick($o;...) fix.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:57:06 +03:00
Ivan Okhotnikov 9d9068163d fix(platform): copy version/talos/images into adopted pool HR values
The adoption jq used a filter-parameter helper `def pick(o; ks)`; inside
the `reduce` the `.` context is the accumulator, so `o | has($k)` tested
the accumulator, not the source object, and `pick(.; ["version","talos",
"images"])` deterministically returned {}. The child HelmRelease was
therefore created without the tenant's talos/version, so Helm rendered
the chart defaults; since talos.version feeds the KubevirtMachineTemplate
content-hash, the KMT name changed and the MachineDeployment rolled every
live worker VM on upgrade — the exact churn this migration must avoid.
Caught by e2e on dev3 (reproduced on 2/2 real clusters). Switch pick to
a $-parameter so it is evaluated once at the call site.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:57:06 +03:00
Ivan Okhotnikov 3a52793483 test(kubernetes-nodes): cover GPU, kubelet-reservation and MHC render branches
The chart split removed the parent pool-render suites and the cross-chart
parity script, leaving the GPU device block, kubelet-reservation math and
maxUnhealthy IntOrString branches (all still in nodegroup.yaml /
talos-reconcile-job.yaml) uncovered. Add GPU and kubelet snapshot cases
so drift renaming the content-hashed KubevirtMachineTemplate fails CI,
and port the behavioral + failure-path assertions (GPU nodeLabels,
kubelet reserved/effective invariants and quantity rejections, MHC
integer/percentage quoting) to the child chart.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov a14413ad6a test(kubernetes): delete KubernetesNodes pools in e2e cleanup paths
The chainsaw harness created a KubernetesNodes CR but never reclaimed it,
leaking the pool CR, its child HelmRelease, release Secret and
WorkloadMonitor into the shared tenant-test namespace and breaking
rerun-in-same-sandbox (the surviving child HR left the MachineDeployment
un-rendered on retry). Delete kubernetesnodeses in all three cleanup
paths, before the parent CR so the pre-delete unpin hook runs.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov e615ca9253 chore(kubernetes): drop remaining nodeGroups residue from docs and fixtures
Remove the orphaned kubernetes.nodeGroups doc comment, the stale kubelet
reservation / nodeGroups storageClass prose in the README, and the inert
nodeGroups blocks in the test value fixtures.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov 4619fb3d0c fix(kubernetes-nodes): make unpin failures visible and scope its RBAC
Check the resource-policy annotation before removing it so a native pool
is a clean no-op while a real annotate failure fails the Job (retries
bounded, logs retained) instead of silently re-introducing the keep
leak. Scope the MD/MHC/WorkloadMonitor Role rules with resourceNames.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov fe958cb9ec test(kubernetes): provision e2e workers via KubernetesNodes
The chainsaw harness created the Kubernetes CR with spec.nodeGroups,
which the control-plane-only chart now ignores, so no MachineDeployment
was created and the suites timed out. Create a KubernetesNodes CR named
<cluster>-md0 (roles: [ingress-nginx]) so the same MachineDeployment is
produced and the MD/ingress waits pass; this also gives the split its
first e2e coverage.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov 3d8617a02a fix(kubernetes): do not hard-fail when the ingress addon lacks a pool
The render-time fail flipped existing degraded clusters from Ready to
Failed on upgrade, blocking all subsequent control-plane changes.
Restore the prior non-breaking behavior (the DaemonSet schedules nothing
if no ingress-roled node exists) and document the pool requirement in
NOTES.txt instead of enforcing it at render time.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov a76d947517 test(kubernetes-nodes): drop parity script from make test
The cross-chart parity script was removed (the parent no longer renders
pools to diff against); the render snapshot under helm unittest is the
drift guard now and already runs in CI.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov 87b41b5fa8 chore(kubernetes): drop stale nodeGroups references from fixtures and docs
Remove dead nodeGroups blocks from the CI values fixtures and point the
storage-immutability doc at the kubernetes-nodes per-pool storageClass.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov 7070cb0fe7 feat(kubernetes-nodes): unpin adopted worker objects on delete
Migration 52 pins adopted worker objects with helm.sh/resource-policy=keep
for the transition window. Left in place it leaks the objects when an
adopted pool is deleted. Add a pre-delete hook that strips the annotation
from the release's own worker objects so Helm deletes them cleanly;
runs on delete only (never on upgrade, so it cannot race the parent
re-render) and is a no-op for natively-created pools.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov 82074cb9db fix(kubernetes): regenerate deepcopy after nodeGroups removal
types.go dropped NodeGroups/NodeGroup/GPU/Kubelet/NodeHealthCheck but
the generated deepcopy still referenced them, so the api module failed
to build. Regenerated with controller-gen.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov a8286a3ac0 fix(platform): harden worker-pool adoption migration
Anchor the KubevirtMachineTemplate sweep to the 6-hex content-hash
suffix so a pool name that is a prefix of a sibling (md0 vs md0-large)
no longer mis-adopts the sibling's KMT and deadlocks the migration.
Adopt the worker objects before creating the child HelmRelease so
helm-controller cannot start the child install into un-adopted objects
and fail Helm ownership validation. Emit the same install/upgrade
strategy (RetryOnFailure) and maxHistory cozystack-api sets, add a
pre-flight release-name length check, and document the rollback path.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:56:43 +03:00
Ivan Okhotnikov e96ea2b0f3 test(kubernetes-nodes): replace cross-chart parity with a render snapshot
The parent no longer renders pools to diff against, so the one-shot
extraction parity check is replaced by a self-contained golden snapshot.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:55:15 +03:00
Ivan Okhotnikov 1e5ba15f6d test(kubernetes): update tests for the control-plane-only chart
Drop pool-render tests; assert no pool objects render; add ingress-nginx
guard tests.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:54:25 +03:00
Ivan Okhotnikov 4bd79942f3 feat(kubernetes): require an ingress-nginx pool when the ingress addon is enabled
With pools moved out of the chart the ingress-nginx addon can no longer
rely on an auto-provisioned md0; fail the render with a clear message
when addons.ingressNginx.enabled and no pool carries roles: [ingress-nginx].

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:54:25 +03:00
Ivan Okhotnikov 31336f0e18 feat(kubernetes)!: remove nodeGroups from the Kubernetes API
Worker pools are now managed via KubernetesNodes resources; existing
pools are adopted by migration 52.

BREAKING CHANGE: the Kubernetes CR no longer accepts spec.nodeGroups.
Create KubernetesNodes resources for worker pools instead.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:54:25 +03:00
Ivan Okhotnikov 92b9785daa refactor(kubernetes): remove worker node pools from the chart
The KubevirtMachineTemplate/MachineDeployment/MachineHealthCheck/worker
WorkloadMonitor, the preserve-old-KMT block, the talos-reconcile Job and
its RBAC now live in the kubernetes-nodes chart. The kubernetes chart
renders the control plane only.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:54:24 +03:00
Ivan Okhotnikov 86ecedb3c0 feat(platform): adopt worker pools into kubernetes-nodes releases (migration 52)
Pre-upgrade migration for the Phase 2 kubernetes chart split. For every
parent kubernetes-<cluster> HelmRelease it creates one child
kubernetes-nodes-<cluster>-<pool> HelmRelease and re-annotates the
existing worker objects onto it with helm.sh/resource-policy=keep, so
the now control-plane-only parent upgrade does not prune the tenant's
running workers. Byte-identical child render (PR-2a golden parity) keeps
the KubevirtMachineTemplate content-hash stable, so no worker VM churns.
Idempotent and strict (exit 1 on any failure). Bumps targetVersion to 53
(contiguous with main; rebase to 53/54 if another 52 lands first).

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:52:05 +03:00
Ivan Okhotnikov b2e1707632 fix(kubernetes-nodes): drop dead ephemeralStorage guard, clarify parity scope
The ephemeralStorage guard could never fire (the pool group is built from a
fixed set of chart values that never includes ephemeralStorage), so it read
as protection that is not there — remove it. Note in the parity script header
that only the four pool objects are compared, not the talos-reconcile Job.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:11 +03:00
Ivan Okhotnikov 326dbe0a5b test(kubernetes-nodes): cover GPU and kubelet pool shapes in render parity
Extend the golden-parity matrix beyond the resources-sized pool to also
diff a GPU pool and a kubelet-override pool against the parent render, so
future drift in those branches (which would roll the content-hash-named
KubevirtMachineTemplate) is caught in CI.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:11 +03:00
Ivan Okhotnikov f21365c4f0 fix(kubernetes-nodes): address review nits in guard comment and NOTES
Reference the adoption migration by role (Phase 2b) instead of a number
that does not exist on this branch, and correct the NOTES operational-note
count.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:11 +03:00
Ivan Okhotnikov 9e2f7cc953 test(kubernetes-nodes): run render parity in make test
Wire tests/render-parity.sh into the chart's test target so CI verifies
the pool objects stay byte-identical to the parent render.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov 3d10a1c431 fix(kubernetes-nodes): keep applied TalosConfigTemplate byte-identical to the parent
Restore the inline machineconfig comments (nameservers, GPU node labels)
inside the reconcile Job's data block so the applied TalosConfigTemplate
matches the parent's, avoiding drift on adoption.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov 73be7032a3 fix(kubernetes-nodes): drop unbacked cluster option source
The cluster field referenced x-cozystack-options source 'kubernetes',
which has no provider (cozystack-api returns NotFound), so the dashboard
dropdown 404s. Drop the annotation so the field is a plain text input;
a real provider listing Kubernetes CRs is a follow-up.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov fe32fd8e58 fix(kubernetes-nodes): scope KMT preservation by release and guard foreign pool names
Replace the collision-prone name-prefix match (pool 'gpu' matched
sibling 'gpu-large-<hash>') with a release-name annotation scope, and
add a render-time guard that fails cleanly when a pool's MachineDeployment
already exists under a different release (e.g. naming a pool after a
parent nodeGroup such as the default md0). Document the naming caveat in
NOTES.txt.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov 1e8d44995c fix(kubernetes): scope worker KMT preservation to the release
The preserve-old-KubevirtMachineTemplate block re-emitted any KMT owned
by the CAPI Cluster and referenced by a MachineSet. A kubernetes-nodes
pool's KMT lives in the same namespace under the same Cluster, so the
parent would re-emit it with a foreign meta.helm.sh/release-name and its
next upgrade would fail Helm ownership validation. Restrict the block to
KMTs whose release-name annotation matches the parent release.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov e7947dd0e7 feat(platform): register kubernetes-nodes application
Add the kubernetes-nodes-rd package (ApplicationDefinition), the
cozystack.kubernetes-nodes-application PackageSource (kubevirt variant,
kubernetes-nodes + kubernetes-nodes-rd components), and wire it into the
iaas bundle. Mirrors the vm-instance/vm-disk registration precedent.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov 277b32306c test(kubernetes-nodes): add unit tests and golden-parity check
Unit tests cover naming/selectors, clusterName wiring, autoscaler
annotations, omitted replicas, bootstrap TCT reference, KMT hash name,
worker WorkloadMonitor, and the empty-cluster / mis-prefixed-release
render guards. render-parity.sh diffs the pool objects against the
parent kubernetes chart and asserts byte-identity.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov f842c0a17e feat(kubernetes-nodes): generate API types for KubernetesNodes
Generated by cozyvalues-gen + controller-gen from values.yaml.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Ivan Okhotnikov fd14126bea feat(kubernetes-nodes): add worker node pool chart
Introduce the kubernetes-nodes application chart: one HelmRelease per
worker node pool, attached to a parent Kubernetes cluster by name
convention (kubernetes-<cluster>). Renders the KubevirtMachineTemplate,
MachineDeployment, MachineHealthCheck, worker WorkloadMonitor and the
talos-reconcile Job with its RBAC, extracted from the monolithic
kubernetes chart. Pool objects are byte-identical to the parent's render
for the same group, so a later adoption never churns live worker VMs.

This is the additive half of the Phase 2 split (community#8); the parent
kubernetes chart is left unchanged.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
2026-07-21 22:35:10 +03:00
Myasnikov Daniil 427ed68761 fix(backupstrategy-controller): give chBackupClientImage a producer
platform-migrations was pinned at two different digests: v1.5.0 in
packages/core/platform/values.yaml, stamped by its Makefile, and
v1.4.0-rc.2 as .backupStrategyController.chBackupClientImage, stamped by
nothing at all. The second is the Altinity strategy Pod's curl+jq runner,
deliberately reusing the migrations image rather than shipping a second
one-binary tag that release CI would have to re-digest every cut. That
reuse only holds while the two move together, and the values.yaml comment
said so -- but a comment was the only thing enforcing it, so the copy with
no producer froze two minors back.

This is a release blocker, not just staleness. Promotion retags by digest,
copying every collected <repo>@<digest> to <repo>:<stable>. Two digests
under one repository produce two copies competing for one destination tag:
the first wins and the second hits the write-once guard in
hack/promote-retag.sh, which fails the promotion. A dry run against the
tree emitted exactly that pair. v1.6.0 is the first release cut through
promotion rather than a full rebuild, so it would have been the first to
hit it -- every earlier release ran `make build`, which restamped both
copies as a side effect and hid the missing producer.

packages/core/platform/Makefile now stamps both keys with the same ref, so
they are in lockstep by construction rather than by instruction, and the
committed value is aligned to the digest core/platform already carried.
The image is used only as a curl + jq + ca-certificates runner, so moving
it forward two minors changes no behaviour the strategy depends on; it
renders and the chart's helm-unittest suite passes.

hack/image-pin-consistency.bats adds the general guard -- no repository
pinned at more than one digest, driven through the real promotion selector
so it tracks whatever set promotion acts on -- plus a specific lockstep
assertion so a recurrence names its cause. The general check could not be
added before now: it fails on the very duplicate this commit removes.

Relates to #3143 (the unowned-pin class) and unblocks the v1.6.0 promote.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 23:15:46 +05:00
Myasnikov Daniil 91c076137f test(ci): pin shape 3's per-map digest binding
The shape-3 fixture held a single repository/digest pair, so a rule that
binds one digest per file and reuses it for every repository satisfied
it -- and passed all three suites. That is not a theoretical mutation:
against the real linstor values it hands piraeus-server's digest to
linstor-csi, which retags and mirrors the wrong image under the right
name. Silently shipping wrong bytes is worse than the missing-tag bug
this branch started from.

The fixture now carries two shape-3 maps with distinct digests in one
file and asserts both exact canonical refs, so the digest must be bound
per map rather than per file.

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 22:54:19 +05:00
Myasnikov Daniil a16f218875 test(ci): correct a stale comment and isolate the fail-open fixtures
Two review nits, both comment- or fixture-level.

The completeness oracle's comment still said the per-shape test "matches
on the repository substring, so a shape that loses its registry rejoin
still satisfies it". That stopped being true in the same commit that
introduced it -- the substring match was replaced with an exact
canonical-ref assertion, which kills both the shape-2 and shape-3 rejoin
mutations. The comment understated coverage.

The fail-open test copied the tree once and looped three rc versions over
it, so the second and third iterations ran against a tree the first
rewrite had already mutated. Each now gets a fresh copy, so each tests
the input it names.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 22:51:00 +05:00
Myasnikov Daniil 539a7b8e0b test(ci): catch shape-4 stapling inside global, and spaced marker keys
Round-5 review. The shape-4 negative fixture placed its decoy outside
`global` entirely, so it only caught a document-wide broadening. Mutating
the rule to `.global | ..` instead passed the per-shape test and all three
suites while still stapling the document-level registry onto unrelated
maps inside `global` -- manufacturing first-party refs that promotion
would then retag and mirror to destinations nothing ever built. The
fixture now carries two decoys at different depths, one inside `global`
but outside `global.images` and one outside `global`, and the mutation is
caught.

Markers also required the key to be immediately followed by its colon, so
`registry : ghcr.io/...` was missed. No producer writes that spelling and
it is valid YAML rather than anything in the tree, so this is theoretical
-- but tolerating whitespace costs one character class and keeps the
comment's exhaustiveness claim true lexically as well as structurally.
Marker count is unchanged at 43 files with the same two allowlist entries.

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 22:26:38 +05:00
Andrey KolkovandClaude Opus 4.8 c13a102274 feat(etcd-operator): bump to v0.5.3 (S3 checksum fix for non-AWS backends)
v0.5.3 carries the S3 request-checksum fix (cozystack/etcd-operator#342): the
snapshot agent sets RequestChecksumCalculation=WhenRequired on BOTH the S3 client
and the transfer manager, so multipart snapshot uploads (>5 MiB — every real etcd
snapshot) no longer carry the CRC32 STREAMING-UNSIGNED-PAYLOAD-TRAILER that Ceph
RGW rejects with 400 InvalidArgument. Also pulls in #340 (observed EtcdMember
versions) and #341 (--watch-namespace).

- system/etcd-operator: appVersion v0.5.2 -> v0.5.3 (image tag follows AppVersion),
  bump ETCD_OPERATOR_REF, update deployment unittest expectations.
- system/etcd-operator-crds: re-vendor CRDs from v0.5.3. etcdmembers gains the
  additive #340 status.version field + "Running" printer column and picks up the
  peerAutoTLS spec field the vendored copy was already missing (all additive,
  backward-compatible; etcdclusters/etcdsnapshots unchanged). Fix the stale
  `make update`: upstream has no config/crd kustomization — CRDs live in
  charts/etcd-operator/crd-bases — so vendor those directly and stamp
  helm.sh/resource-policy: keep.

Verified end-to-end on a Ceph RGW backend: the v0.5.2 (client-only) agent fails a
multipart snapshot upload with the 400 above; v0.5.3 uploads an 11 MiB multipart
snapshot successfully.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
2026-07-21 21:12:10 +04:00
Myasnikov Daniil fa4ae65d04 test(ci): assert usable refs, quoted keys, and shape-4 over-collection
Round-4 review. Both reviewers independently found the declared-extra
test was passing while the thing it guards was broken, and each found one
more.

That test asserted `grep -q "@sha256:<digest>"`, which any token
containing the digest satisfies. Its fixture was a split-host ref, which
a textual scrape can never reconstruct -- only the contiguous tag value
is recoverable, so ref_repo() reduces it to `latest` and every consumer
drops it at the ownership filter. The fixture was therefore
enumerated-but-uncollected, which is exactly the conflation the test was
added to catch, and gutting the scrape to emit a bare digest with no
repository left the whole suite green. The fixture is now a contiguous
ref in an unparseable template (single-quoted, so the scrape's new
quote-tolerance is exercised) and the assertion pins the exact canonical
ref. The limitation that a split-host ref in an unparseable file is
recoverable by neither route is recorded rather than implied away: the
file-level marker check is what covers that case.

The scrape also captured a leading quote, so any quoted ref in a .tag
file or declared extra yielded a token every consumer silently discards.
Nothing in the tree is affected today, but it meant the double-pass
rescued only unquoted refs.

Markers missed quoted YAML keys: an unenumerated template using
"registry": / "repository": passed the entire suite. They now tolerate
quoting on the key as well as the value, still matching 43 files with no
new allowlist entries.

Shape 4 could be broadened from .global.images[] to every map in the
document and pass everything, because asserting that the expected ref
appears cannot see an unexpected one. That scoping is load-bearing -- the
host is a document-level key, so a global binding staples kubeovn's
registry onto unrelated repositories and manufactures owned refs that
were never built, which promotion would then retag and mirror. The shape-4
fixture now carries a digest-pinned sibling outside global.images and the
test asserts its absence.

Two per-shape fixture gaps closed while there: shape 2 had no sibling
`registry:`, so it losing its rejoin was invisible here and caught only
by one nightly-mirror fixture; and shape 1 had no tagless case, so
narrowing it to require a :tag before the digest went undetected -- a
digest-only pin is an ordinary Helm spelling.

Also corrected the residual-limit comment on the markers. Coverage is
exhaustive, not merely broad: the collector joins registry + "/" +
repository, so a split can only fall on a path-separator boundary and
markers 1-3 cover all four possibilities. A split elsewhere does not need
covering because it does not produce a first-party ref.

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-21 22:05:48 +05:00