Commit Graph
7777 Commits
Author SHA1 Message Date
Aleksei Sviridkin 5be4e417b3 docs(mongodb): correct the operator-behaviour claims and the upgrade impact
The TLS notes made several claims the operator's own behaviour contradicts.
The CA is issued for a year and renewed about a month early, not for a decade
with rotation disabled — so a bundle copied once stops verifying, and the
retrieval steps now say to re-read it. Sharded mongos SANs are complete,
because the operator emits them for every topology; the real gap is that no SAN
covers the single external LoadBalancer fronting mongos, so external hostname
verification is unsupported there. preferTLS is the operator's own default
rather than a posture adopted to accommodate backups, which run under the same
certificates. External access in replica set mode is one LoadBalancer per
member with driver-side primary discovery, not a single load-balanced
endpoint — the old description contradicted the external SAN note. And the
posture section named only one of its two inherited weaknesses: unverified
clients are accepted as well as plaintext ones.

The lifecycle section was wrong in both directions. The operator owns its TLS
Secrets and its cert-manager objects through the CR, so they are collected on
uninstall rather than left behind. The rotation snapshots are the opposite of
what was written: the operator deletes them itself once the merged CA has
landed, and reads the previous CA back out of them while a rotation is in
flight — so the advice to delete them by hand could destroy the only copy of
the old CA. Only an interrupted rotation strands them.

Document what an upgrade actually does to a release already running with
external access: the new hostnames become certificate SANs, the leaf is
reissued, and the replica set rolls. Gate the retrieval steps on the component
that publishes the trust anchor and on cert-manager, without which the declared
source does not exist and the command returns nothing.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-01 11:18:42 +03:00
Aleksei Sviridkin 2670f7efc1 build(mongodb): run the chart's unit tests
The unit-test runner discovers charts by probing for a test target, and this
chart had none — so its suites had never run in CI.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-01 11:18:42 +03:00
Aleksei Sviridkin 612ae08e5f fix(mongodb): omit splitHorizons where it would render empty
With no external hostname to advertise — zero replicas, or a tenant host the
controller has not populated yet — the horizons range produced nothing and the
key was emitted with a null value. The CRD types it as an object without
nullable, so the API server rejected the whole PerconaServerMongoDB and the
HelmRelease with it. The values schema permits zero replicas, so this was
reachable. Guard on both and omit the key entirely.

Pin the surrounding invariants, which were asserted in comments and enforced
nowhere. The chart must render no cert-manager objects of its own, since the
operator owns the chain and anything rendered here would be orphaned while
looking like it participated. It must set neither of the two fields that
switch TLS off at defaulting time while the rendered mode still reads
preferTLS. And the external-access case now supplies a tenant host, so it
isolates the flag it names rather than passing on the host guard.

Correct the header comment, which fused two false claims: that setting
spec.secrets.ssl is what keeps TLS on, and that it is what engages the
cert-manager integration. The operator defaults that name unconditionally and
runs the cert-manager path regardless.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-01 11:18:42 +03:00
Aleksei Sviridkin 27f71bfbe9 fix(mongodb): deliver the TLS trust anchor without private key material
The tenant-facing paths handed over both Secrets the PSMDB operator creates
for TLS. <release>-ca-cert holds the CA private key, so read access is a full
trust-chain compromise; <release>-ssl holds the server private key, so read
access allows impersonation. A client verifying the endpoint needs ca.crt and
nothing else, so neither object belongs on a tenant-facing path.

Declare the trust anchor by name instead. The operator creates its CA Secret
itself and offers no way to label it, so the ApplicationDefinition names it and
the CA-extraction controller lifts ca.crt alone into the key-free projection.
The include selector matches that projection by label, and the dashboard Role
drops both key-bearing names without gaining the projection by name: a name
grant conveys whatever occupies the name, while the label path conveys only
what the platform vouched for.

Witness both delivery paths against a running cluster, since the two that
matter are separate. The Role is asserted as an exact grant with bounded rule
counts, because a rule with no resourceNames — or one granting every resource
— would otherwise pass an absence check while conveying every Secret in the
namespace. The label path is asserted as a negative on all three key-bearing
Secrets. A bats check pins the declaration's names and keys, which nothing else
covers: the field is pruned at admission until the controller ships, so drift
there would surface as an empty trust anchor with nothing red.

Drop the mongodb-rd unit test. That package has no test target, so the file
never ran, and what it asserted was the leak.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-01 11:18:41 +03:00
Arsolitt 7f8f32b1ba fix(mongodb): always enable operator-managed TLS, drop tls.enabled field
Address review feedback from lexfrei (B3): the tri-state tls.enabled field
could not actually disable TLS. The PSMDB operator defaults spec.tls to
preferTLS whenever it is unset, so tls.enabled: false rendered no spec.tls
while the operator still served TLS — a misleading no-op.

Remove the field and unconditionally render spec.secrets.ssl and
spec.tls.mode: preferTLS to match operator behaviour. splitHorizons SAN
injection now keys off external alone. Also drop the Makefile jq patch that
re-injected the tls schema property.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-09-01 11:11:27 +03:00
Arsolitt e9f276195f test(mongodb): cover TLS modes, splitHorizons, and RBAC
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
2026-09-01 11:10:59 +03:00
Arsolitt f6816e1f44 feat(mongodb): add operator-managed TLS with external hostname SANs
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
2026-09-01 11:10:55 +03:00
Andrei Kvapil 53412c3033 fix(valkey): add cozystack-engine dependsOn edge to the PackageSource
The valkey-application PackageSource ships system/valkey-rd (an
ApplicationDefinition); it must dependOn cozystack.cozystack-engine so the
ApplicationDefinition CRD is registered before the -rd HelmRelease reconciles.
Mirrors redis-application.yaml and satisfies check-applicationdefinition-crd-ordering.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:44:29 +04:00
Andrei Kvapil 785b0feb68 feat(valkey): use native engine:Valkey instead of a redis-compat image
With the operator on freshworks-oss/redis-operator, RedisFailover supports
spec.engine: Valkey — the operator runs valkey-server/valkey-cli natively. Drop
the thin Redis-compat wrapper image entirely and point the chart at the official
valkey/valkey image directly:

- RedisFailover: set spec.engine: Valkey, image valkey/valkey:<version>
- remove images/ (Dockerfile + per-version tags), the image build target, the
  root build wiring and the images helmignore entry

Depends on the freshworks-oss operator migration.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:44:29 +04:00
Andrei Kvapil a32f26df84 fix(valkey): address review feedback
- servicescrape: fix target_label -> targetLabel (snake_case was ignored by the
  VM operator), drop the empty metricRelabelConfigs and annotations keys
- service: drop the redundant inner external conditional (the whole manifest is
  already gated on .Values.external)
- update-versions.sh: write through the trap-tracked temp file and cat back to
  preserve values.yaml permissions
- regenerate README.md parameter table with the pinned cozyvalues-gen v1.6.0
  (column widths differed from the committed version)

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:44:07 +04:00
Andrei Kvapil df1345f56d test(valkey): port e2e to chainsaw suite
The app e2e harness on main is chainsaw (hack/e2e-chainsaw/<app>/), not the
older bats layout. Add hack/e2e-chainsaw/valkey/ mirroring the redis suite so
select-e2e.sh maps the valkey-application source to the valkey suite, and drop
the misplaced hack/e2e-apps/valkey.bats.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:44:07 +04:00
Andrei Kvapil 440c3b58f3 test(valkey): add e2e test and wire build, labeler and docs
Add the Valkey e2e bats test (mirrors redis), build the compat image in the
top-level build target, map the valkey scope to area/database in the PR
labeler, and list valkey among charts with an immutable storageClass.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:44:07 +04:00
Andrei Kvapil 4e6313b608 feat(valkey): register Valkey application in the platform
Wire the managed Valkey app into the platform: add the valkey-rd package
carrying the Valkey ApplicationDefinition, the valkey-application PackageSource
(reusing the existing redis-operator), enable it in the paas bundle, and grant
tenant roles access to the valkeys resource.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:43:10 +04:00
Andrei Kvapil 6660a44376 feat(valkey): add managed Valkey application package
Add a managed Valkey service mirroring the managed Redis app. Valkey is the
BSD-3-Clause, Linux Foundation-governed fork of Redis 7.2 and a drop-in
replacement for it, offered as a governance-clean alternative to the now
source-available (AGPLv3/SSPL/RSALv2) Redis 8.x.

Valkey runs on the same Spotahome redis-operator: the operator hardcodes the
redis-server command and redis-cli in its (non-overridable) probe scripts, so
the chart deploys a thin compat image (FROM valkey/valkey:<ver>) that adds the
redis-* symlinks over the official upstream binaries — no Valkey code rebuilt.

Versions offered: v8 (8.1.8, default) and v7 (7.2.13, the BSD Redis 7.2 line).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-09-01 11:43:10 +04:00
myasnikovdaniil e84a446893 chore(keycloak): bump KMS proxy to 0.2.2 (#3995)
## What this PR does

This PR updates keycloak-kms-proxy to v0.2.2 and pins image to published
digest. New version fixes decryption for reads whose result set spans a
join.

### Screenshots

Not applicable.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
chore(keycloak): update keycloak-kms-proxy to v0.2.2 with a fix for decrypting reads whose result set spans a join
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated the Keycloak encryption component to a newer verified image
version.
* Improved deployment reliability by pinning the image to a specific
integrity digest.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-31 12:07:39 +05:00
Alexey Artamonov 06624bacd0 fix(vm-disk): fail on storageClass/source edit of existing disk
Editing a VMDisk's storageClass or source after its DataVolume already
exists was silently ignored: dv.yaml reused the existing DataVolume.spec
verbatim (DataVolume.spec is immutable in CDI), so the edit rendered as a
no-op while the HelmRelease reported success and the CR diverged from the
cluster.

When a DataVolume already exists, fail with a clear message if the
requested storageClass or source differs from what the disk was created
with. Both compare the operator's input recorded in an annotation at
creation (vm-disk.cozystack.io/storage-class and .../source), written only
at creation and preserved verbatim afterwards — never overwritten with an
unapplied edit. Recording the input rather than reading the live object
matters on both axes: the image-clone PVC name carries a chart-version
prefix (vm-image- was renamed to vm-default-images-, commit bec35e3a) that
drifts across upgrades with no edit, and the live PVC storageClassName of an
adopted or foreign DataVolume may legitimately differ from the CR default
without being a change. The annotation key presence (not its value) marks a recorded class, so a disk
created with the cluster default "" is recorded and still caught if later
changed to a concrete class.

Disks with no annotation (created earlier, or out-of-band) are handled
conservatively so a working release never deadlocks: source is compared
against the stored spec.source only when that stored source is a shape the
chart itself produces, so an adopted DataVolume using spec.sourceRef or a
non-chart source (registry, snapshot, http+secretRef, ...) is left alone
rather than deadlocked. The image-clone exemption fires only when the stored
source is a cozy-public pvc AND the request is itself an image, so a change
away from an image source to another type is still caught. storageClass has
no safe live fallback, so an un-annotated disk is not guarded here (apiserver
CEL, #2657, is the eventual guard). An unrecognized source type
now fails explicitly. Storage growth is unaffected and still handled by
pvc-resize-hook.yaml.

Mark source as immutable in values.yaml so the generated schema carries
the same x-kubernetes-validations rule already present on storageClass,
and document the contract, the exemption, and the source-annotation schema
coupling in docs/storage-immutability.md and the chart README.

Add helm-unittest coverage for the immutability guard (rejection incl.
clearing storageClass to empty and changing a non-image disk to image,
no-op reuse with annotations preserved, legacy image-disk drift, annotation
non-poisoning, un-annotated storageClass skip, adopted default-class disk,
unknown source) and initial-creation source rendering.

Fixes #2985

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-08-28 18:26:36 +03:00
Myasnikov Daniil 6d9444056d chore(keycloak): bump KMS proxy to 0.2.2
Pin the released image by digest so package rendering is reproducible.

Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-28 17:11:41 +05:00
Andrey Kolkov b9a95a0c4a fix(postgres): let in-place restore re-bootstrap instead of wedging (#3959)
## What this PR does

An in-place `RestoreJob` (one whose `targetApplicationRef` is omitted,
so the app is restored back into itself) re-bootstraps the Postgres app
under its own name. On stock v1.6.2 it never comes back up — the cluster
is deleted and then hangs in `Setting up primary` with
`*-full-recovery-*` pods crash-looping, or the HelmRelease upgrade fails
outright. There are two independent causes, both in the same restore
patch, and both surface only in place because a to-copy restore targets
a different app.

**1. Archive-vs-recovery `serverName` collision.** The driver set the
restored cluster's WAL-archive `serverName` equal to the recovery source
(`bootstrap.serverName = bootstrap.oldName = sourceServerName`), while
the chart pinned the archiver `serverName` to the release name. In place
those coincide, so the cluster archived onto the source's own WAL prefix
and `barman-cloud-check-wal-archive` refused to start with `Expected
empty archive`. Fixed by giving the restored cluster a fresh
`bootstrap.newServerName`, keyed by the `RestoreJob` UID (unique per
restore, stable across reconciles); the chart archives there while
`externalClusters` still recovers from `bootstrap.serverName`. This is
what manual `CNPG` strategies hit (their `serverName` defaults to the
cluster name, including the shipped
`examples/backups/postgres/10-cnpg-strategy.yaml`); the `cozy-default`
flow dodged it because its `serverName` is `<namespace>-<app>`.

**2. `useSystemBucket` guard.** The restore populates explicit S3
coordinates (`destinationPath` / `s3CredentialsSecret` / `endpointCA`)
but left `backup.useSystemBucket=true`, so the chart's
`bootstrap.enabled is incompatible with useSystemBucket` guard failed
the HelmRelease upgrade and the cluster never re-rendered. This is what
the default `cozy-default` (system-bucket) flow hits on an in-place
restore. Fixed by clearing `useSystemBucket` in the restore patch, since
the app is now on the explicit-creds path. Verified live on a v1.6.2
cluster: with `useSystemBucket` cleared, the same in-place restore
re-bootstraps to a healthy cluster and the sentinel row survives.

Both are exercised only by an in-place restore; the postgres backup e2e
drove only to-copy, so neither surfaced in CI. A step 50 in-place
restore now guards cause 1 (manual-strategy flow); cause 2 is covered by
a Go unit assertion (a system-bucket in-place e2e needs the platform
bucket wired in CI and is left as a follow-up).

This is a bug in released v1.6.2 (chart and driver are byte-identical on
`main`), so it is a backport candidate.

### Screenshots

Not applicable (no UI changes).

### Downstream repositories

The only externally visible schema surface is one new field,
`spec.bootstrap.newServerName`, in the postgres `values.schema.json` /
CRD; it is set by the CNPG backup driver on restore and never authored
by tenants, exactly like the existing `bootstrap.serverName` /
`bootstrap.oldName`. (The `useSystemBucket` clearing is internal to the
Go driver — `backup.useSystemBucket` already exists in the schema.) The
chart `README.md` is regenerated from `values.yaml` automatically by the
website docs bot on release, and `terraform-provider-cozystack` models
user-authored spec (its guard tracks top-level fields, not driver-set
nested ones), so neither needs a change in practice. Leaving every box
unticked per the trigger-map guidance so a maintainer confirms rather
than acting on a speculative follow-up.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(postgres): in-place restore no longer wedges — the restored cluster archives WAL under a fresh serverName distinct from the recovery source, and a system-bucket app is switched to explicit credentials so the bootstrap/useSystemBucket guard no longer fails its HelmRelease.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added support for specifying a distinct WAL archive destination when
restoring PostgreSQL clusters.
- In-place restores now automatically use a fresh archive location while
continuing to recover from the original source.
- Added configuration and schema support for the optional
`bootstrap.newServerName` setting.

- **Bug Fixes**
- Prevented archive conflicts that could cause in-place restores to
fail.

- **Documentation**
  - Documented the new restore archive configuration and behavior.

- **Tests**
- Added coverage for unique restore destinations and in-place restore
validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-28 15:08:35 +04:00
Andrey Kolkov ac00577eb7 fix(clickhouse): scheme the backup S3_ENDPOINT on the system-bucket flow (#3963)
## What this PR does

On the default `cozy-default` / `useSystemBucket` backup flow, the
ClickHouse chart pointed the `clickhouse-backup` sidecar's `S3_ENDPOINT`
straight at the projected `cozy-backups-creds.endpoint`, which the
backup credentials projector stores as a **bare host** (no scheme) by
design. `altinity/clickhouse-backup` runs `S3_ENDPOINT` through the
aws-sdk-go-v2 endpoint resolver, which rejects a scheme-less value with
`Custom endpoint 's3.host' was not a valid URI` — so every ClickHouse
backup and restore on the default flow failed. Reproduced live on a
v1.6.2 cluster: a `cozy-default` `BackupJob` against a `useSystemBucket:
true` ClickHouse hangs, and the sidecar logs `S3 ResolveEndpoint ... was
not a valid URI`.

The fix prepends `https://` on the `useSystemBucket` path via a helper
env plus `$(VAR)` expansion (a `secretKeyRef` value can't be
string-interpolated), mirroring the platform's
`backupstrategy-controller.endpoint` helper which force-prepends
`https://` for the always-TLS external S3 ingress. The legacy and
bring-your-own-Secret paths keep reading `endpoint` verbatim, since that
value is tenant-supplied and already carries its own scheme.

ClickHouse was the only `cozy-default` app hit by this. I audited every
default-flow app: the CNPG/postgres, etcd and velero strategies already
get a scheme'd URL from the endpoint helper, and mariadb and
foundationdb deliberately pair a bare host with a separate TLS flag;
mongodb reads a full URL from its own chart values. So this is the
single scheme-class fix needed.

Not addressed here (separate, non-scheme gaps surfaced during the audit,
flagged for follow-up): MongoDB has no `useSystemBucket` wiring at all
(its `cozy-default` strategy assumes a manually-configured
`backup.endpointURL` + per-app S3 creds), and FoundationDB's
self-signed-CA trust to the operator pod is a known gap.

### Screenshots

Not applicable (no UI changes).

### Downstream repositories

No schema or API change (only a chart template and its helm-unittest).
`values.yaml` / `values.schema.json` / the CRD are untouched, so nothing
downstream is affected. Leaving every box unticked per the trigger-map
guidance.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(clickhouse): backup and restore now work on the default system-bucket (useSystemBucket) flow — the clickhouse-backup sidecar receives a scheme'd https:// S3_ENDPOINT instead of the projected bare host, which the S3 client rejected as an invalid URI.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Corrected ClickHouse backup endpoint configuration when using the
system bucket.
* Ensured backup connections consistently use the required HTTPS
endpoint format.
* Improved handling of backup credentials across system bucket, legacy,
and custom-secret configurations.

* **Tests**
* Added coverage validating endpoint and credential wiring for supported
backup configurations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-28 15:05:14 +04:00
Myasnikov Daniil c88da18139 test(tests): keep trace switch suite ASCII-only
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-28 15:31:22 +05:00
Myasnikov Daniil e949f39800 fix(e2e): preserve previous-log cap slots
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-28 15:29:29 +05:00
Myasnikov Daniil 50ed307fef fix(backupstrategy-controller): gate the default MongoDB Strategy on its CRD
Upgrading a cluster to a build that carries the new MongoDB backup driver
wedges the backupstrategy-controller release permanently:

  Helm upgrade failed: resource mapping not found for name:
  "cozy-default-mongodb" ... no matches for kind "MongoDB" in version
  "strategy.backups.cozystack.io/v1alpha1"; ensure CRDs are installed first

This chart ships its strategy.backups.cozystack.io CRDs as ordinary
templates, and Helm resolves every document in a release manifest through
the cluster's RESTMapper in Build() BEFORE applying any of them. A
Strategy CR whose CRD is not served yet therefore fails Build(), so the
release applies ZERO objects — including the CRD that would have made the
mapping resolve — and every retry renders the same manifest. Nothing
recovers on its own.

The chart already knows about this trap: it is why every default Strategy
gates on the bucket name. That gate covers the first install of a fresh
cluster, where nothing is resolvable and no Strategy renders at all. It
cannot cover shipping a NEW kind to a chart that is already installed,
which is what happened here: the bucket name resolved releases ago, so
the first upgraded revision renders the new CRD and the new Strategy CR
together and wedges.

Gate the CR on its CRD being served, via a crdEstablished helper that
asks about the CRD OBJECT rather than listing the custom kind — `lookup`
on a kind the apiserver does not serve is a render error, not an empty
result, which would trade this wedge for a different permanent failure.
apiextensions.k8s.io/v1 is always served, so that lookup can only answer
yes or no. Established=True rather than mere existence, because that is
when the kind enters the RESTMapper.

Convergence afterwards is the mechanism the bucket gate already relies on
and is NOT the HelmRelease interval: once the CRD is served, the
controller's DefaultObjectsGate sees the kind mapped and the CR absent
and forces a real Helm upgrade whose render includes it. Kinds that do
not map at all are skipped there (meta.IsNoMatchError), which is what
stops the gate forcing upgrades against a render that cannot yet produce
the object.

Found by the release upgrade E2E lane on the v1.6.2 → main path, where
`mongodbs.strategy.backups.cozystack.io` is the only definition added
since the tag.

Verified against a live cluster that is in exactly the pre-upgrade state
(seven older strategy CRDs, no mongodbs): `helm template --dry-run=server`
of main's chart fails with the error above and produces zero documents,
while the same render with this change succeeds with 21 documents and the
Strategy correctly withheld. The helper's two branches were probed the
same way — a served CRD answers true, an absent one answers empty.

The new unit case pins the gate offline through the external-S3 path,
where the bucket name resolves without the offline override so only the
CRD gate can hold the CR back; removing the gate turns exactly that case
red.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-28 11:29:59 +05:00
Myasnikov Daniil 76d397523b docs(tests): record why an INT/TERM trap cannot cover a killed test
The comment added with the trace switch pointed at a signal trap as the
fix for the one case quiet mode does not cover: a test killed rather
than failed never reaches the fail handler, so its trace is gone rather
than already printed. That advice is wrong, and wrong in a way that
costs whoever follows it.

POSIX sh defers a trap until the current foreground command returns, so
installing an INT/TERM handler stops the runner dying on the signal at
all. Measured: a suite hung on `sleep 30` exits in 30s with the trap
installed against 2s under the default disposition. GitHub's
cancellation escalates INT, TERM, then KILL, so the deferred handler
gets SIGKILLed before it runs -- no dump, and the grace period spent
waiting for a hang. Built it, measured it, reverted it.

Making it work means running the test pipeline in the background and
waiting on it, because `wait` is the interruptible one. That is surgery
on the path every suite in this repo takes, unit and e2e, to recover a
partial trace in a rare case that already has a 48-second escape hatch.
Not taken; recorded so the next reader does not spend the afternoon
finding out the same way.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 22:30:18 +05:00
Myasnikov Daniil a5c90d984e fix(tests): guard the finishedAt leaf and make three guards actually bite
Findings from two independent adversarial reviews of the preceding two
commits. Each was reproduced locally before being fixed.

The ordering inverted itself on a shape I had not probed. `finishedAt`
is a metav1.Time behind `json:",omitempty"`, which cannot omit a
struct, and its zero value marshals to `null`; the kubelet does
synthesize a terminated state with no time (ContainerStatusUnknown on a
pod deleted mid-run, which also bumps restartCount and so clears the
restart filter). Two guards deep left the leaf exposed, and kubectl
renders it as the literal `<no value>` -- which sorts ABOVE every digit,
so that row took the front of the queue from the most recent real
restart, and the header printed `previous instance ended <no value>`, a
claim about a time nothing observed. Verified with a text/template probe
over seven lastState shapes: `null` and a missing key both produced it.

Fixed at both layers, because they fail independently. The template now
guards the leaf, so the string cannot be produced. And the sort
normalises any field-6 value not shaped like a timestamp to empty, so
the ordering stays correct and the header stays quiet even if some other
producer emits one. Earlier verification against a live cluster showed
no `<no value>` and I read that as the guard being right; it only meant
no container on that cluster was in that state.

Three guards did not bite:

  - both stability fixtures were pre-sorted ascending, which is exactly
    what sort's last-resort whole-line compare reproduces with no -s at
    all, so the tests written for -s passed without it. Reversed; they
    now fail when -s is dropped.
  - no test fed a six-field row through the whole script, so the
    timestamp reached the dump header uncovered -- deleting the read
    variable left the suite green at 39/39. One integration test now
    asserts the header names the time.
  - the e2e guard matched any mention of COZYTEST_TRACE, so documenting
    the design over there would have turned it red. It now matches an
    assignment, and tolerating a comment is verified too.

Also: a failed sort no longer discards a whole namespace group. The
caller compares row counts to report the cap, so losing a group silently
became "0 of N captured, N more not captured" and sent the reader after
a cap that never fired. It falls back to discovery order and says so.
LC_ALL=C on the sort, since the key is fixed-format ASCII.

And the claim itself was too strong. "Nothing diagnostic is lost" holds
only for a test that RETURNS: the fail handler runs after the pipeline,
and there is no INT/TERM trap, so a test killed rather than failed -- a
hang reaped by the job timeout -- never dumps its log, and under quiet
mode its trace is gone rather than already printed. The comments in
hack/cozytest.sh and the Makefile now say so and name the escape hatch.
A trap that flushed the tail of the log would close it; left out
deliberately rather than bundled in here.

Every fix is mutation-proven: reverting the leaf guard, removing the
key normalisation, dropping the read variable, dropping -s, and
silencing an e2e recipe are each caught. Unit lane 1513 passed, 1 failed
(hack/ghcr-mirror_test.bats, which reads a live cluster and fails the
same way under main's runner).

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 22:06:53 +05:00
Myasnikov Daniil 599ace69fd test(tests): guard the COZYTEST_TRACE switch against silent regression
The quiet unit lane rests on one invariant: run_one tees the raw stream
to $log and the fail handler prints all of it for whichever test fails,
so suppressing the live stream drops the output of PASSING tests only.

That invariant fails silently. It is exercised only when a test fails,
so a green CI run proves nothing about it - a refactor of run_one's
reader loop that also broke the dump would ship, and would be found by
whoever next needed a red unit run diagnosed, with no trace and the run
already lost. Nothing in the tree covered it.

Seven tests, fixtures that fail on purpose, following the structure of
hack/cozytest-capture-gate.bats:

  - quiet mode streams no trace for a passing suite
  - the unset default stays verbose, which is what the e2e call sites in
    packages/core/testing/Makefile rely on
  - a failing test still dumps its whole trace under quiet mode, xtrace
    included, which is the load-bearing one
  - quiet mode still fails the run
  - the per-test start/end markers survive, so a hang still names the
    test that never returned
  - the unit recipe asks for the quiet runner
  - the e2e recipes do not silence theirs

Every one is mutation-proven: removing the fail-handler dump, flipping
the default, making the switch a no-op, dropping the variable from the
unit recipe, adding it to an e2e recipe, and swallowing the exit code
are each caught by exactly one test. The last of those is worth a note -
it disables the mechanism the outer runner uses to report failure, so
the run exits 0 while the failure is in its output. A probe keyed on the
exit code alone reads that as uncaught.

Two traps are recorded in the file's header rather than left to be
rediscovered. Fixtures are built with printf because the awk transform
harvests a heredoc's own @test line into the suite that contains it. And
the default is asserted under `env -u COZYTEST_TRACE`, because the
Makefile passes the variable as a command prefix, which exports it - a
nested runner started from a suite that `make unit-tests` is running
would otherwise inherit 0 and the default would never be under test.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 22:06:33 +05:00
Myasnikov Daniil 7741843178 test(e2e): order previous-instance log capture by restart recency
The container cap (COZY_PREVLOG_MAX=12) is spent in namespace order, and
"unrelated" has meant only "in another namespace" - time was never a
key. So a container that restarted once during cluster bring-up
competed on equal footing with one that crash-looped during the test,
and won whenever the pod list reached it first. Two measured e2e jobs
spent 1034 and 1545 lines on virt-api and cdi-apiserver, restarted once
at install time by `Unable to create certwatcher` hours before any app
suite ran; on a development cluster 178 containers carried a restart, their
timestamps spanning 24 days, so the cap binds hard enough for the order to
decide everything.

Ordering, not filtering, and deliberately: an install-time restart IS
the explanation when the install is what broke, and this capture has no
reference point for "when the test began" that would make a cutoff
honest. Namespace stays the primary key - a stale restart in the
namespace under test beats a fresh one three namespaces away. So this
fixes cap displacement, not the raw volume of any single run.

lastState.terminated.finishedAt joins the row, guarded so a missing
lastState cannot error the list call and fail the capture open (verified
against a live cluster: no <no value>, empty where the kubelet has
already dropped it). Reading it also fixes a latent bug the sixth field
would otherwise have caused - `read -r ... restarts` folds a trailing
field into the last variable, so the dump header would have printed
`restarts=1|2026-08-22T09:06:00Z`. It now names the time instead, which
is the first thing that decides whether the dump is worth reading.

Sorting is lexicographic over fixed-width RFC3339 UTC, so no date
parsing: reverse puts newest first and leaves undated rows last, and -s
keeps a node reboot's simultaneous restarts in input order so the
selection is comparable between runs. Rows with no timestamp pass
through untouched, which is what makes this a no-op for the existing
five-field fixtures.

Five tests added; both mutations are caught (a passthrough
sort_recent, and dropping -s - the latter by the pre-existing namespace
test). capture-previous-logs 39/39, cozyreport 184/184,
bats-no-exit-trap 22/22.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 22:06:33 +05:00
Andrei Kvapil c8e4d97df9 feat(ingress-nginx): bump to 1.11.5 and restrict the admission webhook (#3798)
## What this PR does

Bumps ingress-nginx to chart 4.11.5 / controller v1.11.5, and closes the
admission webhook to everything except the API server.

### Version bump

The shipped `v1.11.2` controller is a cozystack rebuild that **already
carries a backport** of the CVE-2025-1974 fix (v0.28.2, March 2025), so
this is not a fix for a live vulnerability. The problem is that it
reports itself as a plain `v1.11.2 build git-d3bb2b4`: the binary is
built from the upstream 1.11.2 tag with the patch applied on top, which
does not change the version ldflags, and the image labels are inherited
from the `registry.k8s.io` base layer. A patched build is therefore
indistinguishable from an unpatched one by anything except its digest,
and anything scanning by version string flags it. This bump removes that
ambiguity.

`packages/system/ingress-nginx` feeds three consumers — the platform
ingress, tenant Ingress applications (`extra/ingress`), and the ingress
addon of tenant Kubernetes clusters — so one bump covers all three. The
protobuf-exporter sidecar moves in lockstep, since its tag tracks the
upstream controller version.

`CHART_VERSION` is now pinned in the package Makefile. `helm pull`
carried no `--version`, so `make update` would have fetched whatever is
latest upstream (4.13.9 today) against a controller image that only
exists for selected versions.

### Admission webhook policy

The webhook is only ever called by the API server, but it was reachable
from every pod in the cluster, in any namespace — avoidable attack
surface independent of any specific CVE.

The new `CiliumNetworkPolicy` is deny-only and disables default-deny. A
Cilium rule carrying an ingress section otherwise puts the selected
endpoints behind a default-deny, which here would mean enumerating every
port the controller serves (80, 443, metrics, the exporter, tcp/udp
services) and breaking traffic on the first one missed. Subtracting one
port is the safer construction.

Two callers are deliberately not denied:

* **host-network sources**, which are not Cilium endpoints — this covers
a management cluster, where the API server runs on the host network;
* **the konnectivity agent**, which is not optional. On a Kamaji-hosted
tenant cluster the API server lives outside the cluster and reaches
in-cluster webhooks through the konnectivity tunnel (Kamaji defines only
the `cluster` egress selection), so the call arrives from the agent pod
in `kube-system`, not from the API server address. A `fromEntities:
[kube-apiserver]` allow would not match it, and every Ingress
create/update in every tenant cluster would start failing.

The policy is namespaced rather than clusterwide on purpose: the same
package is installed into several namespaces of one cluster, so a
cluster-scoped object with a fixed name would collide between releases.
It can be disabled with `admissionWebhookPolicy.enabled`.

### Verification

Tested on a live Kamaji tenant cluster (k8s 1.35.6, Cilium 1.19), with
the addon enabled for the test and disabled afterwards:

| | probe from a pod in `default` | Ingress create |
|---|---|---|
| without policy | `open` | works |
| with policy | `Connection timed out` | works |

Also confirmed on the management cluster that `kube-apiserver` runs as
static pods on the host network, so the host-network carve-out holds
there.

The package previously ran no unit tests at all; a `test` target is
added with 7 assertions covering the policy. It is scoped with
`--with-subchart=false` because the vendored chart carries upstream's
own suites, which assert upstream defaults and fail under our values.
The full `hack/helm-unit-tests.sh` sweep is green.

The controller and exporter images are rebuilt by
cozystack/ingress-nginx-with-protobuf-exporter#5, which vendors the
`11843.diff` patch instead of fetching it from a still-open upstream PR
at build time.

### Screenshots

Not applicable — no UI changes.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff, which touches `packages/system/ingress-nginx/` and one comment
line in `hack/promote-retag.sh`:

* **website** — no package added, renamed or removed under
`apps/`/`extra/`; no platform component added or removed; no change to
`packages/core/platform/values.yaml`, variants, bundles or release
assets. The `package Makefiles` trigger was considered because of
`CHART_VERSION` and the new `test` target: `CHART_VERSION` does not
change the documented contract of `make update`, and `test: helm
unittest` is an existing pattern in the repo
(`packages/system/cilium-networkpolicy/Makefile`), not new tooling.
* **terraform-provider-cozystack** — every trigger is about
`packages/apps/`, `values.schema.json`, version enums or
`ApplicationDefinition`; none apply to a system package.
* **ansible-cozystack** — no change to installer values, platform keys
the role sets, variants, waited-on objects or node prerequisites.
* **ccp** — `hack/package.mk` and `hack/common-envs.mk` untouched;
nothing under `hack/` moved or renamed, and the edit to
`hack/promote-retag.sh` is a comment with no behaviour change; layout
and namespaces unchanged.
* **external-apps-example**, **talm**, **cozyhr**, **cozy-proxy**,
**cozystack-telemetry-server**, **examples** — no trigger matches.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
feat(ingress-nginx): update to chart 4.11.5 with controller v1.11.5, and restrict the admission webhook so it is reachable only by the API server. The previously shipped v1.11.2 image already contained a backport of the CVE-2025-1974 fix; it reported itself as a stock v1.11.2 build, which this update makes unambiguous.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added optional network protection for the admission webhook with
restricted access rules.
* Added PrometheusRule support and improved ServiceMonitor
configuration.
  * Webhook service ports are now configurable.

* **Updates**
  * Upgraded ingress-nginx and controller images to version 4.11.5.
  * Improved default-backend availability and topology-spread settings.
* Added Pod Security Admission guidance and refreshed configuration
documentation.

* **Bug Fixes**
* Corrected webhook service rendering and default-backend disruption
budget behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-27 16:59:06 +02:00
myasnikovdaniil 3d12ece2af chore(ci): namespace the backport labels without breaking the trigger (#3844)
Takes over #3433 and finishes it. Closes #2585.

The rename itself is IvanHunters' work, replayed here on current main
with his commits intact. What was blocking it was my own review, and
this fixes that plus one thing that appeared on main afterwards.

## What was blocking it

The gate in `backport.yaml` matched `kind/backport` only after the
rename. The `aliases:` in `labels.yml` migrate labels already attached
to issues and pull requests, but they do not change what an external
applier does going forward, and the org-level dosubot still applies the
legacy names. So the bot would have quietly stopped firing for the share
of backports it labels.

Both spellings are now accepted, in the `if:` gate and in the JS step
that reads them.

## What appeared afterwards

#3569 landed while #3433 sat, and it added a concurrency guard that
appends `-label` to the group for any labeled event that is not a
backport label, so a backport label never cancels a run already in
flight. That guard enumerated the two legacy names, so once the labels
are namespaced a `kind/backport` event takes the `-label` branch and the
guard stops covering the case it exists for. Extended to the new names.

## The dual acceptance is transitional and says so

Accepting two spellings is a bridge, not a design. The plan is to switch
dosubot's pull request labelling off and leave the in-repo labeler as
the only applier, at which point the legacy arms can go. The comment in
the workflow names all three deletion sites so that cleanup is
mechanical: the `if:` gate, the JS step, and the concurrency group.

Landing this before or after dosubot is switched off both work, which is
the point.

## Checks

`actionlint` clean on the workflow. There is no bats coverage for this
gate on main; #3813 is what adds `hack/pr-labeler-contract.bats`, so the
two are worth landing in that order.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Workflow Improvements**
- Backport automation now supports standardized `kind/backport` and
`kind/backport-previous` labels while maintaining compatibility with
legacy labels during the transition.
- Stale pull request handling and backport detection now follow the
standardized labeling scheme.

- **Documentation**
- Updated contribution and release documentation with the new labels,
labeling requirements, and troubleshooting guidance.

- **Tests**
- Expanded validation to cover both standardized and legacy backport
labels.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-27 17:26:20 +05:00
Myasnikov Daniil cb60f743d3 test(ci): extend the backport label contract to both spellings
The concurrency-group key and prepare's `if:` gate now name four labels
rather than two, and release-freeze-contract.bats pinned the pair on
both sides — as a literal and as an exact count. Every namespaced name
appended to the workflow therefore reported as a widening, which is the
right default: the count exists because the literal is a substring check
and an appended exclusion satisfies it unchanged.

Here the appended names are not a widening. `kind/backport` and
`kind/backport-previous` are genuine backport requests, so they belong
on the same side of the concurrency split as the legacy pair, and the
gate has to admit them or the rename stops the bot firing. Both counts
go to four and both literals name all four operands, so a half-finished
rename still reports.

Marked TRANSITIONAL at each site, matching the comment in backport.yaml:
when dosubot's PR labelling is switched off and the legacy arms are
dropped, these go back to the pair in the same pass.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 13:08:16 +05:00
Myasnikov Daniil a0c297a08a docs(ci): mark the legacy backport label arms as transitional
The dual acceptance exists so the rename can land in any order relative
to disabling dosubot's PR labelling, not because two spellings are
wanted long term. Naming the deletion sites keeps that from becoming
permanent by default.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 13:08:16 +05:00
Myasnikov Daniil 624a424d29 fix(ci): keep the backport concurrency guard working for namespaced labels
The guard from #3569 appends -label to the concurrency group for any
labeled event that is not a backport label, so a backport label never
cancels a run already in flight. It enumerated the two legacy names
only, so once the labels are namespaced a kind/backport event takes
the -label branch and the guard stops covering the case it exists for.

Both spellings are listed for the same reason the trigger lists both:
the org-level dosubot still applies the legacy names.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 13:08:15 +05:00
IvanHunters 0fa6cf72a5 chore(ci): complete the backport label rename across all consumers
The initial rename missed several literal consumers of the old label
names beyond labels.yml and backport.yaml:

- stale.yaml exempt-pr-labels still listed backport,backport-previous.
  actions/stale matches label names exactly and does not resolve
  label-sync aliases, so after the rename a kind/backport PR would lose
  its stale exemption and get auto-closed after inactivity. This is the
  one behavioural regression; the rest are documentation drift.
- PULL_REQUEST_TEMPLATE.md told contributors to apply the backport label.
- release.md label-target table and the gh pr list --search commands
  referenced the old names (the searches would silently return nothing).
- contributing.md claimed a backport label is auto-applied on the
  [Backport] prefix; pr-labeler only adds area/release, so the line was
  already stale and is corrected while renaming.

Historical narrative in release.md and the backport scope-to-area doc
rows (a Conventional-Commit scope, not the label) are left as-is.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-08-27 13:08:15 +05:00
IvanHunters 7b71053a0b chore(ci): namespace the backport labels under kind/
The labels.yml header declares that names follow the Kubernetes scheme,
but backport and backport-previous were top-level, violating it. Rename
them to kind/backport and kind/backport-previous, grouped under the kind/
section, and carry the old names as aliases so EndBug/label-sync renames
in place without dropping history on already-tagged issues and PRs.

Update the label literals in backport.yaml in lockstep (the if filter and
the label-detection script). pr-labeler.yaml's 'backport' entry is a
Conventional-Commit scope-to-area mapping, not a label reference, so it is
left unchanged. The top-level release label is a functional trigger keyed
by name in several workflows; renaming it is a wider cross-workflow change
out of scope here, so it is left as-is intentionally.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-08-27 13:08:15 +05:00
Myasnikov Daniil fdb2e89fe7 test(unit): make the bats trace stream opt-in, quiet on the unit lane
hack/cozytest.sh runs every @test body under `set -x` and streams the
trace live. Across the ~40 hack/*.bats unit suites that is 233,444 of
238,794 lines - 98% of a 31 MB job log - and the single failing
assertion sits at line 238,573 of it. The web log viewer is unusable at
that size, and anyone (or anything) reading the job to find out what
broke pays for the whole file first.

Nothing in that stream is load-bearing on the unit lane. run_one
already tees the raw output to $log and the fail handler dumps all of
it for whichever test fails, so suppressing the live stream drops the
output of PASSING tests only.

Add COZYTEST_TRACE, default 1 so the e2e call sites in
packages/core/testing/Makefile keep the live stream - there it is the
only progress signal, and the only record at all when a step timeout or
a dead runner kills the job before the fail handler can run. The
bats-unit-tests target sets 0; running the runner by hand stays
verbose, which is the case where a human is actually watching.

Measured on 25 suites locally, quiet vs verbose: 29,528 -> 951 lines,
3.9 MB -> 94 KB, and 71s -> 48s wall, the last because the per-line
formatter forks date twice for every line it prints. Quiet output is
byte-for-byte the non-trace subset of verbose, and a hanging test still
shows as a ╭ with no matching ╰.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-08-27 12:19:17 +05:00
Andrei Kvapil 02c6942adc fix(kubeovn-webhook): restrict the webhook port to the API server (#3955)
## What this PR does

Closes the exposure behind GHSA-g883-q79m-8225 by constraining who can
reach the kube-ovn webhook's serving port. The `/mutate-pods` handler
returns namespace-derived annotations to any caller that reaches it, and
until now nothing stopped an arbitrary in-cluster pod from being that
caller.

### Why the client-certificate path does not close it

cozystack/cozystack#2698 added `--client-ca-file`, deliberately off by
default, as observation mode. Turning it on requires the API server to
**present** a certificate, which it does only when its
`AdmissionConfiguration` names a `kubeConfigFile` for the webhook
plugins.

Checked against a running Cozystack control plane rather than assumed:
the `AdmissionConfiguration` there carries a `PodSecurity` plugin block
and nothing else, so no client certificate is offered to admission
webhooks at all. Flipping to `RequireAndVerifyClientCert` on such a
cluster would reject the API server itself, and with `failurePolicy:
Fail` that stops pod creation cluster-wide.

This matters for how the remaining work was scoped: the plan had been to
observe issuer logs for a day and then pin a CA. No amount of
observation changes the outcome, because the absence is a property of
the control plane's configuration, not of the traffic. Making
`--client-ca-file` usable is a separate, larger change to how the
platform configures the API server, and it is not a prerequisite for
closing the exposure.

### What the policy does

Denies pod traffic and world traffic to the serving port, and needs no
carve-out for the legitimate caller: the API server runs on the host
network, so it is not a Cilium endpoint and `fromEndpoints` never
matches it. Unlike the ingress-nginx admission webhook this component
exists only on the management cluster, so there is no
konnectivity-tunnelled caller to exempt either — and consequently no
forgeable pod label anywhere in the selector.

It is a `CiliumNetworkPolicy` rather than a core `NetworkPolicy`, and
that distinction is the reason it works at all:
`packages/system/kubeovn/values.yaml` sets `ENABLE_NP: false`, so
kube-ovn programs no ACLs and a core NetworkPolicy would be created and
never enforced. The values comment that claimed NetworkPolicy was simply
not an option is corrected — it was true only of the core kind.
`webhookPolicy.enabled` lets a cluster whose networking variant ships no
Cilium turn the object off.

Deny-only with default-deny explicitly disabled, matching the
ingress-nginx admission policy: putting these pods behind a default-deny
would mean enumerating every port they serve and breaking traffic on the
first one forgotten.

### Verification

Eight helm-unittest cases cover the port, the all-namespaces span, the
absence of any label-based exemption, the world deny, the disabled
default-deny, the Cilium kind, the selector and the off switch. The
rendered object was additionally accepted by a live API server via
`kubectl apply --dry-run=server`, so the CRD validates it as written.

Not verified: that a denied pod actually fails to reach the port at
runtime. That needs a live negative test against the policy, which is
the same gap the ingress-nginx admission policy carries.

### Screenshots

Not applicable — no UI change.

### Downstream repositories

Walked the trigger map against the diff. Nothing is reached: the change
is one chart template, its values and its tests under
`packages/system/`, with no package added or removed under
`packages/apps/` or `packages/extra/`, no
`packages/core/platform/values.yaml` key, no platform variant or bundle
change, nothing under `hack/`, and no node prerequisite.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(kubeovn-webhook): the kube-ovn webhook's serving port is now closed to in-cluster pods and external traffic by a CiliumNetworkPolicy. It previously answered any in-cluster pod with namespace-derived annotations (GHSA-g883-q79m-8225). Disable with webhookPolicy.enabled=false on clusters whose networking variant ships no Cilium.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a Cilium network policy to protect the namespace-annotation
webhook.
* The policy is enabled by default and restricts access to the webhook
port, blocking unauthorized in-cluster and external traffic.
* Standardized the webhook port across its deployment, service, and
network policy configuration.

* **Bug Fixes**
* Improved webhook network isolation when standard Kubernetes
NetworkPolicy enforcement is unavailable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-26 23:04:24 +02:00
Andrei Kvapil a7cef35d88 fix(kubeovn-webhook): address review on the webhook network policy
Derive the serving port from a single helper so the container port, the
Service targetPort and the policy cannot drift apart. With default-deny
disabled a disagreement failed open: the policy would deny a port nothing
serves while the real one stayed reachable, removing the control without
breaking anything visible. The knob that allowed the drift is gone and a
test asserts the three agree.

Move the rationale into template comments. Helm strips only {{/* */}}, so
the block was being copied verbatim into every rendered manifest. Record
in it what cilium monitor showed on a live kube-ovn cluster: the API
server reaches the webhook from the node InternalIP with identity host or
kube-apiserver, never from the join subnet and never as world.

Drop the egress default-deny toggle, which does nothing without egress
rules, and note that the policy removes in-cluster pods and external
traffic rather than every caller.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-08-26 19:01:24 +02:00
Andrei Kvapil eef95f456d docs: add CNCF General Technical Review (GTR) questionnaire (#3572)
## What this PR does

Adds the CNCF General Technical Review (GTR) questionnaire for
Cozystack, completed for the Sandbox → Incubation Due Diligence (tracks
cncf/toc#1916). It answers the Day 0 / Day 1 / Day 2 questions from the
CNCF TOC project-reviews template, with versioned links to the v1.6
documentation and to the repo's governance and security files.

**Status: DRAFT / work in progress.** The questionnaire itself is a
living document per the template's guidance. Everything repository
evidence settles has been answered and verified against the tree; the
banner at the top of the file names the seven substantive gaps that need
measurement, publication or a maintainer decision — load/scale results
and tested maximums, published SLOs/SLIs, measured platform baseline
overhead, the upgrade→downgrade CI matrix, whether upgrades trigger
Talos node reboots, the security-team rotation and diversity policy, and
publication of the CVE-scanning pipeline's policy documents.
Version-specific links should be pinned to the submitted tag before
final submission.

## Changes beyond the new file — please read this part

Review feedback found two factual errors that originate **outside** this
questionnaire, so fixing them only in the GTR would have left a reviewer
clicking through to the contradiction. Three existing files are
therefore touched:

- **`SECURITY.md` — supported-versions table. This is a policy change
and needs a policy review, not a wording check.** The table was dated
"As of March 17, 2026" and named `v1.1.x` as the current stable line,
five minors behind `v1.6.1`. It is rewritten to be keyed to **observable
patch activity on a line** — supported means a patch release within the
last 6 months, limited support 6 to 18 months, end of life beyond that —
so an adopter can determine their own status from the Releases page and
the table cannot drift again. Two intermediate versions of this PR got
the policy wrong in opposite directions and both are reverted: one
narrowed support to the latest two minors, which contradicts practice
since several `1.x` lines receive patch releases concurrently; the other
removed the end-of-life category altogether and extended discretionary
backports across all of `0.x`. What lands here restores an end-of-life
category, puts the `v0.41` cutoff back with its last-patch date, and
states explicitly that the six- and eighteen-month marks describe
current behaviour rather than committing the project to patch any line
for that long. `@kvaps`, as a CODEOWNER for this file: please review the
**policy**. If the project would rather state a guaranteed backport
window, or keep support entirely at maintainer discretion with no stated
marks, that is a governance decision and this PR deliberately does not
make it.
- **`SECURITY.md` — automated security analysis.** It said "two scanners
run continuously" and listed CodeQL and Scorecard. It now also describes
`zizmor` (GitHub Actions SAST, gating on PRs), the organization-wide
Trivy CVE-scanning pipeline, and Renovate's actual coverage. It also
records that CodeQL is **not** currently among the required status
checks on `main` — those are `pre-commit` and `E2E Tests` — so the
CodeQL gate is review convention rather than branch protection today.
- **`CONTRIBUTOR_LADDER.md`** — Reviewer nomination described adding a
username to an `OWNERS` file. No `OWNERS` file exists in this
repository; line 158 of the same document already refers to
`CODEOWNERS`. Corrected to name the files that exist. This does not
change the process.
- **`docs/security/self-assessment.md`** — brought in step with the
above on the same three points (CodeQL's gate status, the Trivy pipeline
and its limitations, Renovate's real coverage), so the GTR,
`SECURITY.md` and the self-assessment agree rather than contradict each
other in front of the TOC.

## Two things for `@kvaps` to decide, not to approve as wording

1. **Which release lines are supported — and the evidence points two
ways.** The six-month activity test in the rewritten table classifies
every `1.x` line from `1.0` to `1.6` as supported, because each has had
a patch release inside six months (`v1.1.7` as recently as 2026-06-03,
`v1.0.7` on 2026-03-30). But *recent* activity looks like three lines,
not seven: of the sixteen most recent backport branches, fifteen target
`release-1.4`, `release-1.5` or `release-1.6`, and only `1.4`, `1.5` and
`1.6` have shipped a release since 2026-07-03. Older backport branches
for `release-1.2` and `release-1.3` exist but are months old — and since
branches are never deleted, their existence proves nothing about the
present, which is the same trap the table's own lead-in now warns about.

So the concrete question is: **is the intended set `1.4`/`1.5`/`1.6`
supported with `1.0`–`1.3` on limited support, or is it genuinely all
seven?** If a critical fix would today be backported to `v1.1`, the wide
table is right. If it would not, the six-month window is the wrong
instrument and should be shortened. I have deliberately not chosen —
either answer is defensible and it is a maintainer call, not a
documentation one.

**A third file turns on the same answer.**
[`docs/security/incident-response.md`](https://github.com/cozystack/cozystack/blob/main/docs/security/incident-response.md)
step 5 says a fix ships "across all affected supported release lines",
pointing at the `SECURITY.md` table — so widening that table to seven
lines widens this commitment with it. This PR does not touch that file
and does not assert the claim, but the two should move together: if the
answer is "all seven", incident response is right and `SECURITY.md`
should stop hedging; if it is "1.4–1.6", the table, the window and that
sentence all change at once.
2. **Whether there is a backport rule at all.** No project document,
public or private, describes how a fix is chosen for backporting to a
given supported line. An intermediate version of this PR asserted one
(by severity, default-config reachability and adopter count); that was
invented and has been removed. The file now says the decision is taken
per fix and the rule is not documented. If a rule exists informally,
writing it down would answer a question the seven-supported-lines table
otherwise raises.

## Gaps this documents rather than fixes

Recorded here because they surfaced while verifying the questionnaire's
claims and each is a real finding:

- CodeQL is not a required status check, though two documents described
it as gating merges.
- There is no top-level `NOTICE` file, for a distribution bundling on
the order of fifty upstream components.
- No default Kubernetes audit policy is shipped; audit logging is opt-in
with the policy left to the adopter.
- Release artifacts are not signed and no issue tracks adding signing;
build provenance is explicitly disabled (`--provenance=false`); SBOM
generation is implemented but defaulted off.
- The CVE-scanning pipeline lives in a repository that returns 404 to
non-members. Its aggregate reporting is now published (below); its
policy documents and scanning scripts, which hold no finding data, are
not, so the method behind the numbers is still not independently
auditable.
- `nightly.yaml` has been failing since 2026-08-02 and `e2e-tag.yaml`
failed around the v1.6.1 release; current CI health is disclosed next to
the workflow links.

## New: published security reporting, and a backlog audit that changes
the SCA answer

`docs/security/reports/` now holds monthly vulnerability-handling
reports plus a [backlog
reconciliation](https://github.com/cozystack/cozystack/blob/main/docs/security/reports/reconciliation-2026-08.md).
The reconciliation exists because the CVE-scanning pipeline has no
write-back from this repository, so it has never marked a finding as
fixed and its counts were misleading in both directions. Audited against
what actually shipped:

- Remediation did happen — a clear majority of confirmed findings relate
to a component whose in-tree version now carries the fix — but as **one
wave in June 2026**, not a steady stream. The changes that did most of
the work name no CVE at all: re-vendoring the dashboard console removed
the vulnerable npm dependencies rather than upgrading them; refreshing
the Alpine bases cleared fourteen; bringing the Go toolchain current
satisfies all 62 `stdlib` findings.
- **57% of the backlog was never applicable or cannot be assessed.** The
largest block is packages containing no executable vulnerable code —
chiefly kernel headers, since a container never provides a running
kernel, which is the same reasoning the maintainers already applied 80
times to `kernel-headers` and never to `linux-libc-dev`. Reachable
exposure in a stock install is an estimated 2–5% of the tracker.
- **Four component bumps are available upstream and not taken**:
`nats-server`, the CoreDNS image, OpenBao, `fast-uri`.
- Seven defects in the pipeline itself, with a write-back specification
to fix the root cause. Two issues counted as findings are actually
quarterly-review tracking tasks.

The disclosure rule moved in the open direction: findings with no
upstream fix will be named with mitigation guidance, because under the
previous rule that class — a fifth of the backlog — stayed secret
permanently, and it is exactly where an adopter needs to know a
mitigation is the only control. Conversely, remediation is no longer
attributed to individual months, because crossing a count of still-open
findings with one month's publicly enumerable component bumps narrows
the search for unpatched exposure more than a bare count does.

Several findings warrant maintainer attention ahead of the backlog and
are being raised privately rather than named here.

### Downstream repositories

- [x] No downstream repository is affected by this change (docs-only; no
code, API, manifest or packaging changes).

### Release note

```release-note
docs: add CNCF General Technical Review (GTR) questionnaire for the Sandbox → Incubation review
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Documentation**
- Added a comprehensive technical review covering planning,
installation, operations, architecture, security, and troubleshooting.
- Added April–August 2026 security reports and a complete CVE backlog
reconciliation.
- Expanded security self-assessment, scanning coverage, reporting
guidance, disclosure rules, and audit limitations.
- Clarified supported-version policies, release backport decisions, and
private vulnerability-reporting procedures.

- **Process Updates**
- Updated reviewer nomination requirements to use current ownership and
maintainer records.
- Clarified severity- and risk-based backport decisions for coordinated
security releases.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-26 16:53:10 +02:00
scooby87 9dd2841c74 feat(clickhouse): add configurable version parameter (#3476)
## What this PR does

The ClickHouse server and Keeper images were hardcoded to `24.9.2.42` in
the chart templates, so the chart exposed no top-level `version` values
key. The Cozystack API returns a managed database's engine version by
passing the HelmRelease's `spec.values.version` straight through into
the Application object, so ClickHouse instances carried no
`spec.version` in the API — unlike postgres, mariadb, mongodb,
opensearch, rabbitmq and redis, which all ship a `version` parameter.
This closes the ClickHouse half of the gap tracked in #1246.

This adds the same version-selection scaffolding the other
managed-database charts already use:

- `files/versions.yaml` maps a `major.minor` key to the full image tag,
shared by the ClickHouse server and Keeper images (both are pinned to
one version).
- `templates/_versions.tpl` (`clickhouse.versionMap`) resolves the tag
and fails the render on an unsupported version.
- `values.yaml` gains a `version` enum (`v25.8`, `v25.3`, `v24.9`),
consumed by `templates/clickhouse.yaml` and `templates/chkeeper.yaml`.
- `hack/update-versions.sh` + `make update` refresh the map from Docker
Hub, keeping only tags published for *both* the server and Keeper images
and preserving the current default so a regeneration never bumps the
major of existing deployments.

The default stays `v24.9` → `24.9.2.42`, so existing installations
render byte-for-byte identically and are never upgraded to a new major
implicitly. The generated `values.schema.json`, `README.md`, the Go API
type and the `clickhouse-rd` `openAPISchema` are regenerated with
`cozyvalues-gen` v1.6.0. A new `tests/version_test.yaml` helm-unittest
suite covers the server/Keeper mapping for each version and the
schema-level rejection of an unsupported value.

### Downstream repositories

The diff changes `packages/apps/clickhouse/values.schema.json` by adding
a new `version` field with an `enum`. Per the trigger map, this reaches
`terraform-provider-cozystack`, whose ClickHouse resource is
hand-written and would need a matching `version` attribute with a
`stringvalidator.OneOf` list and its expand/flatten pair. There is no
existing PR or issue there for it yet. This is left for a maintainer
decision rather than a speculative cross-repo PR, so the box below is
intentionally left unticked and flagged here. No other downstream
repository is reached (the website reference page is regenerated from
`README.md` by the release docs bot; ClickHouse is already in its app
list).

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: needs a `version` attribute on the ClickHouse resource
(maintainer decision — see note above)
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
feat(clickhouse): add a configurable `version` parameter (v25.8, v25.3, v24.9; default v24.9) so the deployed ClickHouse version is selectable and returned through the API. Existing installations are unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a configurable ClickHouse version setting for both server and
Keeper images, defaulting to `v24.9`.
* Supported versions are `v25.8`, `v25.3`, and `v24.9`, with compatible
image patches selected automatically.
* **Documentation**
  * Documented the version setting, default, and supported values.
* **Validation**
* Unsupported versions are rejected with an error listing allowed
options.
* **Tests**
* Added coverage for version mapping, defaults, invalid values, and
update behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-26 17:45:53 +03:00
scooby87 94614c021a fix(etcd): spread members across nodes with default soft pod anti-affinity (#3962)
## What this PR does

The `etcd` chart hardcoded only a **soft** `topologySpreadConstraints`
(`whenUnsatisfiable: ScheduleAnyway`) and never rendered
`spec.affinity`, even though the `EtcdCluster` CRD supports it. On small
clusters the scheduler therefore co-locates multiple members on one
node; losing that node takes out quorum (or the whole cluster).

We hit this in production: a single node reboot left several 3-member
etcd clusters with 2–3 members on the downed node, and one tenant's
control-plane etcd lost quorum until the node came back.

This renders `EtcdCluster.spec.affinity` from a new `affinity` value:

- **Default (value left empty):** the chart applies a **soft**
(preferred) `podAntiAffinity` on `kubernetes.io/hostname`, so members
spread across nodes on a best-effort basis. Soft on purpose — a hard
rule would leave a member `Pending` when replicas exceed schedulable
nodes, or when node-local storage pins a member to a node, so it is safe
to default on.
- **Override:** set `affinity` to any rule — e.g. a hard
`requiredDuringSchedulingIgnoredDuringExecution` podAntiAffinity — to
forbid co-location outright on clusters with enough nodes and
non-node-pinned storage. A supplied value replaces the default cleanly.

The existing `topologySpreadConstraints` block (and its
`_cluster.scheduling.globalAppTopologySpreadConstraints` override) is
unchanged.

Generated artifacts (`values.schema.json`, `README.md`, and the embedded
`openAPISchema` in `etcd-rd`) were regenerated with `make generate`;
`make test` passes (18/18).

### Screenshots

n/a — no UI changes.

### Downstream repositories

Walked the trigger map against the diff: this only adds one values key
to an app chart. No package is added/renamed/removed, `kind`/`plural`
are unchanged, no installer values or release assets or variants change,
and `README.md` is regenerated here (the website bot regenerates
reference pages from it on release). No manual downstream follow-up is
required.

- [x] No downstream repository is affected by this change

### Release note

```release-note
fix(etcd): etcd clusters now spread their members across nodes by default via a soft podAntiAffinity, and expose an `affinity` value so operators can set a hard rule; previously a single node could host a quorum of members and take out the cluster when it failed.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added configurable pod affinity rules for etcd deployments.
* By default, etcd members use soft anti-affinity to spread across hosts
without blocking scheduling.
  * Custom affinity settings can override the default behavior.
* Exposed affinity configuration in the etcd application schema and
dashboard.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-26 14:27:15 +03:00
scooby87 6bb946fa5b fix(postgres-operator): bump CloudNativePG to 1.30.0 for the Cluster scale-subresource selector (#3951)
## What this PR does

Bumps the vendored `cloudnative-pg` chart from 0.27.1 to 0.29.0
(operator + CRDs from 1.28.1-patched-to-1.28.2 up to 1.30.0) and drops
the hand-maintained `patches/cloudnative-pg-1.28.2.patch` entirely —
0.29.0 is a published chart, so the version-alignment patch is no longer
needed.

**Why this is required.** Database read-replica autoscaling (community
proposal `design-proposals/database-horizontal-autoscaling`) drives the
CNPG `Cluster` scale subresource with a stock HPA (via KEDA). That only
works when the subresource serves `status.selector`, which
[cloudnative-pg#8996](https://github.com/cloudnative-pg/cloudnative-pg/pull/8996)
added and shipped in CNPG 1.28.4 / 1.29.2 / 1.30.0. cozystack shipped
1.28.2, whose scale subresource has no selector, so the HPA fails with
`InvalidSelector`.

**Why 1.30.0 and not 1.28.4.** The floor is ≥ 1.28.4, but the upstream
chart repo publishes no chart for 1.28.4 (the 1.28 line stops at chart
0.27.1 / appVersion 1.28.1), so 1.28.4 could only be reached by
hand-patching the 0.27.1 chart's operator image and CRDs up to the
backport. A live PoC on a dev cluster proved that path is broken: the
1.28.4 operator running under the 0.27.1 (1.28.1) chart templates
crash-loops on a missing RBAC grant it needs but the older chart does
not provide — `endpointslices.discovery.k8s.io is forbidden: ... cannot
list resource "endpointslices"` — so `status.selector` is never
populated and the HPA still cannot scale. 1.30.0 (chart 0.29.0) is the
only published chart that carries the selector together with matching
RBAC/CRDs/templates, and the 1.28 line is EOL (2026-06-30), so 1.30.0
also keeps us on a supported release. cozystack uses the barman-cloud
plugin (not native barman, removed in 1.31) and no CNPG Poolers, so the
1.30 deprecations and the known pooler bug do not apply.

**Validation on a dev cluster (CNPG 1.30.0).** Operator rolls out clean
(no crashloop, the `endpointslices` RBAC error is gone); all existing
`Cluster` objects return to Ready; `/scale` now returns a populated
`status.selector` (`cnpg.io/cluster=<name>,cnpg.io/podRole=instance`);
an HPA pointed at a `Cluster` no longer reports `InvalidSelector`
(`AbleToScale=True, SucceededGetScale`); psql connectivity and streaming
replication verified.

The version-guard unittest is updated to 1.30.0 and now also asserts the
clusters CRD scale subresource carries `labelSelectorPath:
.status.selector`.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file. The change touches only the vendored
`cloudnative-pg` operator chart (CRDs, RBAC, values), the package
`Makefile`, and the version-guard test. The `packages/apps/postgres` app
and its `version` enums are unchanged,
`packages/core/platform/values.yaml` is unchanged, and the
`clusters.postgresql.cnpg.io` CRD is not one the terraform provider
types by hand (it travels as opaque JSON). No downstream repository
restates anything this diff changes.

- [x] No downstream repository is affected by this change

### Release note

```release-note
fix(postgres-operator): bump CloudNativePG to 1.30.0 so the Cluster scale subresource serves status.selector, enabling HPA-driven read-replica autoscaling (previously the HPA failed with InvalidSelector on 1.28.2)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Upgraded the CloudNativePG operator and Helm chart to 1.30.0/0.29.0.
* Added expanded image catalogs, pooler settings, pod selectors,
scheduling options, monitoring TLS, and LZ4 backup compression.
* Improved permissions for database roles, endpoint slices, and lease
monitoring.

* **Bug Fixes**
* Strengthened configuration validation and removed the deprecated
workload reference option.
  * Updated compatibility requirements to Kubernetes 1.29 or newer.

* **Documentation**
* Expanded installation, configuration, verification, upgrade, and
uninstall guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-26 13:55:11 +03:00
Andrey KolkovandClaude Opus 4.8 d646c15f8d fix(clickhouse): scheme the backup S3_ENDPOINT on the system-bucket flow
On the useSystemBucket / cozy-default flow the chart pointed the
clickhouse-backup sidecar's S3_ENDPOINT straight at the projected
cozy-backups-creds.endpoint, which the backup credentials projector
stores as a bare host by design. altinity/clickhouse-backup runs
S3_ENDPOINT through the aws-sdk-go-v2 endpoint resolver, which rejects a
scheme-less value with "Custom endpoint 's3.host' was not a valid URI",
so every backup and restore on the default flow failed (reproduced on a
live v1.6.2 cluster). ClickHouse was the only cozy-default app hit by
this: the CNPG/etcd/velero strategies get a scheme'd URL from the
backupstrategy-controller.endpoint helper, and mariadb/foundationdb
deliberately pair a bare host with a TLS flag.

Prepend https:// on the useSystemBucket path via a helper env plus
$(VAR) expansion (a secretKeyRef value can't be string-interpolated),
mirroring the platform's endpoint helper which force-prepends https for
the always-TLS external S3 ingress. The legacy and bring-your-own-Secret
paths keep reading endpoint verbatim, since that value is tenant-supplied
and already carries its own scheme.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-26 13:26:31 +04:00
Andrey Kolkov 9858fb63ac test(etcd): cover default and overridden affinity rendering
The affinity value added in this branch introduced a two-branch conditional
in templates/etcd-cluster.yaml (default soft podAntiAffinity vs. a supplied
value) with no coverage. Add a helm unittest suite pinning both branches:
the default preferred per-host rule, and that a supplied affinity replaces
the default outright (no preferred rule leaks through).

Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
2026-08-26 13:26:26 +04:00
Timofei Larkin f5b957ee27 feat(kubernetes)!: manage worker pools as KubernetesNodes, adopt on upgrade (Phase 2, breaking) (#3315)
## What this PR does

Phase 2 of the Kubernetes-app split
([cozystack/community#8](https://github.com/cozystack/community/pull/8))
— the **breaking** half. Stacked on #3314 (review that first).

Removes worker node pools from the `kubernetes` chart (they moved to
`kubernetes-nodes` in #3314) and adopts existing pools on upgrade:

- `kubernetes` chart is now **control-plane only**: drops the pool
`range` block, the `talos-reconcile` Job + RBAC (deleted; moved to
`kubernetes-nodes`), the preserve-old-KMT block, the
`kubernetes.nodeGroups` helper, and the per-group dashboard RBAC.
- **`spec.nodeGroups` is removed from the `Kubernetes` CR** (BREAKING).
Worker pools are now separate `KubernetesNodes` resources.
- **Migration 54** (bumps `targetVersion` to 55): pre-upgrade hook that,
for every `kubernetes-<cluster>` release, creates one
`kubernetes-nodes-<cluster>-<pool>` HelmRelease and re-annotates the
existing worker objects onto it with `helm.sh/resource-policy=keep`, so
the now control-plane-only parent upgrade does not prune running
workers. The default `md0` (implicit when `nodeGroups` was empty) is
materialised explicitly. Idempotent. Genuine infrastructure/read
failures fail closed (`exit 1`, retried on the next upgrade); hostile
stored shapes (a pool name over Helm's 53-char release-name cap, an
RFC-1123-invalid or whitespace/slash map key, a non-object `nodeGroups`)
are warned, pinned, and skipped rather than deadlocking the entire
fleet's upgrade.
- The `ingressNginx` addon no longer auto-provisions a node (the
implicit `md0` default is gone). If enabled without a pool carrying
`roles: [ingress-nginx]`, the controller DaemonSet simply schedules
nothing (degraded, not failed) — the requirement is documented in the
chart NOTES.txt rather than enforced at render time.

Because the child chart renders the pool objects **byte-identically** to
the old parent render (golden parity in #3314; the
KubevirtMachineTemplate content-hash name is preserved), adoption does
not churn live worker VMs.

### Migration number

`54` is contiguous with main (latest is `53`) and `targetVersion` is
`55`. The `52`/#3201 collision noted in earlier revisions has been
resolved: this branch was rebased and merged past it, so the numbering
is contiguous and `run-migrations.sh` has no gap to hard-fail on.

### Not yet done (blocking draft → ready)

- [ ] **dev e2e upgrade validation** (`cozystack-pr-test`): upgrade a
custom-`nodeGroups` cluster and a default-`md0` cluster; assert zero
worker-VM churn (KMT hash, Machine uid/creationTimestamp stable) and
that the adopted child HRs reconcile.
- [ ] downstream trigger-map walk (website docs for the `nodeGroups` →
`KubernetesNodes` change; terraform-provider for the removed API field).

### Downstream repositories

Deliberately unticked (draft) — the breaking API change affects docs and
the terraform provider; follow-ups to be filed before ready.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:

```release-note
Worker node pools of a managed Kubernetes cluster are now managed as separate `KubernetesNodes` resources. `spec.nodeGroups` on the `Kubernetes` CR is removed; existing pools are adopted automatically on upgrade.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Worker pools are now separate `KubernetesNodes` resources with their
own lifecycle.
* Added a render-time compatibility guard to prevent worker pools from
running ahead of the parent control-plane minor version.
* Improved upgrade/uninstall behavior to better preserve already-adopted
worker objects.

* **Breaking Changes**
* Removed per-pool `nodeGroups` and worker node health settings from the
`Kubernetes` chart/CR; pool configuration now lives in
`KubernetesNodes`.
* Ingress requires at least one `KubernetesNodes` pool with `roles:
[ingress-nginx]`.
* `cluster` and `storageClass` are immutable after creation; pool
`version` may lag but must not be ahead of the parent minor.

* **Documentation**
* Updated READMEs, notes, and schemas to reflect the `KubernetesNodes`
split and the immutability/version rules.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-26 12:06:18 +04:00
Alexey Artamonov e707475f96 fix(clickhouse): fetch tags via skopeo and fail closed on a missing pin
Address review of the version generator (hack/update-versions.sh):

- Fetch tags with `skopeo list-tags` (registry v2 /tags/list) instead of
  the Docker Hub v2 API, whose anonymous pagination caps at 1000 entries
  and then returns an error page with no `.results`, silently truncating
  the list so the frozen 24.9.2.42 fell outside the window and the run
  failed. skopeo is already in build-deps and is how the sibling
  generators (postgres, redis, mongodb, opensearch, kubernetes) fetch.
- Make a frozen default tag missing from the registry a hard error
  instead of silently falling back to a newer patch, which would move
  the default image for existing installs.
- Restore 0644 on the regenerated files (mktemp's 0600 survived the mv).
- Exclude /hack from the chart tarball via .helmignore.
- Refresh files/versions.yaml: v25.8 -> 25.8.32.4, the true latest 25.8
  patch now visible with the complete tag list.
- bats: add a missing-frozen-tag hard-error case and make the atomicity
  test reach the resolution phase after a valid default rather than
  failing at the default-in-set check.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-08-26 00:15:22 +03:00
Alexey Artamonov c83a22e077 fix(etcd): spread members across nodes with default soft pod anti-affinity
The etcd chart only set a soft topologySpreadConstraints and never rendered
spec.affinity, so the scheduler could place several members of a 3-node
EtcdCluster on the same node; losing that node then took out quorum (observed
in production after a single node reboot).

Render EtcdCluster.spec.affinity from a new `affinity` value. When it is left
empty the chart applies a soft (preferred) podAntiAffinity on
kubernetes.io/hostname, so members are spread on a best-effort basis without
ever leaving a member Pending on small or node-pinned-storage clusters.
Operators can override the value — e.g. a hard requiredDuringScheduling rule —
where node count and storage allow.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-08-25 23:09:40 +03:00
Kirill Ilin 212d0e1944 feat(keycloak): let the KMS proxy trust a private Vault CA (#3874)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does

The KMS-encrypting DB proxy could only reach a Vault whose certificate
chains to a publicly trusted root. A Vault fronted by an internal load
balancer with a self-signed certificate was therefore unreachable over
HTTPS, leaving plain HTTP as the only option for the DEK wrap/unwrap
traffic.

`encryption.kms.vault` now accepts the CA either inline as `caBundle`
(the chart renders the Secret) or as an already-seeded `caSecretName`,
with `caSecretKey` naming the key to project.

The CA reaches the proxy as `SSL_CERT_FILE` **and** `SSL_CERT_DIR`.
`SSL_CERT_FILE` on its own would only *add* the CA to the roots the
image ships: `crypto/x509` replaces its file list but then scans
`SSL_CERT_DIR` unconditionally, and the pinned proxy image carries
`/etc/ssl/certs/ca-certificates.crt`. Setting both collapses the pool to
the mount, so the connection that guards the KEK trusts the configured
CA and nothing else.

The database leg is unaffected either way, because it never consults the
system trust store: it is plaintext today, and will verify against an
explicit `RootCAs` pool once `KKP_BACKEND_CA_FILE` is wired.

Supporting guarantees: a non-PEM `caBundle` fails the render instead of
being dropped silently at runtime; rotating an inline `caBundle` rolls
the pods through a `checksum/vault-ca` annotation, since Go memoizes its
root pool per process; both CA keys are inert unless
`kms.backend=vault-transit`, matching every other `vault.*` key; and
setting `caBundle` together with `caSecretName` fails the render instead
of silently picking one.

Existing installations are unaffected: with neither key set the proxy
keeps using the image's system trust store, and no volume or env var is
added.


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
feat(keycloak): the KMS-encrypting DB proxy can now trust a private CA for Vault, supplied inline as encryption.kms.vault.caBundle or as a pre-seeded Secret via encryption.kms.vault.caSecretName (encryption.kms.vault.caSecretKey names the key). The Vault connection is pinned to that CA alone, which makes a Vault behind an internal load balancer with a self-signed certificate reachable over HTTPS instead of plain HTTP.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added Vault TLS CA configuration for encrypted deployments.
* Supports inline certificate bundles or existing Secrets with a
configurable key.
* Configured secure Vault connections to use the selected CA
certificate.
* **Bug Fixes**
* Rejects invalid certificate bundles and prevents conflicting CA
sources.
  * Restricts custom CA settings to Vault Transit connections.
  * Preserves the system trust store when no custom CA is configured.
  * Triggers pod rollouts when inline CA bundles change.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-25 22:38:58 +05:00
Timofei Larkin abc31ef17f fix(etcd-operator): gate VMServiceScrape on VictoriaMetrics operator CRD
The etcd-operator HelmRelease rendered a VMServiceScrape unconditionally, so it failed to install with "no matches for kind VMServiceScrape" on any cluster without the VictoriaMetrics operator CRDs. Wrap it in the same Capabilities.APIVersions gate linstor's podscrape.yaml uses, leaving the metrics Service unconditional.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-08-25 19:03:23 +03:00
Alexey Artamonov e23fceffe7 docs(postgres-operator): distinguish chart version from appVersion in comment
Reword the values.yaml comment so it does not call appVersion 1.30.0 a
"published chart": the published chart version is 0.29.0 and it carries
appVersion 1.30.0. Keeping the two distinct stops a maintainer from running
`helm pull --version 1.30.0`.

Addresses CodeRabbit review feedback on #3951.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-08-25 18:52:30 +03:00
Andrei Kvapil 091d8ce143 fix(ingress-nginx): re-pin both images to the rebuilt v1.11.5
The controller digest this pinned was a build dated 2025-03-26, whose
nginx.tmpl calls buildHTTP3Listener — a function no v1.11.5 binary
defines. Go binds template functions at parse time, so the
{{ if $cfg.UseHTTP3 }} guard around it does not help and the controller
crash-loops on a default config, serving no webhook at all.

The tag pointed at that image because the controller had not been built
since March: the build in the fork failed on a luarocks step, and the
cause was outside the repository. luarocks resolves a rock name through
the luarocks.org index, and that index has grown past LuaJIT's hard
limit of 65536 constants per function, so no client can load it and
every lookup returns "No results matching query were found". Neither a
luarocks bump nor GNU wget fixes that; installing from a rockspec URL,
which skips the index, does. Fixed in
cozystack/ingress-nginx-with-protobuf-exporter#6 and the v1.11.5 tag
re-pointed at it.

Verified in the rebuilt image: zero references to buildHTTP3Listener,
and both lua-protobuf and lua-iconv present — the rocks this fork exists
to add.

The exporter is re-pinned in the same commit rather than separately. It
had drifted a build ahead of its tag before this rebuild, and refreshing
one half alone would leave the PR looking updated while the controller
was still the March build.

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-08-25 15:45:35 +02:00
IvanHunters bf38655ab5 docs(computeplane): nodeGroups allowlist drops unlisted fields, not pass-through
The nodeGroups docs still described the old denylist behaviour: "unlisted
NodeGroup fields pass through to the wrapped chart unvalidated". After the switch
to an allowlist (pick) both halves are wrong: fields go to the kubernetes-nodes
pool chart (not the wrapped kubernetes chart), and anything the pick does not name
is dropped, not passed through. A tenant that sets e.g. talos.version on a
nodeGroup gets no error and no effect (accepted-but-inert). Correct the source
@param and the hand-written README intro, regenerate schema/README/cozyrd, and
refresh the stale "omit" wording in the splice test comment to say allowlist.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-08-25 16:39:19 +03:00