Commit Graph
3858 Commits
Author SHA1 Message Date
Andrey Kolkov ed3cb34cbf fix(backups): adopt-guard Kafka Backup artifact and correct URI-template doc
- createKafkaBackupArtifact now rejects adopting a same-named Backup that
  describes a different application, so a retained Backup reused for a
  different app of the same name is not reported as this run's success.
  Mirrors the RabbitMQ driver's guard; pinned by a Go test.
- ArtifactURITemplate godoc said empty leaves status.artifact unset, but
  the driver hard-fails an empty template. Reword to state it is required,
  and regenerate the CRD description to match.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
2026-09-10 22:22:07 +04:00
Andrey Kolkov 0270650237 fix(backups): harden Kafka metadata S3 handling and own the artifact
- Drop `-k`: mirror the RabbitMQ S3 layer (ARTIFACT_URI as the single
  source of truth, forcePathStyle honoured, whole base URL kept) so
  curl --aws-sigv4 verifies TLS against the ACME endpoint the helper
  returns. The derived endpoint is default-port, so the curl 7.76.1
  --connect-to port workaround is gone too; this also fixes the IPv6
  and path-prefix mangling in the old host-splitting.
- Own the artifact: a cleanup mode in the script plus cleanupKafkaBackup
  wired into BackupReconciler.cleanupOnDelete delete a deleted Backup's
  object (best-effort, waited on) instead of orphaning it, matching the
  RabbitMQ precedent. Kafka has the identical own-your-object property.
- Restore probes existence with --list (fail closed), so an unreachable
  broker aborts instead of being misread as "topic absent" and created.
- Hold the fixes with tests at the right level: a colliding topic pair
  (audit.events / audit-events, distinct partition counts) in the e2e so
  a revert of the \Q...\E literal-match breaks the round-trip, and Go
  tests pinning the scope stamp and the cleanup Job.
- Correct the docs/comments: endpoint-shape row, the "cached on nodes"
  claim, and the narrower curl 7.76.1 S3 envelope (empty-body payload
  hash) that makes a small modern-curl client image the right follow-up.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
2026-09-10 22:22:06 +04:00
Andrey Kolkov 537aadf389 fix(backups): fix Kafka backup S3 signing and literal topic names
Reproduced end-to-end against a real Kafka 3.9.1 broker:

- [CRITICAL] The Strimzi image ships curl 7.76.1, whose --aws-sigv4
  omits a non-default port from the SigV4 canonical Host while sending it
  in the Host header, so every S3 PUT/GET 403s and the driver never
  uploads or downloads. Sign/send the bare host and redirect the
  connection to the real port with --connect-to (the same fix the
  topic-data example already carries), and accept the in-cluster
  self-signed cert with -k.
- [CRITICAL] kafka-topics --topic is a Java regex, not a literal: a
  backup entry for "audit.events" also matched a live "audit-events", so
  restore repartitioned an unrelated topic irreversibly and backup
  recorded the wrong partition/replication values. Wrap the name in
  \Q...\E on every filter call (--describe on both paths, --alter on
  restore) so it matches exactly one topic; --create and
  kafka-configs --entity-name are already literal. Applied in the example
  helpers too.
- [MAJOR] cozy-default's Kafka route looked like a data backup at
  runtime. Stamp scope=topic-metadata on the produced Backup's
  DriverMetadata and say "topic definitions and configs only; no message
  data" in the Ready condition, so kubectl describe backup is honest.
- [MAJOR] Add reconcile tests for the two Go behaviours a mutation could
  silently drop: the Strimzi-Ready precondition on backup (BackupJob
  stays out of Succeeded and creates no Job) and the applicationRef Kind
  gate on the restore target.
- [MINOR] Fail closed on the config-capture grep (exit != 1 aborts) and
  clarify the image pin has no automated producer.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
2026-09-10 22:21:25 +04:00
Andrey Kolkov 405eb25095 fix(backups): harden Kafka metadata backup
- Fail closed on topic listing: set -o pipefail, capture kafka-topics
  --list --exclude-internal exit separately, and abort on failure so an
  unreachable broker can no longer produce an empty backup reported as
  Succeeded. An empty list after a successful call stays legitimate.
- Check each kafka-topics --describe exit and validate the parsed
  partition count / replication factor before writing a record, so a
  timed-out describe cannot write a corrupt orders\t\t\t line.
- Store topic configs one per line with key and value in separate
  tab-delimited fields, sourced from kafka-configs --describe (one config
  per line), so a value containing a comma (cleanup.policy=compact,delete)
  round-trips; on restore each config is applied with the value bracketed
  only when it contains a comma.
- Detect topic divergence on restore instead of a silent --if-not-exists
  no-op: grow partitions with --alter when the backup asks for more, and
  fail loudly (naming the topic and both values) when it asks for fewer
  partitions or a different replication factor. A restore that cannot
  reach the recorded state no longer reports Succeeded.
- Add KafkaStrategyKind to the RestoreJob cleanup-dispatch no-op group so
  deleting a Kafka RestoreJob does not run a pointless Velero cleanup; add
  a regression test alongside the siblings, plus a reconcile Kind-gate
  test and a rendering test that actually renders the strategy template.
- Wire the kafka-metadata Chainsaw suite into both selectors
  (select-install suite_to_source, select-e2e src_to_suites) so editing
  the kafka app or the suite selects it and the bats unit tests pass.
- Pin the client image to the operator's default broker version
  (kafka 3.9.1) with a digest, and align the example default; fix the
  values comment. Point the RabbitMQ comment references (which shipped in
  the CRD description) at the Job strategy actually mirrored, and trim the
  in-pod comments that rendered into the manifest.
- Document the Strimzi Topic Operator caveat: for a topic owned by a
  KafkaTopic CR the operator reconciles its config from the CR, so an
  in-place config restore is durable only for out-of-band topics.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
2026-09-10 22:21:24 +04:00
Andrey Kolkov a410a3f48b feat(backups): Kafka topic-metadata backup via a dedicated strategy driver
Add a strategy.backups.cozystack.io/Kafka driver that backs up the topic
metadata of a Cozystack-managed Kafka application — partition counts,
replication factors and non-default topic configs — through the Kafka
Admin API, and restores it with kafka-topics --create + kafka-configs
--alter. Modelled on the Job/Rabbitmq strategies: the driver renders a
one-shot batch Job, gates on the Strimzi Kafka cluster reporting Ready,
and keys each backup by .BackupName so runs do not overwrite each other.

Scope is deliberately narrow: message payloads, consumer-group offsets,
ACLs, quotas and KafkaUsers are out of scope, and there is no PITR.
Restored topics are created directly (unmanaged on the target) — the
accepted tradeoff of an Admin-API-only driver. Both restore flows are
implemented: in-place recreates dropped topics into the source, to-copy
applies the source's topics onto a freshly-bootstrapped empty cluster.

Wire it into cozy-default (strategy-kafka-default.yaml gated on the
bucket name, a BackupClass binding, read RBAC on kafka.strimzi.io/kafkas
for the readiness gate, and the Strimzi Kafka client image), document it
in backup-classes.md, and add an honest round-trip example plus Chainsaw
e2e (examples/backups/kafka-metadata, kafka-3-metadata-roundtrip) that
proves a per-run retention.ms sentinel survives both restores.

Unit tests cover the driver's pure helpers (application-ref validation,
readiness verdict, parameter round-trip, per-run artifact-URI keying).

Not yet verified on a live cluster.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
2026-09-10 22:20:27 +04:00
scooby87 f7cb9e7854 fix(postgres): follow-up review remarks from #3954 (KEDA autoscaling) (#4188)
Follow-up to #3954 (postgres read-replica autoscaling via KEDA), closing
the non-blocking review remarks tracked in #4187. Each change is small,
isolated, and covered by a red→green test where it is a behaviour
change.

## Changes

- **fix(postgres): pin `job="kube-state-metrics"` on the autoscaler
read-load query.** The trigger query joined on `kube_pod_labels` without
the job pin the sibling `DatabaseAutoscaler*` alerts already carry; a
duplicate `kube_pod_labels` series from another exporter would multiply
the join. Pinned on every selector; unittest asserts it in both the base
and lag-brake query shapes.
- **fix(cozy-lib): reject a negative `minReplicaCount` in
`cozy-lib.keda.scaledObject`.** Only `max >= min` was checked, so a
mixed-sign pair (`min -5, max -3`) passed and rendered a nonsensical
floor. Now failed closed before the ordering check, with a unit case.
Also added coverage for the previously-untested `annotations` parameter.
- **docs(postgres): correct the KEDA-missing guard rationale and note
the gradual client-side rollout.** The guard comment still described the
replaced drop-instances design; corrected to the seed design (a skipped
ScaledObject leaves a fixed-size cluster with no HPA, not a collapse to
1). The cozyrds apply-strategy annotation now notes that the client-side
switch rolls out per-app as each Application is next written through the
aggregated API, not fleet-wide at upgrade.

## Testing

`helm unittest` green: postgres 66/66, cozy-lib-tests 45/45, keda 12/12.
Both behaviour changes are mutation-verified (reverting the fix reddens
the suite). No generated artifacts affected (no
`values.yaml`/`values.schema.json`/`Chart.yaml`/`README.md` changes).

## Release note

```release-note
NONE
```

Closes #4187


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved PostgreSQL autoscaling metrics by ensuring pod data is
sourced from the correct metrics exporter, preventing duplicate data
from distorting scaling decisions.
  - Added validation to reject negative minimum replica counts.
- Preserved caller-provided annotations on generated autoscaling
resources.

- **Documentation**
- Clarified PostgreSQL autoscaling fallback behavior when the
autoscaling controller is unavailable.
- Documented the gradual transition of existing PostgreSQL resources to
the updated apply behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-10 19:32:24 +03:00
Aleksei Sviridkin 7c739a14fa fix(cdi): omit the CDI clone strategy override when unset (#4138)
## What this PR does

`cdi-cr.yaml` rendered `cloneStrategyOverride` unconditionally, so there
was no way to let CDI pick a clone strategy per StorageProfile. The CRD
enum is `copy`/`snapshot`/`csi-clone` with no empty member, so neither
obvious workaround got there: an empty value rendered
`cloneStrategyOverride: ""`, which that enum does not accept, and a null
one rendered a bare key. Both checked against `main` with `helm
template` rather than assumed.

Now the key is guarded on a non-empty value, same as
`uploadProxyURLOverride` just below it in the same file. With the field
absent, CDI reads the target StorageClass's StorageProfile and uses its
clone strategy.

Default stays `csi-clone`. Rendered at the default and diffed against
`main`, the output is byte-identical, so nothing already deployed moves.

### Tests

`tests/clone_strategy_test.yaml` pins both directions: absent when the
value is empty, absent when it is null, `csi-clone` on the chart
default, `snapshot` through unchanged. It sits alongside the existing
`importer-resources` and `standalone_render_test` suites, with its own
file and suite name and no shared test names; `make test` runs all
three.

I checked the suite can fail rather than assuming it works. Reverting
the guard reddens the two absent-field cases, deleting the field
outright reddens the default and passthrough cases, hardcoding the value
reddens the passthrough case, and rendering a bare key reddens all four.
The two sibling suites stay green through every one of those, so the
reds are attributable to this suite.

The null case is pinned as an input rather than argued from `with`
treating null and empty alike. It cannot pass vacuously: flipping its
assertion to expect the default returns `unknown path`, not a
`csi-clone` mismatch, which is what shows the null actually reaches the
template instead of the assertion passing on a value that never arrived.

### Known gaps

`copy` is accepted by the enum but no test drives it, so the suite does
not pin that it reaches the CR. Concretely: wrapping the guard in `{{-
if ne . "copy" }}`, which silently drops that one strategy, still passes
all nine tests. Named here rather than fixed so the boundary is visible
to the next reader instead of being rediscovered.

### Scope

Context: https://github.com/cozystack/cozystack/issues/2908. This is the
first of the three directions that issue lists, so it does not close it
and carries no `Closes`. The default is deliberately left at
`csi-clone`, since whether that is the right default is the second
direction and a separate decision. The third, `vm-disk` warning on a
cross-StorageClass request, is untouched. An operator who has not set
`cloneStrategyOverride: ""` sees the reported symptom exactly as before.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff. It is one guarded key in a
`packages/system` chart, a values comment, and a test suite: no package
added or renamed, no `values.schema.json` (this chart has none), no
`ApplicationDefinition`, no namespace, nothing under `hack/`, no
platform values key. The `ccp` entry covers `hack/package.mk` and
make-target behaviour, and this touches neither. Nothing in the map
reaches it.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(cdi): `cloneStrategyOverride` in the `kubevirt-cdi` package can now be left empty, which omits the field and lets CDI pick a clone strategy per StorageProfile. The default is unchanged at `csi-clone`.
```
2026-09-10 19:22:21 +04:00
Aleksei Sviridkin 4b6b1dc2da fix(vm-instance): let complete resources replace the instance type (#4109)
## What this PR does

Closes #2909.

The matcher keyed off `instanceType`, the domain sizing keyed off
`resources`, and neither side looked at the other, so the chart could
render both. KubeVirt refuses that, and the two halves of its rule are
different shapes: `validateCPU` conflicts on `domain.cpu.cores` and
`domain.cpu.sockets` only when they are non-zero, while `validateMemory`
conflicts on `domain.resources.requests.memory` whenever the key is
present. `values.yaml` ships `instanceType: "u1.medium"`, so clearing
the field in the dashboard just falls back to that default. Fill in cpu,
sockets and memory after clearing it and you get a VM the webhook
rejects, naming fields the form gives you no way to clear.

Resources that size the VM completely now suppress the matcher and size
it directly. `packages/apps/kubernetes-nodes` already does this for its
worker VMs. Beside an instance type, sizing that is only partial stops
the render, naming what sized the VM and what did not; with no instance
type in play there is nothing to conflict with, so that half renders as
it does today. Letting resources win unconditionally would silently drop
the instance type for a VM that sets memory alone, which is a different
bug in the same place.

Completeness is decided from what `domainResources` emitted, never from
which keys the user set, and all three fields go through that one
reading. The two readings come apart in more than one place.
`resources.cpu` on its own emits nothing, so it conflicts with nothing
and goes on rendering a plain instancetype-sized VM. cpu and sockets
pass through `int64`, which yields 0 for a quantity the schema accepts,
so `cpu: 500m` beside a whole sockets and a memory would read as
complete from the values, drop the matcher, and hand KubeVirt a `cores:
0` the user never asked for. And a `resources` block nulled outright
leaves `.Values.resources` nil rather than the chart's `{}`, so a raw
field read is a render-time nil pointer where the emitted object is
merely empty.

The existence check and the pre-upgrade update hook read the same
effective value. Keying the `lookup` on the raw one would fail a
resource-sized render over an instance type it never references, which
is the same confusing error this fixes. Reading `.Values.instanceType`
in the hook would see the leftover value, skip the removal patch, and
leave `spec.instancetype.revisionName` on a VM Helm had just given
explicit domain resources.

### What changes in the rendered output

| `instanceType` | `resources` | Before | After |
| --- | --- | --- | --- |
| set | unset | matcher only | unchanged |
| unset | cpu, sockets and memory | domain cpu and resources | unchanged
|
| set | cpu, sockets and memory | matcher plus domain sizing, refused by
KubeVirt | domain sizing, no matcher |
| set | any subset with memory, or cpu with sockets | matcher plus a
conflicting domain field, refused by KubeVirt | render stops, message
names both sides |
| set | cpu alone, or sockets alone | matcher only | unchanged |
| set | cpu and sockets that both truncate to zero | matcher plus
`cores: 0` | unchanged |
| unset | anything short of all three | already refused by the existing
check | unchanged |

The rows that stay unchanged are deliberate.
`virtual-machine.domainResources` writes `domain.cpu` only when cpu and
sockets arrive together, so cpu alone emits no override at all. A
`cores: 0` beside a matcher is not a conflict either, because
`validateCPU` guards on non-zero, so KubeVirt admits it and the instance
type sizes the VM. Both shapes render today and go on rendering.
`cpuModel` is left alone for a related reason: it writes
`domain.cpu.model`, which KubeVirt flags only when the instance type
declares a model of its own, and no instance type shipped in
`packages/system/kubevirt-instancetypes` declares one.

### Why no working release breaks

Every shape that now fails at render was already being rejected by the
KubeVirt validating webhook at admission, so it could not have produced
a running VM in the first place. The failure moves from an opaque
admission error to a render error that names the fields. Nothing that
worked stops working.

The unchanged rows above are measured. Each non-failing shape in the
table, plus firmware on and off, an explicitly nulled `resources` block
and the three update-hook branches, was rendered against a mocked
cluster on both sides and compared document by document. Across 14 value
shapes and 78 rendered documents, 77 come out byte-identical. The one
that differs is the row the table marks as changed, which is also what
the harness was checked against first, so it is known to be able to
fail.

### Live-VM impact

This changes a rendered VM spec, so an upgrade can resize or restart a
running VM. Adding a complete `resources` block to a VM currently sized
by an instance type drops its matcher, and the pre-upgrade hook patches
`spec.instancetype` off the live object and writes the domain resources
in its place. Cozystack runs KubeVirt with `vmRolloutStrategy:
LiveUpdate` (`packages/system/kubevirt/templates/kubevirt-cr.yaml:125`),
so KubeVirt applies what it can to the running VMI and marks the rest
`RestartRequired`; I have not measured which half a given resize lands
in, so plan the upgrade as if it restarts.

Dropping the matcher drops the whole instance type, sizing and
everything else it carries. Of the 69 shipped cluster instancetypes, 36
declare `dedicatedCPUPlacement`, 36 declare `hugepages` and 26 declare
both. 20 declare NUMA passthrough, 8 declare `realtime`. A VM moved from
`rt1.medium` to explicit resources loses those characteristics along
with the type. Before this change that combination was rejected
outright, so the VM never existed; now it is admitted, and the field
help says so.

### Tests

`tests/instancetype_resources_test.yaml` covers matcher only, resources
only, both complete, each failing partial shape, the shapes that emit
nothing, both cpu and sockets truncating to zero, `cpuModel` beside a
matcher and beside complete resources, a stale instance type with and
without complete resources, an explicit `instanceType: null`, an
explicitly nulled `resources` block, and every branch of the update
hook's instancetype selector, in both directions plus the two no-op
cases. 24 cases in this package, 2127 across `make helm-unit-tests`, all
green. Every case pinning new behaviour was checked against a reverted
template to confirm it goes red; the rest pin behaviour this change
deliberately leaves alone.

A ten-mutation ladder over the helper, `vm.yaml` and the update hook
leaves no survivor. The memory branch took the most work. Swapping its
emitted read for a raw `.Values.resources.memory` looks like an
equivalent mutant, since `domainResources` emits
`resources.requests.memory` under the same truthiness condition. It is
not: a `resources` block nulled outright leaves `.Values.resources` nil,
and the raw read dereferences it. The case pinning that shape kills the
mutant.

The update-hook cases assert the `kubectl patch` command the Job will
run, which is as far as a render-level test reaches. They do not show
that KubeVirt accepts the resulting object, that helm-controller
converges on a live VM whose `spec.instancetype` also carries a
`revisionName` written by virt-controller, or which half of a resize
lands as a live update rather than `RestartRequired`. That needs a
cluster.

The `dig` calls survive the shapes that reach them. `values.schema.json`
rejects a non-object `resources` before render; Helm coalescing turns a
nulled one into a nil `.Values.resources`, which digs to an empty
object; and for any other non-map intermediate, current Helm turns the
underlying sprig type assertion into a render error rather than a panic.

### Interaction with the firmware block

`firmware` (UEFI/BIOS selection, Secure Boot, persistent EFI NVRAM)
landed in this chart in the meantime, and it does not touch the sizing
question. `domainResources` does not read it, so it takes no part in the
completeness decision. KubeVirt's instancetype spec carries neither
firmware nor `domain.features`, so nothing an instance type declares can
conflict with what the firmware block renders. The boot mode a VM
inherits when the block is empty comes from the preference, and the
preference is not what gets dropped when complete resources replace the
matcher. Both firmware states are covered on each side by the
byte-identity comparison above.

`README.md`, `values.schema.json`,
`api/apps/v1alpha1/vminstance/types.go` and the ApplicationDefinition
all come out of `make generate` in the package, not out of a hand merge.
Re-running it on the result produces no further diff, and the root `make
generate` reports no drift on top of it.

### On the ticket's second proposal

#2909 also asks whether the non-empty `u1.medium` default belongs in the
chart at all. That requirement is met, and not by removing the default.
The default was a problem because a schema-generated form gave no way to
say "no instance type": clearing the field falls back to `u1.medium`.
Filling cpu, sockets and memory now says it directly, so the obstacle
the ticket names is gone while the default stays. Dropping the default
on top of that would change the rendered spec of every release that
never set the field, with nothing left to gain.

### Named improvements, not done here

The failure message names the fields that did not size the VM, computed
from what was emitted, so for `cpu: 4` with no `sockets` it names
`resources.cpu` even though the user set it. The message states the pair
rule that explains this, but naming the actually-missing field first
would read better. The two tests that pin `domain.cpu.cores: 0` document
the pre-existing `int64` truncation in prose rather than through a
`FIXME` reference, so a later reader has to read the comment to tell a
contract from a bug we chose not to fix. Both are wording; neither
changes behaviour.

`resources.cpu`, `sockets` and `memory` are checked for shape and not
for value, so `memory: "0"` and `-1Gi` pass the schema and fail further
down. Both forms fail before and after this change and only the place of
the failure moves, so that belongs with
[#4104](https://github.com/cozystack/cozystack/issues/4104) rather than
here.

The new template comments run to about 46 lines against about 28 lines
of logic. All of it is why rather than what and all of it is accurate,
but the helper's third paragraph restates the commit body, and two
sentences would carry that weight. The seven-line block in `vm.yaml` is
the one worth keeping: swapping that gate to the effective value
survives the entire suite, so what the block records is an equivalence
no test can pin, and trimming it would delete the only record of it.

The `instanceType` description runs to about 370 characters and reaches
the dashboard as tooltip help, through `values.schema.json` and the
ApplicationDefinition. That is long for a tooltip. It is accurate as it
stands, and rewriting accurate text for length is a fresh unmeasured
claim in the surface with the most readers, so it is not done here: the
place for the detail about what a dropped instance type takes with it is
the chart README, leaving the field help carrying the precedence rule
alone.

Choosing a `cx1.*` type and then giving it complete resources drops what
the type guaranteed along with the type, and nothing says so at runtime.
That is a consequence of the precedence rule rather than a defect in it,
and the field help states it. A runtime signal would need the chart to
compare what the type declares against what `resources` sets, which is
its own task.

Defaulting `sockets` to 1 would let a user size a VM from cpu and memory
alone and would remove the commonest way to land in the partial case. It
is more than the ticket asked for and it changes the rendered spec of
stored releases that set cpu and memory without sockets, so it needs its
own call on compatibility.

### Not changed

Two pre-existing quirks stay as they are, because both reach past the
conflict this fixes. `resources` fields are read for truthiness, so an
explicit `0` reads as unset, in `domainResources` and in the existing
"Either instanceType or resources" check alike. And with no instance
type in play, a fractional `resources.cpu` still truncates to `cores: 0`
and renders a VM sized by nothing, with no error. The new guard covers
only the half where a matcher is present. The tests pin both as they
stand, which is what keeps working releases rendering.

### Screenshots

No dashboard code is touched. The only user-visible surface is two field
descriptions the dashboard renders as help text, regenerated into
`values.schema.json` and the ApplicationDefinition. `instanceType` now
reads "Ignored when `resources` sizes the VM, which needs whole-number
cpu and sockets plus memory; the whole type is dropped then, so any
dedicated CPU placement, hugepages, NUMA or realtime settings it carries
go with it. While this field is set, a `resources` block that sizes only
part of the VM is rejected outright instead of ignored, so complete it
or clear it." `resources` states the same rule from its own side.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

I walked the trigger map against the diff and no listed trigger fires:
no package added, renamed or removed; no `values.schema.json` field
added, renamed or removed; no default changed; nothing under `hack/`
moved; `templates/service.yaml` and its `wholeIP` and `allowICMP`
annotations untouched. Two repositories do carry prose this makes
incomplete. I left every box empty rather than push follow-ups nobody
agreed to, so a maintainer decides.

`cozystack/website`: `content/en/docs/next/virtualization/resources.md`
opens with "Each virtual machine has these two configuration settings"
and lists only `instanceType` and `instanceProfile`. The page is
hand-written, so the release-time bot that regenerates app reference
pages from `README.md` will not reach it.

`cozystack/terraform-provider-cozystack`:
`internal/provider/vminstance_schema.go` describes `instance_type` as
"Virtual machine instance type." Its `kubernetes_schema.go` and
`kubernetes_nodes_schema.go` already spell out the equivalent precedence
rule for their own charts. No schema field or default changed here, so
its new-field test guard has nothing to catch.

### Release note

```release-note
fix(vm-instance): a complete `resources` block (whole-number cpu and sockets, plus memory) now sizes the VM directly and suppresses the `instanceType` matcher, instead of rendering both and having KubeVirt refuse the VM. Beside an `instanceType`, a `resources` block that sizes the VM only in part now stops the render with a message naming both sides, where it previously produced a VM the admission webhook rejected. Note that suppressing the matcher drops the whole instance type, including any dedicated CPU placement, hugepages, NUMA or realtime settings it carries. A release whose values already carry both converts on the next platform upgrade rather than at a moment the operator picks: the pre-upgrade hook strips `spec.instancetype` off the live VM and writes the domain sizing in its place, so plan that upgrade as if the VM restarts.
```
2026-09-10 19:21:59 +04:00
Aleksei Sviridkin 1f04e31ed3 feat(backups): add Redis backup/restore strategy driver (#3992)
## What this PR does

Adds backup/restore support for the `Redis` app (spotahome
`RedisFailover`) via a dedicated `strategy.backups.cozystack.io/Redis`
strategy controller. The spotahome operator ships no backup CRD, so the
driver follows the generic-script-Job pattern (a specialisation of the
Job/Altinity drivers): it runs a one-shot Job that dumps an RDB snapshot
from the current master to the system bucket and replays it on restore.

- **Backup**: discover the current master via the operator's sentinel
Service (`rfs-<name>`, master `mymaster`) → `redis-cli --rdb` → `curl
--aws-sigv4` PUT to `s3://<bucket>/<namespace>/<app>/dump.rdb`.
- **Restore**: GET the object → convert RDB → RESP (`hdt3213/rdb`) →
`FLUSHALL` + `redis-cli --pipe` into the target's master. Both
**in-place** and **to-copy** are implemented; a to-copy restore reads
the source's prefix via `.Backup.ApplicationRef.Name` while writing into
a differently-named target.
- **PITR**: not supported — RDB is a discrete snapshot with no
continuous log to replay to an arbitrary time (the etcd-like case).

The driver is a specialisation of the Job/Altinity script-Job flow with
a `Redis` applicationRef gate and an app-readiness precondition
(surfaces a precise `Ready=False` instead of a silent hang; blocks only
on an explicit `Ready=False` so a missing condition never wedges
backups). Strategy parameters round-trip through the Backup's
`DriverMetadata`; there is no `underlyingResources` snapshot (the Job
pattern re-renders on restore).

### System bucket

Coordinates and credentials reach the Job as env `secretKeyRef` from the
projected `cozy-backups-creds` (the ClickHouse/Altinity delivery model),
so **the app chart is untouched** — no `backup.useSystemBucket`, no new
RBAC (`batch/jobs` and `apps.cozystack.io/*` are already granted). The
bare-host `endpoint` gets `https://` prepended when unscheme'd; against
a self-signed endpoint the strategy verifies with the CA from
`backupStorage.endpointCASecretName` when set, otherwise against the
image's system CA store; skip-verify is a separate explicit
`backupStorage.insecureSkipTLSVerify` opt-in, never a fallback. The
`cozy-default-redis` strategy is gated on `$bucketName` (CRD-ordering)
and bound in the `cozy-default` BackupClass.

### Changes

- `api/backups/strategy/v1alpha1/redis_types.go` + generated CRD and
deepcopy.
- `internal/backupcontroller/redisstrategy_controller.go` (+ unit tests)
wired into both dispatch switches, `supportedBackupStrategyKinds`, and
the no-op restore-cleanup case.
- `cozy-default-redis` strategy template + `cozy-default` BackupClass
route + helm rendering-gate coverage.
- `images/redis-backup` (redis-cli + curl + RDB→RESP converter) +
Makefile target + `backupStrategyController.redisBackupClientImage`.
- `examples/backups/redis/` — an honest marker round-trip for both
restore flows.

### Validation

`go build`/`vet`, `go test ./internal/backupcontroller/...
./api/backups/...`, `make generate` (clean), and `helm unittest` (10/10)
all pass. Not yet cluster-verified.

## Downstream

None.

```release-note
Add backup and restore support for the Redis application (spotahome RedisFailover) via a new Redis backup strategy, wired into the cozy-default BackupClass. Supports in-place and to-copy restore; PITR is not supported.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Redis backup and restore support using RDB snapshots in
S3-compatible storage.
  * Supports restoring data in place or to a separate Redis instance.
* Added configuration for Redis backup images, custom HTTPS certificate
authorities, and TLS verification.
* Added a complete Redis backup and restore demonstration with automated
setup, verification, and cleanup.

* **Documentation**
* Documented Redis storage, credentials, endpoints, TLS, and restore
behavior.
  * Added administrator and tenant scenario guides.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-10 19:17:26 +04:00
Alexey Artamonov 9033d41bc0 test(postgres): assert the job pin on each autoscaler selector, not once for the query
The pin check matched job="kube-state-metrics" anywhere in the trigger query, so
one pinned selector satisfied it no matter what the other three did: unpinning
$joinReplica, $idleFloor or $joinCluster individually left the suite green while
the join went back to matching any exporter's series. Anchor one assertion per
selector on the label that identifies its site so a single dropped pin fails.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-10 15:31:31 +03:00
Alexey Artamonov 0d8b249b15 fix(cozy-lib): guard fractional-negative minReplicaCount and cover the pause-merge collision
The negative-floor guard read $min, already int .minReplicaCount, and int
truncates toward zero, so a fractional negative such as -0.5 became 0 and slipped
past into exactly the scale-to-zero-capable ScaledObject the type check above
exists to prevent. Test the untruncated value instead.

The annotations test only set an extra key with no pause, so it exercised the
pass-through branch and never the merge where the pause pair is the destination
and must win a key collision. Pass a colliding paused-scale-in="false" alongside a
distinct key and assert the pause value overrides it on the rendered object.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-10 15:31:24 +03:00
Aleksei Sviridkin b083d8e9e6 test(platform): pin the opensearch-operator DNS base wiring
Two ends of the same path. The platform suite pins that the shared
values Secret carries networking.clusterDomain to the operator under
the subchart key, and the package suite pins that a domain set on that
key reaches the manager container as DNS_BASE, with the upstream
cluster.local default intact when nothing sets it.

Every pattern in the platform suite anchors dnsBase to its
opensearch-operator and manager parents rather than matching the key
anywhere in the payload. An unanchored pattern is satisfied by any
sibling block that happens to carry the same domain, which leaves the
assertion green after the block it is meant to pin has been deleted.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-09 23:57:29 +03:00
Aleksei Sviridkin 857ad4a128 fix(opensearch-operator): resolve in-cluster names on the platform domain
The operator reads DNS_BASE and builds every in-cluster name it needs
from it: the securityconfig job target, its own OpenSearch API client
URL, the Dashboards host, and the SANs it writes into node
certificates. The vendored chart defaults that to cluster.local, so on
a cluster whose domain is cozy.local the securityconfig job never
resolves its target, the .opendistro_security index is never created,
and the OpenSearch nodes never go Ready.

The domain the platform already holds could not reach the operator
before: _cluster stops at each package's own templates and never
reaches a vendored subchart, and the wrapper's values.yaml is not
templated. Carry it in the shared values Secret under the subchart key
instead, the same way the KEDA cluster domain travels, so the vendored
chart picks it up with no patch and a package rendered without the
Secret keeps the upstream cluster.local fallback.

A global key would have worked for the operator but reaches every
release on the cluster, and the vendored smee chart in bootbox cannot
render once global is non-empty. Scoping the value to the subchart
that consumes it avoids putting every other package on that path.

The diagnosis and the first working fix came from fuad00 in an earlier
pull request.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-09 23:56:12 +03:00
Alexey Artamonov 2d1e425339 docs(postgres): correct the KEDA-guard rationale and note the gradual client-side rollout
The KEDA-missing fail-closed guard's comment still described the replaced
drop-instances design ("db.yaml drops spec.instances ... CNPG collapses to 1");
under the shipped seed design db.yaml seeds instances at max(replicas,
effectiveMin), so a skipped ScaledObject leaves a fixed-size cluster with no HPA
- autoscaling silently absent, not a collapse. Correct the rationale. Also note
on the cozyrds apply-strategy annotation that the client-side switch rolls out
per-app as each Application is next written through the aggregated API, not
fleet-wide at upgrade (the ApplicationDefinition reconciler syncs only
chartRef/valuesFrom/labels onto a live HelmRelease). Follow-up from #3954 (#4187).

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-09 21:36:54 +03:00
Alexey Artamonov 154548f57c fix(cozy-lib): reject a negative minReplicaCount and cover the annotations parameter
cozy-lib.keda.scaledObject only checked max >= min, so an all-negative or
mixed-sign pair (min -5, max -3) passed the ordering guard and rendered a
ScaledObject with a nonsensical floor. Reject minReplicaCount < 0 before the
ordering check. Also add a cozy-lib-tests case for the documented annotations
parameter, which had no caller and no coverage. Follow-up from #3954 (#4187).

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-09 21:36:54 +03:00
Alexey Artamonov 1f2ff31470 fix(postgres): pin job=kube-state-metrics on the autoscaler read-load query
The read-load trigger query joined on kube_pod_labels without pinning
job="kube-state-metrics", while the sibling DatabaseAutoscaler* alert rules
already pin it. A second exporter emitting a kube_pod_labels series (or a
duplicate scrape) would multiply the join and distort the desired-count
arithmetic. Pin the job on every kube_pod_labels selector in the query for
parity with the alerts. Follow-up from #3954 (#4187).

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-09 21:36:54 +03:00
scooby87 afc003c464 feat(postgres): database read-replica autoscaling via KEDA (#3954)
## What this PR does

Adds automatic horizontal autoscaling of a PostgreSQL database's **read
replicas** in response to load, implementing the merged design proposal
[Database Horizontal
Autoscaler](https://github.com/cozystack/community/tree/main/design-proposals/database-horizontal-autoscaling)
(community#44). The mechanism is entirely stock — no bespoke operator,
no new CRD:

- **KEDA as an optional platform component** (`packages/system/keda`,
wired into `core/platform` as a source + system-bundle entry). It claims
the `external.metrics.k8s.io` APIService that nothing served before.
- **`cozy-lib.keda.scaledObject` helper**
(`packages/library/cozy-lib/templates/_keda.tpl`) renders a KEDA
`ScaledObject`, fail-closed on missing/invalid arguments.
- **`packages/apps/postgres` autoscaling** — an `autoscaling` values
block (`enabled`, `minReplicas`, `maxReplicas`, `metric`, `target`,
`maxReplicationLagSeconds`, `dryRun`, `transition`) validated by
`values.schema.json`; under active autoscaling the chart renders a
`ScaledObject` and seeds `spec.instances` to a constant floor
(`max(replicas, effectiveMin)`), while the postgres release is forced to
client-side (two-way) Helm apply so the seed is a merge no-op on upgrade
and the KEDA-managed HPA — the runtime writer via the CNPG `Cluster`
scale subresource — is not reverted.
- **Single-value metric (§1)** — one PromQL query returns `Σ(read load
over standbys) + target`, so a stock HPA computes `desired =
ceil((Σ+target)/target) = 1 + ceil(Σ/target) = primary + desiredRead`,
with no per-pod emission.
- **The four static brakes (§5)** — quorum floor `max(minReplicas,
maxSyncReplicas+1, 2)` (quorum wins over `maxReplicas`), scale-down
pacing (`Pods/1/600s`), a replication-lag clamp folded into the query
(freeze both directions while lag exceeds the threshold and the primary
is writing, with `max_over_time` hysteresis), and dry-run (render
paused).
- **Two-phase enable/disable migration** via a `transition` sub-flag:
phase 1 stages `spec.instances` to the live count and brings KEDA up
with its HPA paused in both directions; flipping to phase 2 hands the
runtime count to KEDA, and because the seed re-renders the same count
the flip is a merge no-op with no dip.
- **Observability** — a database-autoscaling Grafana dashboard and
PrometheusRule alerts.

**Live validation (dev cluster, CNPG 1.30.0).** The full path works
end-to-end: the CNPG `Cluster` scale subresource serves
`status.selector`, KEDA renders the HPA, the metric reaches it
(`ScalingActive=ValidMetricFound`, `TARGETS 151/150`, `desiredReplicas=2
= 1 primary + 1 replica` — the §1 arithmetic), and the lag-clamp query
returns the base value on live VictoriaMetrics. Two packaging fixes were
needed and are included: KEDA must run on the cozystack `cozy.local`
cluster domain (the upstream `cluster.local` default makes the metrics
gRPC target NXDOMAIN and every HPA hangs on `FailedGetExternalMetric`),
and the operator's client-go throttle is raised so a single-replica
operator does not lose its leader lease under scale-subresource polling.

Implements #3953.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff. Two repositories are affected; both are follow-ups to open once
this lands (and after its dependency #3951 merges and the values surface
is final), left unticked here for a maintainer to confirm rather than
claimed with a speculative link:

- **cozystack/terraform-provider-cozystack** — this adds the nested
`autoscaling` block to `packages/apps/postgres/values.schema.json`. The
provider hand-models each app's fields (schema + model +
expand/flatten), so it needs the new block to expose it in Terraform.
- **cozystack/website** — KEDA is a new platform component
(`guides/platform-stack`), and the postgres reference page gains the
`autoscaling` params. The managed-app reference is regenerated from
`README.md` by the release bot; the platform-stack component list is
hand-maintained.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up: to open (new platform component + postgres autoscaling
params)
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: to open (new `autoscaling` block in postgres
values.schema.json)

### Depends on

- #3951 — bump CloudNativePG to 1.30.0. The HPA can only drive the CNPG
`Cluster` scale subresource on CNPG ≥ 1.28.4, where it serves
`status.selector`; without it the HPA fails with `InvalidSelector`. This
PR should not merge before #3951.

### Release note

```release-note
feat(postgres): add optional horizontal autoscaling of read replicas via KEDA, configured through the new `autoscaling` values block; KEDA ships as a new optional platform component. Requires CloudNativePG >= 1.28.4 for the Cluster scale subresource selector. Note: every Postgres release now applies client-side (`release.cozystack.io/helm-server-side-apply: "false"`) so the autoscaler's live `/scale` count is not reverted by Helm; a kind-wide side effect is that chart-rendered fields on Postgres releases are no longer re-asserted by server-side-apply drift correction (driftDetection is unset), a documented trade-off with a per-instance apply strategy as follow-up.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added configurable PostgreSQL read-replica autoscaling with KEDA.
* Added CPU and read-connection metrics, replica bounds, replication-lag
safeguards, dry-run mode, and transition handling.
* Added optional KEDA platform support, Grafana autoscaling dashboard,
and Prometheus alerts.

* **Documentation**
* Clarified integer thresholds, prerequisites, configuration, and
transition guidance.

* **Bug Fixes**
* Improved validation and actionable errors for unavailable KEDA or
invalid settings.

* **Tests**
* Expanded coverage for autoscaling, observability, package selection,
and validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-09 21:28:34 +03:00
Andrey Kolkov 92edfb810b fix(backups): run the strategy bats under bash; move template essays to {{/* */}}
The behavioural bats test ran the rendered `set -euo pipefail` script via
`sh`; the CI unit-tests runner's /bin/sh is dash, which rejects `-o pipefail`
at line 1 and reddened the whole suite (production is busybox ash, which
supports it, so the script itself is fine). Run it via `bash`, matching ash
semantics and the hack/migration-50-etcd-adopt.bats precedent.

Convert the multi-paragraph `#` rationale blocks in the shipped strategy
template to `{{/* */}}` so Helm strips them at render instead of copying the
prose verbatim into every rendered Strategy CR, where it drifts as the code
changes. Short one- to three-line guard comments stay as `#`.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-09 19:23:37 +04:00
Andrey Kolkov 1289f522fa fix(backups): unbreak install, kill destructive cleanup, reach the resource knob
Store the Redis strategy Template schemaless, as the Rabbitmq strategy
already does: without the markers controller-gen emitted the full
PodTemplateSpec schema and the CRD reached ~602 KB, pushing the chart's
Helm-release Secret past the 1 MiB cap so backupstrategy-controller failed
to install at all.

Give the example strategy a cleanup branch, and both scripts an explicit
`elif restore` / `else exit 1`, so a deleted Backup rendered with
Mode=cleanup deletes its object instead of falling into the restore path
and FLUSHALLing the live master.

Forward a backupStrategyController block through the platform bundle
alongside backupStorage, so redisBackupResources — the restore-Pod sizing
knob the loader-not-ready message points at — is reachable via
spec.components.platform.values.backupStrategyController, not merely
settable on a Helm-managed Strategy CR that reverts on the next upgrade.

Correct the endpointCASecretName doc row to match the now-optional mount,
and `int` the activeDeadlineSeconds interpolation so a large values-file
override renders as an integer, not exponent form.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-09 16:53:50 +04:00
Andrey Kolkov 651a6e3023 fix(backups): harden the redis restore AOF wait and image build
Wait for the throwaway loader's AOF rewrite to finish - no rewrite in
flight, last one ok, base AOF on disk - instead of a bare
aof_rewrite_in_progress:0 check, which right after `appendonly yes` can
fall through before the scheduled rewrite starts and replay an absent or
half-written base AOF. Same fix in the example strategy.

Declare the package image targets PHONY and serialize image-redis-backup
after image-backupstrategy-controller so their `yq -i values.yaml` digest
stamps cannot race under `make -j`. Run the demo cleanup under `set -e`
(every delete is already --ignore-not-found) so a real API failure aborts
instead of printing "Cleanup complete" over a half-torn-down demo.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-09 16:23:12 +04:00
Andrey Kolkov b4d621f4f7 fix(backups): delete Redis backup objects on cleanup and size the restore Pod
Delete the backup object when its Backup is removed. The Redis driver
owns its S3 object (one per BackupJob, no engine retention prunes it), so
it now spawns a cleanup-mode Job that deletes the object and holds the
Backup's finalizer until it completes, mirroring the Rabbitmq driver.
Without this a retention-pruned Plan grew the bucket unboundedly and the
key was lost with the CR.

Size the strategy Pod from backupStrategyController.redisBackupResources
so an admin can raise the restore ceiling (the throwaway loader holds the
whole dataset in memory) via chart values, which a direct edit of the
cluster-scoped, Helm-managed Strategy CR could not survive. The
loader-not-ready diagnostic now names the memory limit, the likeliest
cause, instead of only the image version.

Make the endpoint CA volume optional: nothing projects that Secret into
the app namespace, so a required mount wedged every backup Pod on
FailedMount; a missing ca.crt now falls through to the system CA store
and fails the handshake fast, which is not the same as skipping verify.

Promote the Pod-level activeDeadlineSeconds to the Job so a wedged run is
bounded once, not (backoffLimit+1) times. Use find, not ls, for the base
AOF so its no-match diagnostic is not dead code under pipefail.

Pin the object key reaching the rendered Job: both fixtures now render
--key={{ .ObjectKey }} and assert it, so zeroing the key (which left the
package green) is caught. Document .ObjectKey and the cleanup mode in the
CRD render-context, list cozy-default-redis among the Helm-managed CRs,
and mirror the CA/TLS and find fixes into the example strategy.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-09 12:19:02 +04:00
Andrey Kolkov 56e10b1580 fix(backupstrategy-controller): gate the default MongoDB Strategy on its CRD (#3980)
## What breaks today

Upgrading any existing cluster to a build that carries the new MongoDB
backup driver wedges the `backupstrategy-controller` HelmRelease
permanently, with `resource mapping not found for name:
"cozy-default-mongodb" ... no matches for kind "MongoDB" in version
"strategy.backups.cozystack.io/v1alpha1"; ensure CRDs are installed
first`.

This chart ships its `strategy.backups.cozystack.io` CRDs as ordinary
templates, and Helm resolves every document in a release manifest
through the cluster's RESTMapper in `Build()` **before** it applies any
of them. A Strategy CR whose CRD is not served yet therefore fails
`Build()`, so the release applies **zero** objects — including the CRD
that would have made the mapping resolve — and every retry renders the
same manifest. Nothing recovers on its own.

## Why the gate that exists does not cover it

The chart already knows this trap: it is why every default Strategy
gates on the bucket name. That gate covers the **first install** of a
fresh cluster, where nothing is resolvable yet and no Strategy renders
at all. It cannot cover shipping a **new kind** to a chart that is
already installed — there the bucket name resolved releases ago, so the
first upgraded revision renders the new CRD and the new Strategy CR
together and wedges.

## The fix

Gate the CR on its CRD actually being served, through a `crdEstablished`
helper. The helper asks about the **CRD object** rather than listing the
custom kind, because `lookup` on a kind the apiserver does not serve is
a render *error*, not an empty result, which would trade this wedge for
a different permanent failure; `apiextensions.k8s.io/v1` is always
served, so that lookup can only answer yes or no. It requires
`Established=True` rather than mere existence, because that is when the
kind enters the RESTMapper.

Convergence afterwards is the mechanism the bucket gate already relies
on, and is **not** the HelmRelease interval: once the CRD is served, the
controller's `DefaultObjectsGate` sees the kind mapped and the CR absent
and forces a real Helm upgrade whose render includes it. Kinds that do
not map at all are skipped there (`meta.IsNoMatchError`), which is what
stops the gate forcing upgrades against a render that cannot yet produce
the object.

`bucketNameOverride` short-circuits the CRD lookup for the same reason
it short-circuits the bucket lookup: it marks an offline render — `helm
template`, a CI diff, a unit test — where there is no apiserver to ask.
Live deploys go through Flux and never set it.

## How it was found

The release upgrade E2E lane in #3276, on the `v1.6.2 → main` path.
`mongodbs.strategy.backups.cozystack.io` is the only definition added to
this chart since the tag, so this is reachable only by upgrading an
existing install — no install-only suite can see it.

## Verification

Reproduced and fixed against a live cluster that happens to be in
exactly the pre-upgrade state (the seven older strategy CRDs present,
`mongodbs` absent). `helm template --dry-run=server` of main's chart
fails with the error above and produces zero documents; the same render
with this change succeeds with 21 documents and the Strategy correctly
withheld. The helper's two branches were probed the same way against
that cluster: a served CRD answers `true`, an absent one answers empty.

The new unit case pins the gate offline through the external-S3 path,
where the bucket name resolves without the offline override so that only
the CRD gate can hold the CR back. Removing the gate turns exactly that
case red; the suite is 19/19 green with it.

```release-note
Fixed a permanent failure of the backupstrategy-controller release when upgrading a cluster to a version that adds a new backup-strategy kind: the default MongoDB Strategy is now withheld until its CRD is served, instead of failing the whole release with "no matches for kind".
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved backup strategy rendering during upgrades by waiting until
the required MongoDB resource definition is available.
* Preserved offline rendering when an explicit bucket name is provided.
* Prevented invalid default strategies from being generated when the
required resource definition is unavailable.

* **Tests**
* Added coverage for offline rendering and unavailable
resource-definition scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-09 11:17:28 +04:00
Aleksei Sviridkin 9e577a5771 docs(rabbitmq): say that deleting the application leaves the TLS Secrets behind (#4143)
## What this PR does

`packages/apps/rabbitmq/README.md` documented the key material that
survives turning TLS off, but not the same outcome on deleting the
application. The chart renders `Certificate` and `Issuer` objects and
never the Secrets, and cert-manager runs with
`enableCertificateOwnerRef: false`, so a Helm uninstall leaves
`rabbitmq-<name>-ca` (the CA private key) and `rabbitmq-<name>-tls` in
the tenant namespace with nothing referencing them. One paragraph names
the two Secrets and the cleanup command. Documentation only.

### Screenshots

Not a UI change.

### Downstream repositories

A README paragraph in one chart; the generated reference page is built
from `values.yaml` and does not move.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Documentation**
  - Added uninstall guidance for TLS-enabled RabbitMQ applications.
- Clarified that generated certificate Secrets must be manually removed
from the tenant namespace after application deletion.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 19:45:21 +04:00
Aleksei Sviridkin 2055341ba9 fix(opensearch-operator): raise vm.max_map_count on every node from a DaemonSet (#4152)
## What this PR does

OpenSearch will not start below `vm.max_map_count=262144` when the store
may use mmap, and nothing in the platform sets it. The Talos profiles
declare no sysctls and no documented node-preparation path sets one, so
nodes sit at the kernel default of 65530. The app chart asks the
operator to cover that with `spec.general.setVMMaxMapCount`, which
appends a privileged init container to every OpenSearch pod. Tenant
namespaces carry no PodSecurity label of their own, so they take the
cluster default, which Talos ships as baseline through the apiserver's
`PodSecurityConfiguration`. That default refuses the container and the
StatefulSet never produces a pod. On vanilla kubeadm, kind or k3s the
default is `privileged` and the same release comes up, which is why this
only bites on a Talos install.

The value cannot come from the pod at all. The kernel does not namespace
`vm.*`, so kubelet rejects it in `securityContext.sysctls`, and refuses
to start if it appears in `--allowed-unsafe-sysctls`. That leaves node
configuration or a privileged pod, and a privileged pod belongs in a
system namespace rather than in every tenant.

This adds a DaemonSet to the `opensearch-operator` package that sets the
value once per node from a privileged init container, and marks the
component privileged so the platform labels `cozy-opensearch-operator`
with `enforce=privileged`. The init container writes to `/proc/sys` and
nothing else, with no hostPath. The pod restarts with the node and
reapplies the value, so there is nothing to persist.

It raises the limit to a floor rather than assigning it, which matters
more than it looks. `vm.max_map_count` is node-global, and a stock
Ubuntu 24.04 node already reads 1048576 from a `sysctl.d` drop-in. An
unconditional write would drop that to 262144 for every workload on the
node, on every boot, and undo whatever the operator set it for. So the
init container reads the current value first and writes only when it is
below the threshold.

The idle main container runs its sleep under a shell that traps SIGTERM,
so a pod exits on its own when the DaemonSet rolls or a node drains,
instead of ignoring the signal and being killed after the full grace
period.

This does not fix OpenSearch on its own. The app chart still requests
the operator's init container through `setVMMaxMapCount: true`, so a
release stays blocked until the flip to `false` lands with #2682, after
this merges. The DaemonSet has to be in place first: flipping it without
one would trade the admission failure for a failed bootstrap check.
Carrying the sysctl in the node contract instead, across the Talos
profiles and the downstream node-prep paths, is tracked in #4144; that
is what would let this DaemonSet be retired.

Two things change on upgrade, and neither is visible from the diff. One
small DaemonSet pod appears per node in `cozy-opensearch-operator`, and
that namespace gains `pod-security.kubernetes.io/enforce=privileged`.
That label has no finer granularity than the namespace, so the operator
Deployment sitting there leaves the cluster default too, not just the
DaemonSet. Keeping the manager at baseline would mean a namespace of its
own and another component entry, which did not look worth it, but it is
a real cost rather than an accident. The operator is in the `paas`
bundle's always-on list, so this lands on every node of every paas
installation and raises a node-global kernel tunable there, whether or
not anyone ever creates an OpenSearchCluster. That is the price of a
node-level sysctl having no cheaper carrier, and it is worth an explicit
decision rather than being noticed later.

The second is `cozypkg add`, which gates privileged components behind a
confirmation or `--allow-privileged`. The gate runs over the resolved
dependency closure, and `opensearch-application` dependsOn the operator,
so installing the app that way now asks one more question. This is not a
new break for scripts: `add` already prompts for a variant on every
package it resolves and fails on EOF, so it was never usable
non-interactively in the first place. Platform-bundle installs go
through the reconciler and are unaffected.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff. It adds a template, a test and a values key under
`packages/system/opensearch-operator/`, plus one field in
`packages/core/platform/sources/opensearch-operator.yaml`. No new app
package, no app schema or version enum, no `ApplicationDefinition`, no
installer value, no namespace rename, no variant or bundle, no `hack/`
path or make target, and no node prerequisite in
`hack/e2e-prepare-cluster.bats`. Nothing in the map matches.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(opensearch-operator): set `vm.max_map_count` on every node from a DaemonSet in the operator's namespace, so OpenSearch no longer depends on a privileged init container that PodSecurity baseline refuses in a tenant namespace. Upgrading adds one small DaemonSet pod per node and labels `cozy-opensearch-operator` with `pod-security.kubernetes.io/enforce=privileged`; installing the package with `cozypkg add` now requires confirmation or `--allow-privileged`. Existing OpenSearch releases are unchanged until a follow-up stops the app chart requesting the operator's own init container.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- OpenSearch Operator installation now runs with the required privileged
configuration.
- OpenSearch deployments automatically ensure nodes meet the required
`vm.max_map_count` setting, improving startup reliability.
  - Existing higher kernel values are preserved.
- Added support for configuring the sysctl image, including use with
air-gapped registries.

- **Tests**
- Added coverage to verify privileged installation, node configuration,
security settings, scheduling behavior, and image customization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 19:45:14 +04:00
Alexey Artamonov 9511f56307 fix(postgres): require service.namespace.svc shape in the autoscaling SSRF host guard
The serverAddress host regex accepted any dotted tail ending in .svc, so a
single-label host like foo.svc passed the in-cluster-DNS check. It then reached
the namespace-pin, whose extraction regex left the unresolvable host unchanged
and rejected it as a namespace outside the tenant subtree - the right outcome by
the wrong path, with a misleading message. Anchor the host to two DNS labels
before .svc so it is refused at the host check with the actionable message, and
drop the stray dot the old label class allowed.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-08 18:06:34 +03:00
Andrey Kolkov 45a2f54c62 feat(backups): back up TLS-enabled Redis apps via the default strategy
The shipped cozy-default-redis strategy connected to Sentinel and the
master in plaintext, so a BackupJob against a Redis app with tls.enabled
(the operator moves both to a TLS-only listener) failed at master
resolution. Mount the app CA the operator publishes at
redis-<app>.ca-cert as an optional Secret, and dial redis-cli with
--tls --cacert exactly when its ca.crt is present, so a non-TLS app
keeps the plaintext path and the throwaway restore loader stays
local-only. Covers the default authClients: no; mutual TLS is out of
scope, since the platform issues no client certificate.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-08 18:47:13 +04:00
Andrey Kolkov b0eb104276 feat(backups): add Redis backup/restore strategy driver
The spotahome RedisFailover operator ships no backup CRD, so add a
dedicated strategy.backups.cozystack.io/Redis driver that runs a one-shot
Job: it dumps an RDB snapshot from the current master (discovered via the
operator's sentinel Service) to the system bucket over S3, and on restore
replays it (RDB -> RESP via redis-cli --pipe) into the target's master.

Structurally a specialisation of the Job/Altinity script-Job pattern with
a Redis applicationRef gate and an app-readiness precondition. Backup
parameters round-trip through the Backup's DriverMetadata (no
underlyingResources snapshot); restore drives from .Backup.ApplicationRef
so a to-copy restore reads the source's object-storage prefix while
writing into a differently-named target. Both in-place and to-copy are
supported; PITR is not (RDB is a discrete snapshot).

- api: Redis/RedisList strategy type + generated CRD and deepcopy
- driver: reconcileRedis / reconcileRedisRestore + pure helpers, wired
  into both dispatch switches, supportedBackupStrategyKinds and the no-op
  restore cleanup case
- default: gated cozy-default-redis strategy (system-bucket coords via
  cozy-backups-creds env secretKeyRef, https:// scheme + optional CA) and
  the Redis route in the cozy-default BackupClass
- image: images/redis-backup bundling redis-cli + curl + an RDB->RESP
  converter, wired via backupStrategyController.redisBackupClientImage
- tests: driver unit tests (gate, precondition, render, param round-trip,
  both reconcile paths) and the helm rendering-gate coverage
- example: examples/backups/redis honest marker round-trip for both
  restore flows

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
2026-09-08 17:56:20 +04:00
Alexey Artamonov 49e8aa58f9 fix(keda): pin job=kube-state-metrics in the ScaleStuck Pending join
The DatabaseAutoscalerScaleStuck Pending-intersection term selected
kube_pod_status_phase and kube_pod_labels without job="kube-state-metrics",
unlike every other selector in this file (the rule's own HPA terms, all of
ScaleBlocked, AtMax). Each VMRule runs against a tenant's own vmselect; if that
vmselect ever carries a second series for either metric under a different job
(a tenant-run KSM alongside federated/remote-written series), the on(namespace,pod)
match becomes ambiguous and the whole rule errors out silently while its siblings
keep working. Pin the job label on both selectors so the join stays one-to-one.
The alerts unittest now asserts the pin on both terms so a revert goes red.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-08 16:35:57 +03:00
Alexey Artamonov 6fa79c74f4 fix(cozystack-basics): strip the autoscaler RBAC rationale from rendered output
The 8-line rationale for the tenant read-on-ScaledObject/HPA grant was a `#`
YAML comment, which Helm copies verbatim into the rendered manifest in every
tenant and which then drifts out of date the moment the grant changes. Convert
it to a `{{/* */}}` Go-template comment so the WHY stays for maintainers but is
stripped at render. No RBAC rule changes; the 58 cozystack-basics unittests
still pass and `helm template` no longer emits the prose.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-08 16:35:57 +03:00
Alexey Artamonov 1979bd3f5c fix(cozystack-basics): let tenants read their own autoscaler objects
The autoscaling chart renders a ScaledObject into the tenant namespace and KEDA
creates a keda-hpa-<release> HorizontalPodAutoscaler beside it, but the
cozy:tenant:view aggregation granted no read on keda.sh or autoscaling. So the exact
failure the feature documents - a monitoring: true tenant whose derived vmselect
carries no kube_pod_labels, whose trigger never resolves - lands as a TriggerError
ScaledObject and a FailedGetExternalMetric HPA the tenant cannot get, surfaced only
in a root-scoped alert they cannot reach. A tenant who flips the knob and sees
nothing happen has no in-band way to find out why.

Grant get/list/watch on keda.sh/scaledobjects and autoscaling/horizontalpodautoscalers
in cozy:tenant:view:base. Read verbs only: the objects are managed by the chart and
KEDA, not edited by the tenant. Pinned by clusterroles-options unittest.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-08 10:48:16 +03:00
Alexey Artamonov a43e3e2709 fix(keda): digest-pin the three KEDA images
The shim rendered the operator, metrics-apiserver and admission-webhooks images
tag-only, and the values comment claimed a @sha256 pin was impossible without
patching the subchart. That is wrong: the vendored template renders
repository:{{ tag | default appVersion }}, and repository:tag@sha256:<digest> is a
valid OCI reference where the digest wins at resolution, so the pin goes straight
into the tag field of these shim values with no subchart change.

Pin all three to the multi-arch image-index digests for the 2.20.2 tag on ghcr.io,
matching how every other cozystack system image is pinned, and replace the
incorrect note. Guard the operator image keeps a digest with a helm-unittest so a
re-vendor that drops the override goes red rather than silently shipping a floating
tag. Bump the digests in lockstep with KEDA_CHART_VERSION on re-vendor.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-08 10:48:06 +03:00
Alexey Artamonov e54d728087 fix(postgres): stop an unready ScaledObject from wedging the release
A Postgres app with autoscaling.enabled but no resolvable metrics endpoint (a
monitoring: true tenant who did not point serverAddress at a vmselect carrying
kube_pod_labels) renders a ScaledObject that stays Ready=False reason=TriggerError
forever. helm-controller's default wait gates the whole HelmRelease on every
rendered resource's kstatus, so that one unready ScaledObject drives the release to
UpgradeFailed and it retries forever - wedging an otherwise-healthy database and
blocking every later chart change to it, while the tenant cannot even read the
ScaledObject to see why (observed on dev3).

Autoscaling health is a degraded-but-alive state surfaced by the DatabaseAutoscaler*
alerts, not a release-readiness gate. Add a spec.release.healthCheckExprs entry on
the postgres ApplicationDefinition that treats the ScaledObject as always-ready for
release readiness (current: "true"); the CNPG Cluster and every other resource keep
their built-in kstatus wait, so the database's own health still gates the release.
Setting any healthCheckExprs couples WaitStrategy to poller
(config.ResolveWaitStrategy); that the poller wait stays equivalent to the default
for the kstatus-checked Cluster is verified on cozystack-pr-test, not here
(render-blind). The cozyrd unittest pins the expr so a regeneration cannot drop it.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-08 10:47:48 +03:00
Aleksei Sviridkin 5db773892b fix(cdi): omit the CDI clone strategy override when unset
templates/cdi-cr.yaml rendered cloneStrategyOverride unconditionally,
so the chart's values offered no way to leave the choice to CDI's own
per-StorageProfile selection. The CRD enum takes copy, snapshot and
csi-clone and carries no empty member, so an empty value rendered
cloneStrategyOverride: "" which that enum does not accept, and a null
one rendered a bare key. Guard the field on a non-empty value, the
way uploadProxyURLOverride is already guarded just below it.

The default stays csi-clone, so a release that overrides nothing
renders what it rendered before. Whether csi-clone is the right
default is a separate question, untouched here.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-08 03:13:35 +03:00
Aleksei Sviridkin 4f337d62e5 fix(kubevirt): verify the vendored operator manifest and add a test target (#4099)
## What this PR does

`packages/system/kubevirt-operator` had only an `update:` target, so
`hack/helm-unit-tests.sh` skipped it: that script decides what to run by
probing `make -C <dir> -n test`. Nothing rendered the chart after a
refresh. The same recipe pulled the upstream manifest with `wget` and
ran two transforms over whatever came back, with no check on the
download.

`update` now fetches outside `templates/`, checks the download against a
pinned sha256, transforms it there, and moves it in last. The vendored
file gets written once per refresh with finished content, and a failure
at any step leaves it alone. It has to stay out of `templates/` because
Helm renders every file in there, and that directory also holds
`prometheusrule.yaml`, which is ours and not upstream's.

Upstream ships no checksum for `kubevirt-operator.yaml`.
`v1.9.0.tar.gz.asc` signs the source tarball, not the generated
manifest. So I measured the pin: downloaded the v1.9.0 asset, hashed it,
got `f11307caafc3c23ffedf9887d8beb5a4419e2694da242fa68f63d1ec820de2e0`,
and confirmed the recipe's two transforms turn that download into the
committed `templates/kubevirt-operator.yaml` byte for byte. It catches
an asset that moved under its tag. It is not authenticity verification,
and the comment says that.

The digest command is resolved at parse time: `sha256sum` when it is on
PATH, `shasum -a 256` otherwise. Worth saying which one actually ran,
because the usual claim about macOS is out of date: this machine picked
`sha256sum`, and it is not GNU coreutils but `/sbin/sha256sum`, a native
Darwin binary hardlinked to `/sbin/md5` that reports `sha256sum (Darwin)
1.0` and prints the same `<digest> <path>` format. I checked the
fallback separately with `make update SHA256SUM='shasum -a 256'`, which
finished and left the tree clean. The fallback still earns its place for
older macOS, but it does not make `update` portable on its own: `sed -i`
and `awk -i inplace` in the same recipe need GNU sed and gawk, and the
comment now says so.

`test:` runs `helm unittest`, then greps the vendored manifest for
`RELEASE`, same shape as `packages/system/reloader`. helm-unittest
cannot see `RELEASE`, so without the grep a pin bumped without
re-running `update` leaves every assertion green against the old
release. The suite pins that the `PrometheusRule` still renders, and
that the `virt-operator` Deployment carries the pinned release in its
image, `VIRT_OPERATOR_IMAGE` and `KUBEVIRT_VERSION`, plus both
transforms.

Every assertion is mutation-tested: removing `prometheusrule.yaml`,
drifting each of the three version strings, reverting the namespace
rewrite, adding a `Namespace` document back, renaming the Deployment,
and bumping `RELEASE` without re-vendoring all go red. `any: true` on
the negated `containsDocument` is doing real work, drop it and the suite
stays green with a `Namespace` document present. The sweep now runs 84
package suites, one of them new, all green.

Closes #4031.

### Screenshots

No UI changes.

### Downstream repositories

Walked the trigger map against the diff. Two entries look close and
neither fires. The website triggers on developer tooling and package
Makefiles, but `content/en/docs/next/development.md` documents `make
update` only as "Update Helm chart and versions from the upstream
source", which still holds, and its `make test` line refers to the e2e
sandbox in `packages/core/testing`, not per-package unit tests. `ccp`
triggers on `hack/` layout and what a make target does, and its skills
anchor on `hack/package.mk` and `hack/common-envs.mk`; nothing under
`hack/` is touched and its `package-bump` skill handles vendored charts,
in-repo image builds and version enums, none of which is this package's
raw manifest download. Everything else in the map needs an app, a
schema, an installer value, a variant or a release asset, and this PR
changes none of them.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(kubevirt): kubevirt-operator now verifies the upstream manifest against a pinned sha256 before vendoring it, and the package has a unit-test target so hack/helm-unit-tests.sh no longer skips it.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved manifest updates so incomplete downloads or integrity-check
failures no longer overwrite the existing working manifest.
  - Temporary download files are cleaned up after failed updates.

- **Tests**
- Added validation for the bundled KubeVirt operator manifest, including
the expected version, image configuration, namespace, monitoring alert,
and removal of an unwanted namespace resource.

- **Chores**
- Pinned the KubeVirt operator release and verified downloaded manifests
against their expected checksum.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 02:53:03 +04:00
Aleksei Sviridkin a6caad7c57 fix(platform): correct what disabledPackages does to ouroboros (#4098)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

`bundles.disabledPackages` does not uninstall anything. Two comments in
the platform chart said it does, offered next to `kubectl delete
package.cozystack.io cozystack.ouroboros` as if the two were
interchangeable.

A name in that list only stops `_helpers.tpl` emitting the Package
document, which is the same suppression you already get from leaving
`publishing.proxyProtocol` false. Every emitted Package carries
`helm.sh/resource-policy: keep`, and that annotation is there precisely
so disabling a package stops deleting it, so Helm leaves the Package CR
alone. The CR stays, the HelmRelease it owns stays, ouroboros keeps
running, and the pre-delete hook never fires because nothing uninstalls
the release. Deleting the Package CR is what gets there: GC takes the
HelmRelease, Flux uninstalls, the hook runs on the way out.

There was a second cost. The guard in the system bundle looks up the
live Package CR and never reads `disabledPackages`, so an operator who
followed the old advice hit the guard anyway with no idea why. New test
case pins that.

Comments only in the templates, so rendered output is unchanged. Chart
suite green at 156 tests.

### Screenshots

Not a UI change.

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

Walked the trigger map against the diff. One repo is reached, and I left
its box empty rather than tick it without a link.

`cozystack/website` has the same wrong instruction verbatim in three
files: `content/en/docs/next/networking/hairpin-proxy-protocol.md:122`,
and the identical line under `v1.4/` and `v1.5/`. It tells the reader
that removing the Package CR "(or add it to `bundles.disabledPackages`)"
triggers helm uninstall and fires the pre-delete hook. It also
contradicts `operations/configuration/components.md:72` on the same
site, which says `disabledPackages` will not remove components that are
already installed. The site fix is cozystack/website#683. It turned out
to be four files rather than three: `origin/main` there also carries a
`v1.6` tree, and all four copies were byte-identical.

Nothing else is reached: no package added or renamed, no
`values.schema.json`, no version enum, no default, no key the Ansible
role sets, nothing under `hack/`.

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/pull/683
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(platform): the platform chart no longer says `bundles.disabledPackages` uninstalls a package. A name in that list only stops the Package document being rendered. `helm.sh/resource-policy: keep` leaves the existing Package CR on the cluster and the component keeps running, so deleting the Package CR is what uninstalls it.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Documentation**
- Clarified proxy protocol safety guidance, including the effects of
disabling the Ouroboros package and the limitations of using
`bundles.disabledPackages`.
- Documented that disabling package rendering does not remove an
existing live package or stop its associated resources.

- **Tests**
- Added regression coverage confirming that disabling the package does
not bypass the proxy protocol safety guard when the live package remains
active.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 02:52:00 +04:00
Aleksei Sviridkin 1fc990dd7f fix(cdi): update CDI to v1.66.1 (#3920)
## What this PR does

CDI v1.64.0 -> v1.66.1, operator manifest refetched with `make update`.

The bump was stuck since v1.65.0: its bundled qemu-img broke HTTP
imports to block volumes on 4Kn devices (DRBD/LINSTOR). This PR first
targeted v1.66.0 on the assumption that
kubevirt/containerized-data-importer#4179 had closed it, and that was
wrong: the qemu-img in the v1.66.0 importer is 10.1.0-21.el9, which does
not carry the first-sector fix, and the E2E run on that head failed
exactly there. The fix is in 10.1.0-24.el9, which reached the release
branch through kubevirt/containerized-data-importer#4274 and shipped in
v1.66.1. Verified by running the released image: `cdi-importer:v1.66.1`
reports `qemu-img version 10.1.0 (qemu-kvm-10.1.0-24.el9)`.

The other v1.64.0 symptom is a resourceVersion storm on the prime PVC
during WFFC imports, which makes the scheduler retry FailedScheduling
dozens of times per import; it was reported by a production user
importing golden images on a local StorageClass. That one is fixed by
commit e547586e9, a one-line guard in `updatePvcFromPod` that clears the
bound-condition annotations only when they came from scratch-space
creation. It is in v1.66.0 and later. The commit is squashed under a
title about kubevirtci lanes, so it appears in no release note and
searching for it by symptom finds nothing.

Two local edits to the vendored manifest survive refetches now. The
startupProbe patch gets the correct -p4 strip level, as written it never
applied from the package directory. The control-plane tolerations were
hand-edited into the manifest with no patch behind them, so any refetch
silently dropped them; they live in patches/tolerations.diff now. `make
update` round-trips to a byte-identical file.

Behavior changes checked against this repo: /metrics on CDI components
now requires a bearer token, nothing here scrapes it. Scratch PVCs
inherit the target StorageClass instead of the cluster default (since
v1.65.0). The PVC-rendering webhook is always registered now, but only
matches PVCs labeled cdi.kubevirt.io/applyStorageProfile=true, which
nothing here sets.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(cdi): update CDI to v1.66.1. Fixes HTTP imports to block volumes on 4Kn devices (DRBD/LINSTOR) with the repaired qemu-img, and the resourceVersion storm on prime PVCs during WaitForFirstConsumer imports.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Kubernetes tolerations for critical add-on and control-plane
nodes.
* Added resource claim requests and webhook PVC rendering configuration
to CDI custom resources.
* Added permissions for token reviews, metrics, and profiling endpoints.
  * Updated affinity feature-gate documentation to beta status.

* **Improvements**
  * Updated CDI operator and operand images to version 1.66.1.
  * Refreshed Kubernetes schema metadata and deployment configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 02:51:10 +04:00
Aleksei Sviridkin d0fce925cf fix(platform): declare the injected _cluster values in system charts (#4102)
## What this PR does

Ten charts under `packages/system` read `_cluster`, and `bucket` and
`monitoring` also read `_namespace`. The platform injects both through
the `cozystack-values` Secret, so installs never notice. Rendering a
chart on its own does: `helm template packages/system/kubevirt` dies on
`index of untyped nil` before it emits anything, and the `_namespace`
readers die on `nil pointer evaluating interface {}.host`.

Each of the ten now declares the keys it reads, with an empty default.
Injected values merge over chart defaults, so an installed release
renders exactly what it rendered before. Checked per chart against a
realistic injected `_cluster`/`_namespace`: six came out byte-identical,
and the other four differ only in `randAlphaNum` secrets, which also
differ between two renders of the same tree.

The declaration on its own is not worth much, so every chart also gets a
suite that renders with nothing injected. The existing suites all set
`_cluster` themselves and cannot see whether `values.yaml` still
declares it. Drop a declaration and exactly that chart's suite goes red,
with the error its comment quotes; I ran that for all twelve. None of
the new suites carries a suite-level `templates:` list, on purpose: that
list narrows what helm-unittest renders, and would hide the same
breakage in a template added later.

`kubevirt-cdi` had no `tests/` and no `test:` target, so it gets both.
`etcd-operator` reads `_cluster` too and stays as it is, since its one
call site already guards with `| default dict`. `cozystack-basics`
already declared the key.

Closes #4030.

### Not in this PR

No CI job runs `helm lint` or a standalone `helm template`, which is why
none of this was red. A gate is worth proposing separately. Its value is
narrow though: `_cluster` is always populated on a real install, and
eight of these ten already had a suite rendering the affected template
with `_cluster` supplied. The two with nothing were `kubevirt`, whose
`vm-exportproxy-*` templates no test touched, and `kubevirt-cdi`, which
had no tests at all.

`helm lint` still exits 1 on `keycloak` and `monitoring` with `chart
metadata is missing these dependencies: cozy-lib`. Both carry a
`charts/cozy-lib` symlink that `Chart.yaml` never declares. That one is
not a two-chart item: 82 charts across
`packages/{system,apps,extra,core}` fail lint the same way, which is
invisible because no job runs `helm lint`. Separate defect, untouched
here.

Fifteen charts outside `packages/system` break the same way:
`apps/{bucket,foundationdb,harbor,kubernetes,mongodb,nats,opensearch,postgres,vpn}`
and `extra/{bootbox,etcd,gateway,info,ingress,seaweedfs}`. `cozy-lib`'s
own helpers already guard every access with `| default dict`, so routing
those reads through the library would close the class instead of
patching fifteen more values files.

`kubevirt`'s suite only pins that the routes stay absent with nothing
injected. Nothing pins that they show up once `_cluster` selects the
service.

### Screenshots

No UI change: the diff is ten `values.yaml` files, ten helm-unittest
suites, and one package `Makefile`.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` file by file
against this diff. No package is added, renamed or removed under
`packages/apps` or `packages/extra`;
`packages/core/platform/values.yaml` is untouched; no variant, bundle,
component, release asset, namespace or `ApplicationDefinition` semantic
changes; no `values.schema.json` and no version enum moves, so the
provider's hand-written schemas and defaults are unaffected; nothing
under `hack/` moves and no existing make target changes behaviour. The
one `Makefile` edit adds a `test:` target to
`packages/system/kubevirt-cdi`, matching the target the other nine
already have, so `make generate` and the ccp skills that drive it see
nothing new.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(platform): declare the platform-injected `_cluster` and `_namespace` values in the ten system charts that read them, so each chart renders with `helm template` outside a cluster instead of failing on a nil map. No change to what an installed release renders.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved standalone Helm chart rendering when platform-provided values
are unavailable.
- Prevented rendering failures across system charts, including bucket,
dashboard, Keycloak, monitoring, KubeVirt, and related components.

- **Tests**
  - Added comprehensive standalone rendering coverage for system charts.
- Added validation for expected ingress, gateway, TLS route, and
resource output under different configuration scenarios.
  - Added a chart test command for KubeVirt CDI.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 02:50:43 +04:00
Aleksei Sviridkin 675163399d fix(platform): accept the container gpu-operator variant in iaas (#4101)
## What this PR does

`bundles.iaas.gpuOperatorVariant: container` failed the whole platform
render. The `cozystack.gpu-operator` PackageSource ships that variant,
`values-container.yaml` configures it deliberately, and
`hack/check-gpu-operator-variants.bats` covers it, but the `iaas` bundle
guard only accepted `default` and `vgpu`.

The exclusion made sense when it was written and stopped making sense a
few minutes later. The variant and the guard came from parallel
branches, neither an ancestor of the other, variant merged first. Before
the guard the bundle hardcoded `default`, so `container` was selected by
hand-writing a Package CR, and leaving it out of a bundle-side list cost
nothing. The guard made the bundle the variant owner and closed that
route without noticing it was one. Its comment calls itself typo
protection and names `passthrough` as the example typo.

Allowing the value is not the whole fix. The same file puts
`extraFeatureGates: [HostDevices]` on the KubeVirt CR whenever
gpu-operator is enabled, and builds a `permittedHostDevices` table under
it. Both are VM-facing. The container variant keeps the host driver
bound and pins `vfioManager` off, so no GPU reaches a VM: the gate would
advertise something the node cannot serve, and the table would name
resources no sandbox device plugin registers there. Both now render for
`default` and `vgpu` only, and `container` gets the gpu-operator Package
with the kubevirt component untouched.

The helm-unittest case that pinned the rejection is gone. Two replace
it: the Package renders at the container variant with no component
overrides, and the KubeVirt wiring stays absent even with
`gpu.permittedHostDevices` and `gpu.mediatedDevicesConfiguration` set.
Comments and docs that called the mirroring unconditional on variant, or
the container variant unmerged, are fixed in the same pass.

No issue for this one, so the history is in the body.

### Screenshots

No UI change.

### Downstream repositories

Walked the trigger map against the diff. The only file it touches that
appears there is `packages/core/platform/values.yaml`, and the website
hit is not the hand-written key table (that page has no GPU section): it
is the documented workaround this change makes obsolete.
`content/en/docs/next/operations/gpu-container-workloads.md` currently
tells readers the bundle rejects `container` and routes them to a
hand-crafted Package CR, and `virtualization/gpu.md` / `vgpu.md`
describe the KubeVirt mirroring as unconditional on variant.

Filed as an issue rather than a PR because the docs must not land ahead
of this change, and the `v1.6` copies describe released behaviour and
stay correct as they are.

Nothing else matches: no package added, renamed or removed, no
`values.schema.json` or version enum touched, no `ApplicationDefinition`
semantics, no installer values key, no `hack/` layout change, no
platform variant or bundle recomposed. `gpuOperatorVariant` appears in
none of `ccp`, `ansible-cozystack` or `terraform-provider-cozystack`.

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/issues/685
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(platform): accept `container` as a value for `bundles.iaas.gpuOperatorVariant`, so the gpu-operator variant the platform already ships can be selected from platform values instead of a hand-crafted Package CR. The KubeVirt `HostDevices` feature gate and `permittedHostDevices` table are now emitted for the `default` and `vgpu` variants only; `container` serves GPUs to pods and gets neither.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added the `container` GPU operator variant for workloads that do not
require VM passthrough.
* Container workloads no longer receive KubeVirt host-device or
mediated-device GPU wiring.
* Clarified GPU configuration behavior for passthrough, vGPU, SR-IOV,
and container workloads.

* **Documentation**
* Updated passthrough guidance, configuration references, examples, and
KubeVirt wiring instructions for all supported variants.
* Added details for manually configured package resources and
platform-specific container setup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 02:50:33 +04:00
Aleksei Sviridkin b7e60855e9 test(tests): wire the gpu-operator suite into the unit sweep (#4110)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

`hack/helm-unit-tests.sh` decides whether to enter a package by asking
`make -C <pkg> -n test`. `packages/system/gpu-operator` declared only
`update:`, so the sweep skipped it silently and the three test cases in
`tests/kernel_module_config_test.yaml` had never run. They pass, so
nothing was hiding behind them. Still worth fixing: the directory reads
as coverage to whoever opens it, and a suite nothing runs is where a
stale assertion sits.

The target is phony. `hack/package.mk`'s `.PHONY` line lists only the
targets that file defines, so a path named `test` next to the Makefile
would make the target up to date and skip the recipe.

Rather than fix the one package I enumerated the rest of the tree. 84
packages hold a `tests/` directory, 83 ship at least one
`tests/*_test.yaml`, and gpu-operator was the only one of those without
the target. The 84th is `packages/system/nfs-driver`, whose `tests/`
holds a PersistentVolumeClaim and a StorageClass to apply by hand. No
suite there, so a `test:` target would only make the sweep fail it for
reporting zero suites. Left alone.

`hack/package-test-target-coverage.bats` pins the class. Every other
guard in the sweep runs inside a directory it already reached, so a
package with no target escapes all of them at once and the run still
reports success. The new case probes with `make -n` instead of grepping
for `test:`, which is the question the sweep asks, and it mirrors the
sweep's other precondition too: the sweep requires `$dir/Makefile` and
returns before probing, where make would also accept `GNUmakefile` or no
makefile at all. The verdict comes from the sweep's command unchanged,
and a failing package is probed a second time with `LC_ALL=C
--no-print-directory` purely to read the diagnostic, since both of those
are visible to a Makefile through `LC_ALL` and `MAKEFLAGS`.

Mutations, because a guard that has never been red proves nothing:

```text
target removed from gpu-operator  -> red, names packages/system/gpu-operator, quotes make's error
Makefile renamed to GNUmakefile   -> red; make still answers 0 here, so this leg is load-bearing
glob pointed at a missing dir     -> red via the blindness leg, instead of passing on an empty set
unmodified tree                   -> green, 83 packages scanned
```

The full `hack/helm-unit-tests.sh` sweep now enters gpu-operator and
finishes green.

One detail that is invisible on a Mac: the probe runs under
`--no-print-directory`. On modern GNU make `-C` turns on `-w`, and the
fatal path prints `Leaving directory` after the error, so `tail -n 1`
picks up the directory trace and drops the diagnostic. Measured on 4.4.1
where that happens, and on 3.81 (still `/usr/bin/make` on macOS) where
`-C` prints no trace at all.

### Screenshots

No UI changes.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

Walked the trigger map against both changed files. Two entries come
close and both miss. `ccp` gates on `hack/package.mk`,
`hack/common-envs.mk` and the `packages/{apps,system,extra,core}`
layout; nothing here is moved or renamed and no existing target changes
what it does, so a new `test:` target sitting beside 80 identical ones
is not something its skills read. The `website` entry for developer
tooling points at `content/en/docs/next/development.md`, whose Testing
section covers the e2e sandbox under `packages/core/testing` and says
nothing about the per-package helm-unittest sweep.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
test(tests): run the gpu-operator helm-unittest suite, which the sweep had been skipping because the package declared no `test:` target
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Tests**
- Added coverage checks to verify that all Helm test suites are included
in the package test sweep.
  - Added diagnostics for packages missing a runnable test target.

- **Bug Fixes**
- Added a test target to the GPU operator package, enabling its Helm
tests to run through the standard package test workflow.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-08 02:49:10 +04:00
Aleksei Sviridkin a3ddba441f fix(opensearch-operator): set vm.max_map_count from a node DaemonSet
OpenSearch refuses to start when vm.max_map_count is below 262144 and
the store may use mmap. The kernel default is 65530 and nothing in the
platform's node configuration raises it: the Talos profiles declare no
sysctls, and neither does any documented node-preparation path.

The value cannot come from the OpenSearch pod. The kernel does not
namespace vm.*, so kubelet refuses it in a pod's securityContext.sysctls
and in --allowed-unsafe-sysctls alike, both of which admit namespaced
sysctls only. The operator's own answer, spec.general.setVMMaxMapCount,
appends a privileged init container to every OpenSearch pod. Tenant
namespaces carry no PodSecurity label of their own and take the cluster
default, which Talos ships as baseline through the apiserver's
PodSecurityConfiguration; that default rejects the container, so the
StatefulSet is created and never produces a pod.

Set the value once per node instead, from a DaemonSet in the operator's
own namespace. The init container raises the limit to a floor rather
than assigning it: the tunable is node-global, so a node already tuned
higher for another mmap-heavy workload keeps its own value. The pod runs
at system-node-critical: it is a precondition for every OpenSearch pod on
the node, so it must not lose a scheduling race to the workload it
enables, though priority is not start ordering: a pod that starts first
just fails its own bootstrap check and retries. The idle container runs
the sleep under a shell with a TERM handler, because PID 1 of a
namespace receives only the signals it installed a handler for and a
bare sleep would be SIGKILLed after the full grace period on every
rollout and drain. The write
goes to /proc/sys only, with no hostPath and no edit to the node's
on-disk configuration; the pod restarts with the node and reapplies the
value. The component is declared privileged so the platform labels that
namespace enforce=privileged, which is what this DaemonSet needs and
what tenant namespaces deliberately do not get.

The file returns to a path bb660b57c emptied. That earlier revision
bind-mounted the host's /etc to append to sysctl.conf and ran an
unpinned image; this one writes no host file and pins the image by
digest.

This removes the reason for an OpenSearch pod to ask the operator for
the sysctl, but the app chart still asks. Dropping that request is a
separate change and is what stops the pods being rejected.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-08 00:35:06 +03:00
Aleksei Sviridkin 59731429a8 fix(vm-instance): let complete resources replace the instance type
KubeVirt refuses a VirtualMachine that carries an instancetype matcher and
explicit domain sizing at once. validateCPU in pkg/instancetype/apply
conflicts on domain.cpu.cores and domain.cpu.sockets when they are
non-zero, and validateMemory conflicts on
domain.resources.requests.memory whenever the key is present. The chart
rendered both, because the matcher keyed off instanceType while the
domain sizing keyed off resources, and neither side looked at the other.

values.yaml ships instanceType "u1.medium", so clearing that field in the
dashboard only falls back to the default. A user who then fills in cpu,
sockets and memory gets a VM the webhook rejects, with an error naming
fields the form gives them no way to clear.

Resources that size the VM completely now replace the matcher, the same
way the kubernetes-nodes chart sizes its worker VMs. Beside an instance
type, sizing that is only partial stops the render, because there is then
no safe reading: dropping the matcher would size the VM from fields the
user never finished, and keeping it either loses to the webhook or
silently discards what was typed. With no instance type in play there is
nothing to conflict with, so that half is left as it renders today.

Completeness is decided from what domainResources emitted, not from which
keys the user set, and the two readings come apart. resources.cpu on its
own emits nothing, so it conflicts with nothing and goes on rendering a
plain instancetype-sized VM. cpu and sockets pass through int64, which
yields 0 for a quantity the schema accepts, so a cpu of 500m beside a
whole sockets and a memory would read as complete from the values, drop
the matcher, and hand KubeVirt a cores of 0 the user never asked for. And
a resources block nulled outright leaves .Values.resources nil, where a
raw field read is a render-time nil pointer and the emitted object is
merely empty.

The existence check and the update hook read the same effective value.
Keying the lookup on the raw one would fail a resource-sized render over
an instance type it never references, which is the same confusing error
this fixes. Reading .Values in the hook would see the leftover
instanceType, skip the removal patch, and leave
spec.instancetype.revisionName on a VM Helm had just given explicit
domain resources.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-08 00:26:21 +03:00
Alexey Artamonov 3a4bade677 docs(postgres): document the kind-wide scope of the client-side apply flip
The release.cozystack.io/helm-server-side-apply annotation moves every Postgres
HelmRelease to client-side apply, including releases that never enable
autoscaling. IvanHunters flagged that this platform-wide side effect was
unexplained and that non-autoscaled releases silently lose SSA drift
re-assertion (client-side diffs previous-vs-new render only, never live state;
driftDetection is configured nowhere in the repo).

Record why the scope is deliberate rather than narrowing it. helm-controller
v1.5.0 resolves an unset Upgrade.ServerSideApply through "auto" from the last
release's apply method, so gating client-side on "autoscaling active" would
force an SSA->client-side ownership handoff on the live spec.instances the
moment a tenant enables autoscaling - the exact live-field handoff the seed
design was chosen to avoid. Keeping the apply method stable for the kind's whole
lifecycle keeps enable/dry-run/transition/disable clean two-way merges with no
apply-method transition. The drift-re-assertion cost on never-autoscaled
releases is called out as an accepted trade-off, with a per-instance apply
strategy as the follow-up if it is ever needed.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 22:06:21 +03:00
Aleksei Sviridkin e104c39f55 docs(rabbitmq): say that deleting the application leaves the TLS Secrets behind
The README covered only the disable path. Deleting the application hits
the same cert-manager behaviour: the chart renders Certificate and Issuer
objects and never the Secrets, and cert-manager does not own what it
issued, so the CA and server private keys stay in the tenant namespace
with nothing referencing them. Name the two Secrets and the cleanup.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-07 21:59:35 +03:00
Alexey Artamonov 678d22f209 docs(postgres): move the serverAddress namespace-pin note out of a template comment
Closes IvanHunters' MINOR on scaledobject.yaml:97. The API-owner-gate marker was a Go
template comment, stripped at render, so the ownership question and the pin's residual
surface were recorded where nobody reads them again. State the constraint on the
serverAddress field doc (propagated into values.schema.json, README and the cozyrds
openAPISchema) including the accepted residual: tenant-root is always in the allow list
because the derived default points there, so a tenant can still aim the operator at a root
Service as a blind in-cluster probe (no response body reaches the tenant), which the egress
NetworkPolicy does not narrow. The API-owner sign-off is tracked on the PR, not in a
stripped comment.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 21:03:15 +03:00
Alexey Artamonov 1192922a88 docs(keda): document the teardown order for the ScaledObject finalizer
Closes IvanHunters' MINOR on keda/values.yaml:19. The resource-policy: keep comment
reasons about an admin removing the keda package but covers only the CRDs, not the
finalizer.keda.sh KEDA stamps on every ScaledObject: with the operator gone a tenant
deleting their Postgres app gets a ScaledObject stuck Terminating and a HelmRelease
uninstall that hangs, recoverable only by an admin finalizer patch. Note that autoscaling
must be disabled on every tenant Postgres before the package is removed, so the operator
clears the finalizers while it is still up.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 21:03:15 +03:00
Alexey Artamonov 07f1daa22d docs(postgres): correct the replicas "restored on disable" claim
Closes IvanHunters' MINOR on values.yaml:58. The replicas description (propagated into
values.schema.json, README and the cozyrds openAPISchema) said the count is "restored on
disable". Under client-side apply it is not: the two-way merge writes instances only when
the rendered value changes, so a cluster KEDA grew past replicas stays there after
autoscaling.enabled: false with no autoscaler left to reduce it — the sibling dryRun doc
already states the true rule, so the two contradicted each other. Reword both the @param
and the @typedef to say the live count is not auto-restored and that the operator lowers
replicas to reclaim the grown capacity; regenerated the derived artifacts.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 21:01:30 +03:00
Alexey Artamonov 7e664d269b test(postgres-rd): pin the client-side apply annotation on the cozyrds AppDefinition
Closes IvanHunters' MAJOR on cozyrds/postgres.yaml:12 (coverage half). Dropping the
release.cozystack.io/helm-server-side-apply annotation left the whole suite green: the
Go builder test pins the plumbing that consumes it but nothing pinned the annotation at
the point it is set, so the single load-bearing invariant of the feature — client-side
apply, without which server-side apply reverts KEDA's live /scale on every reconcile —
was unguarded. Add a helm-unittest that renders the ApplicationDefinition and asserts the
annotation, plus a test target so hack/helm-unit-tests.sh runs it; removing the annotation
now goes red.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 20:58:21 +03:00
Alexey Artamonov 568ccf4835 fix(keda): gate the operator CiliumNetworkPolicy on the cilium.io/v2 API
Closes IvanHunters' MAJOR on keda/values.yaml:60. The shim hardcodes
networkPolicy.flavor: cilium and the vendored templates gate only on
networkPolicy.enabled + flavor==cilium, never on the cilium.io/v2 CRD being served.
keda is offered on every variant from the shared system bundle, so on a variant that
ships no Cilium the CiliumNetworkPolicy is unmappable and the whole HelmRelease fails
to install. Add a .Capabilities.APIVersions.Has "cilium.io/v2" check to the three
CiliumNetworkPolicy templates so the package installs on any CNI; where Cilium is not
served the egress backstop is simply absent and the chart's serverAddress namespace-pin
remains the control. The vendored edit is captured as patches/ so make update re-applies
it instead of silently reverting.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 20:56:18 +03:00
Alexey Artamonov 3a09c3e9dc fix(postgres): guard dry-run/transition against the paused HPA min-clamp
Closes IvanHunters' MAJOR on scaledobject.yaml:39. The render guard rejected
replicas > maxReplicas but not the symmetric low side. Under dry-run/transition the
chart renders the static instances=replicas while the ScaledObject carries
minReplicaCount=effectiveMin with both scale directions paused. The pause disables
the HPA's scaling policies, not its min-replica clamp, so when replicas < effectiveMin
the HPA raises the live count to the floor on its first reconcile — a scale-up in a
mode documented as "actuates neither way" — and because the rendered instances value
never changes, the client-side two-way patch never pulls it back, leaving permanent
extra standbys and PVCs. Fail closed and tell the operator to stage replicas to the
floor first. Under dry-run/transition this floor guard is strictly broader than the
db.yaml quorum wedge (any replicas <= maxSyncReplicas is also < effectiveMin), so it
supersedes it there; the wedge stays exercised on the disabled path.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-07 20:48:11 +03:00
Andrei Kvapil 6384c8f4d1 feat(ingress-nginx): restrict the admission webhook to the API server
The admission webhook only ever needs to be reachable by the API server,
and leaving it open to every pod turns any flaw in it into something any
workload in any namespace can reach -- which is what made CVE-2025-1974
critical rather than merely serious.

The policy is deny-only and explicitly disables default-deny: a Cilium
rule carrying an ingress section would otherwise put the controller
behind one, meaning every port it serves would have to be enumerated and
the first one forgotten would break silently.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-09-07 17:03:44 +02:00