admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-11-04 11:07:03 +09:00
Code Issues Packages Projects Releases Wiki Activity
3,790 Commits 547 Branches 294 Tags
daniil/admin-gitops
Commit Graph
906 Commits
Author SHA1 Message Date
myasnikovdaniil e841688082 fix(postgres): accept integer values for postgresql.parameters in schema (#2687) 
## What this PR does

PostgreSQL server parameters like `max_connections` are natural
integers,
but the chart's schema declared them as `{type: string}` only. When
users
supply bare integers (e.g. `max_connections: 100`) — either directly in
Application CR specs or via the UI — Helm rejects the values at schema
validation time before template rendering, causing HelmRelease failures.

The Helm template already handles both types via `{{ $value | toString |
quote }}`,
so the fix is to teach the schema to accept both.

### Approach

The schema files (`values.schema.json` and the embedded `openAPISchema`
in `postgres-rd/cozyrds/postgres.yaml`) are generated from `values.yaml`
annotations via `cozyvalues-gen`. Hand-editing them does not survive
`make generate` — same regeneration trap that caught #2466 (issue
#2597).

`cozyvalues-gen` v1.4.0 (cozystack/cozyvalues-gen#25) adds an
`intOrString` type alias for map values that emits
`additionalProperties.anyOf: [integer, string]` plus the Kubernetes
`x-kubernetes-int-or-string: true` marker.

### Changes

- `packages/apps/postgres/values.yaml`: annotation flipped from
  `@field {map[string]string}` to `@field {map[string]intOrString}`,
  description updated to note integer values are accepted.
- Regenerated artifacts: `values.schema.json`, embedded `openAPISchema`
  in `packages/system/postgres-rd/cozyrds/postgres.yaml`,
`packages/apps/postgres/README.md`, and
`api/apps/v1alpha1/postgresql/types.go`
  (Parameters is now `map[string]intstr.IntOrString`).
- `packages/apps/postgres/templates/db.yaml`: `$dangerousParams` list
  extended with `archive_cleanup_command` and `recovery_end_command`
  (same RCE surface as `archive_command` / `restore_command`).

### Verification

- `helm template` with `max_connections: 200` (int) and
`max_connections: "200"`
  (string) produce identical render output.
- Dangerous-param block list rejects `archive_cleanup_command` and
  `recovery_end_command` with the expected `SECURITY:` error.
- `make test` in `packages/apps/postgres/`: 13/13 pass.
- `make generate` is idempotent — schema files don't drift on regen.

### Release note

```release-note
fix(postgres): accept integer values for postgresql.parameters to prevent HelmRelease failures when parameters are supplied as unquoted integers; also block archive_cleanup_command and recovery_end_command (RCE surface).
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* PostgreSQL parameters now accept integer or string values; integers
are automatically coerced to strings.

* **Security**
* Enhanced parameter filtering blocks additional dangerous settings:
archive_cleanup_command and recovery_end_command.

* **Documentation**
* Updated docs and charts to describe int-or-string parameter support
and revised blocked-parameter guidance.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2687?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-22 08:58:22 +05:00
Myasnikov DaniilandIvanHunters 14e196cf22 fix(postgres): drive postgresql.parameters int-or-string from the generator 
Ivan's original schema edits were correct in intent but lived in the
generated artifacts (values.schema.json and the embedded openAPISchema
in postgres-rd/cozyrds/postgres.yaml). `make generate` reverts them
because values.yaml's `@field {map[string]string}` annotation is the
source of truth -- same regeneration trap that caught #2466 (issue #2597).

cozyvalues-gen v1.4.0 (cozystack/cozyvalues-gen#25, merged 2026-05-21)
adds an `intOrString` type alias for map values that emits
`additionalProperties.anyOf: [integer, string]` plus
`x-kubernetes-int-or-string: true`. Move the change to source by
flipping the annotation, then regenerate:

- packages/apps/postgres/values.yaml: `@field {map[string]intOrString}`
  with updated description noting integer values are accepted.
- packages/apps/postgres/values.schema.json: regenerated -- now carries
  anyOf + x-kubernetes-int-or-string: true.
- packages/system/postgres-rd/cozyrds/postgres.yaml: embedded
  openAPISchema regenerated to match.
- packages/apps/postgres/README.md: regenerated type column.
- api/apps/v1alpha1/postgresql/types.go: Parameters is now
  `map[string]intstr.IntOrString`.

The result survives future regenerations. The template's
`{{ \$value | toString | quote }}` already coerces both inputs to
quoted strings, verified end-to-end with helm template against both
`max_connections: 100` (int) and `max_connections: "100"` (string) --
identical render output. Dangerous-param block list still rejects
archive_cleanup_command and recovery_end_command additions from the
prior commit.

Co-Authored-By: IvanHunters <xorokhotnikov@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-05-21 16:04:07 +05:00
IvanHunters 388e823916 fix(postgres): block archive_cleanup_command and recovery_end_command 
Add archive_cleanup_command and recovery_end_command to the dangerous
parameters blocklist in db.yaml. Both are PostgreSQL shell-exec hooks
identical in threat model to archive_command/restore_command and must
not be settable by tenants.

Also update the BLOCKED list in values.schema.json, values.yaml, and
the embedded openAPISchema in cozyrds/postgres.yaml to reflect the
two new blocked parameters.

Additionally update the embedded openAPISchema in cozyrds/postgres.yaml
to allow integer values for postgresql.parameters (anyOf string|integer),
matching the fix already applied to values.schema.json.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-05-21 16:04:07 +05:00
IvanHunters 8c89556e79 fix(postgres): accept integer values for postgresql.parameters in schema 
The template already coerces all parameter values to strings via
'toString | quote', but the JSON schema had 'additionalProperties: {type: string}'
which caused Helm schema validation to fail before the template ran
when users passed integer values (e.g. max_connections: 100).

Accept both string and integer types in additionalProperties so that
existing configurations with unquoted numbers continue to work.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-05-21 16:04:07 +05:00
IvanHunters ed1bb53d2c fix(platform): migrate ephemeralStorage to diskSize via pre-upgrade hook 
Add pre-upgrade migration 41 that walks all kuberneteses.apps.cozystack.io
Application CRs and renames nodeGroups[*].ephemeralStorage to diskSize,
preserving the user's value.

Without this migration, clusters upgraded after PR #2454 either fail to
reconcile (hard fail blocks all Flux operations) or silently lose the user's
disk size setting (default 20Gi replaces whatever was configured).

The migration is idempotent and best-effort: a failed patch is logged and
leaves the version stamp at 41 for retry on next upgrade.

Update the chart-side guard error message to direct operators to the
migration job logs when the field appears post-upgrade (regression detector).
Bump migrations.targetVersion 41 -> 42.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-05-21 12:14:55 +05:00
Arsolitt 69a479df5a fix(kafka): reclaim ZooKeeper PVCs on Kafka release deletion 
Set deleteClaim: true on the ZooKeeper persistent-claim storage so the
Strimzi operator removes the PVCs together with the rest of the cluster
when the Kafka CR is deleted. Previously the Kafka broker PVCs were
reclaimed (deleteClaim: true) but the ZooKeeper PVCs were orphaned,
forcing manual cleanup before the same release name could be reused.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-05-20 13:45:07 +03:00
Andrei Kvapil e84677782c revert(ingress)!: remove broken publishing.exposure=loadBalancer mode (#2603) 
## What this PR does

Reverts the `publishing.exposure=loadBalancer` mode for ingress-nginx
added in #2468.

The mode rendered the ingress-nginx Service as `type: LoadBalancer` and
a `CiliumLoadBalancerIPPool` over `publishing.externalIPs`, as a
migration path off `Service.spec.externalIPs` (deprecated upstream in
Kubernetes v1.36 per KEP-5707). In default Cozystack the implementation
produces two failure modes:

- `publishing.externalIPs` is the operator-supplied list of node public
IPs (addresses the cluster nodes already hold and listen on, see [docs
section "4b. Node Public IP
Setup"](https://cozystack.io/docs/v1.3/install/cozystack/platform/#4b-node-public-ip-setup)),
not a virtual-VIP pool. Treating it as one and DNATing those IPs via
Cilium LB-IPAM collides with the existing ingress-nginx DNAT on the same
IPs (`Service.spec.externalIPs` path), so traffic is forwarded
non-deterministically.
- Cilium in default Cozystack runs with `ipam.mode: kubernetes` and no
L2/BGP announcer, so a pool over addresses outside the node range is
allocated but never advertised — the Service stays unreachable.

The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side `IPPool` baked into the tenant chart. That follow-up belongs
in its own PR.

### Removed

- `publishing.exposure` value (and the inline comment) from
`core/platform/values.yaml`.
- `expose-mode` plumbing line in `core/platform/templates/apps.yaml`.
- `CiliumLoadBalancerIPPool` template at
`apps/tenant/templates/cilium-lb-pool.yaml`.
- `type: LoadBalancer + externalTrafficPolicy: Local` branch in
`extra/ingress/templates/nginx-ingress.yaml`. The Service now renders as
`ClusterIP + spec.externalIPs` when externalIPs are set, falling through
to `type: LoadBalancer` when they are not — pre-#2468 shape.
- Exposure mode section from `extra/ingress/README.md`.
- `exposure_test.yaml` fixtures in both charts.

### Compatibility

Operators who explicitly set `publishing.exposure=loadBalancer` will see
their ingress Service revert to `ClusterIP + spec.externalIPs` on
upgrade. Anyone on the default value (`externalIPs`) is unaffected.

### Release note

```release-note
revert(ingress): remove the publishing.exposure=loadBalancer mode added in #2468. The default externalIPs path is unchanged; operators who explicitly opted into loadBalancer will have their ingress Service revert to ClusterIP + spec.externalIPs on upgrade. Migration off Service.spec.externalIPs (deprecated in KEP-5707) will be reintroduced via a mechanism-agnostic API knob in a follow-up.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **Refactor**
* Removed conditional generation of the Cilium load‑balancer IP pool and
simplified service exposure to rely on explicit external IP values.
  * Streamlined configuration for service exposure keys.

* **Documentation**
  * Removed the exposure-mode guidance from ingress documentation.

* **Tests**
* Deleted the exposure-mode-related test suites covering various
external IP and loadBalancer scenarios.

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2603)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-19 10:47:45 +02:00
cozystack-ci[bot] 9f655e29ff Prepare release v1.4.0-rc.2 
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
 2026-05-13 21:48:39 +00:00
cozystack-ci[bot] 252c18f3e8 Prepare release v1.4.0-rc.1 
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
 2026-05-13 12:29:01 +00:00
Andrey Kolkov 92930ad1aa feat(platform): add implementation of backup-strategy controller for foundationdb 
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
 2026-05-13 09:49:19 +04:00
Andrey Kolkov 62605a198b feat(platform): implement backup strategy for mariadb 
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
 2026-05-12 13:56:29 +04:00
Aleksei Sviridkin 55a98b8815 docs(apps): point seven app README preset tables at canonical docs 
Address review feedback from coderabbitai across:
  - packages/apps/kubernetes/README.md
  - packages/apps/mariadb/README.md
  - packages/apps/nats/README.md
  - packages/apps/qdrant/README.md
  - packages/apps/redis/README.md
  - packages/apps/tcp-balancer/README.md
  - packages/apps/vpn/README.md

Each of these READMEs left an inline 7-row preset table listing only
the legacy flat names after the parameters block above already
defaulted to an instance-type value (e.g. `t1.nano`). cozyvalues-gen
only regenerates the `## Parameters` block, so the hand-written
section drifted.

Replace the inline tables with the same short paragraph already used
by postgres and kafka READMEs: name the five series, note that the
seven legacy names remain accepted as deprecated aliases, and link to
the canonical reference at docs/operations/resource-presets.md so the
full size matrix and legacy mapping live in exactly one place.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-12 03:08:52 +03:00
Aleksei SviridkinandMatthieu ROBIN cca54d876a docs(operations): resource presets reference 
Document the five instance-type series (t1/c1/s1/u1/m1) introduced
alongside the legacy preset deprecation, the per-size CPU and memory
matrix, and the 1:1 legacy-to-instance-type mapping table that powers
cozy-lib aliases, migration 39, and the operator-side deprecation
warning.

Single source of truth referenced from:
  - packages/library/cozy-lib/templates/_resourcepresets.tpl
  - packages/core/platform/images/migrations/migrations/39
  - pkg/apis/apps/presets/legacy.go

Includes an explicit callout that legacy `medium` (1 CPU / 1Gi) and
instance-type `*.medium` (2 CPU) are not the same — same word,
different resources.

Replace inline preset tables in packages/apps/postgres/README.md and
packages/apps/kafka/README.md with a short paragraph that names the
five series and points at the canonical doc, since the per-app
inline tables listed only the legacy seven and would drift further
as the canonical matrix evolves.

Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-11 21:10:19 +03:00
Aleksei SviridkinandMatthieu ROBIN 55850aa3d8 test(apps): backward-compat helm-unittest and CI invariants 
Pin the 1:1 equivalence between legacy flat preset names and their
instance-type counterparts in helm-unittest for three charts that
exercise different preset shapes:

  - postgres   — top-level only
  - clickhouse — top + nested clickhouseKeeper preset
  - opensearch — top + nested dashboards preset

Each case asserts that nano <-> t1.nano, small <-> t1.small,
medium <-> c1.small, large <-> c1.medium, and micro <-> t1.micro
render the same CPU and memory.

packages/apps/{clickhouse,opensearch}/Makefile gain a `test:` target
that calls helm unittest so hack/helm-unit-tests.sh actually runs the
new suites in CI; without it the suites would land as dead weight.

hack/check-rd-presets.sh enforces two invariants in `make unit-tests`:

  - every packages/system/*-rd/cozyrds/*.yaml openAPISchema carries
    the canonical 47-preset enum (40 instance-type + 7 legacy);
  - no chart template carries a hardcoded legacy preset literal in
    a defaultingSanitize (list "<legacy>" ...) form.

Both turn an accidental drop of a per-chart Makefile step or an
accidental legacy hardcode into a CI failure rather than silent
production-time validation rejection.

Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-11 21:09:50 +03:00
Aleksei SviridkinandMatthieu ROBIN fdffedbd4b feat(apps): instance-type resourcesPreset across apps and extras 
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:

  nano    -> t1.nano       micro   -> t1.micro
  small   -> t1.small      medium  -> c1.small
  large   -> c1.medium     xlarge  -> c1.large
  2xlarge -> c1.xlarge

Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.

Three hardcoded legacy literals in chart templates migrated:
  - packages/apps/harbor/templates/harbor.yaml:121 (portal)
  - packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
  - packages/system/keycloak/templates/sts.yaml:115 (heap fallback)

packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.

Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.

Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-11 21:08:41 +03:00
Andrey Kolkov 250e1b3376 feat(platform): implement backup strategy for clickhouse application 
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
 2026-05-11 14:46:48 +04:00
Andrei KvapilandClaude f919afe15f revert(ingress)!: remove broken publishing.exposure=loadBalancer mode 
Reverts the ingress loadBalancer exposure mode added in #2468.

The mode rendered the ingress-nginx Service as type: LoadBalancer and a
CiliumLoadBalancerIPPool over publishing.externalIPs, as a migration path
off Service.spec.externalIPs (deprecated upstream in Kubernetes v1.36 per
KEP-5707). In default Cozystack the implementation produces two failure
modes:

- publishing.externalIPs is the operator-supplied list of node public IPs
  (addresses the cluster nodes already hold and listen on), not a
  virtual-VIP pool. Treating it as one and DNATing those IPs via Cilium
  LB-IPAM collides with the existing ingress-nginx DNAT on the same IPs
  (Service.spec.externalIPs path), so traffic is forwarded
  non-deterministically.
- Cilium in default Cozystack runs with ipam.mode: kubernetes and no
  L2/BGP announcer, so a pool over addresses outside the node range is
  allocated but never advertised — the Service stays unreachable.

The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side IPPool baked into the tenant chart. That follow-up belongs in
its own PR.

Removed:
- publishing.exposure value (and the inline comment) from
  core/platform/values.yaml
- expose-mode plumbing line in core/platform/templates/apps.yaml
- CiliumLoadBalancerIPPool template at
  apps/tenant/templates/cilium-lb-pool.yaml
- type: LoadBalancer + externalTrafficPolicy: Local branch in
  extra/ingress/templates/nginx-ingress.yaml (Service renders back as
  ClusterIP + spec.externalIPs when externalIPs are set, falling through
  to type: LoadBalancer when they are not — pre-#2468 shape)
- Exposure mode section from extra/ingress/README.md
- exposure_test.yaml fixtures in both charts

Operators who explicitly set publishing.exposure=loadBalancer will see
their ingress Service revert to ClusterIP + spec.externalIPs on upgrade.
Anyone on the default value (externalIPs) is unaffected.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
 2026-05-11 10:42:29 +02:00
Aleksei Sviridkin 6f83d9a794 feat(ouroboros): bump to v0.7.2 with debug logging on tenants 
The v0.7.2 ouroboros release adds --log-level / OUROBOROS_CONTROLLER_LOG_LEVEL,
plants Debug logs at every informer event handler (AddFunc/UpdateFunc/
DeleteFunc with kind+namespace/name via describeObj) and at reconcile
entry/exit, wires SetWatchErrorHandler on Ingress / Gateway / HTTPRoute
informers, and surfaces cache-sync elapsed at Info. The diagnostic
surface targets the kamaji-tenant-via-konnectivity flake where bats
sees a fresh Ingress create succeed against the tenant apiserver but
the controller's coredns-custom rewrite lands roughly 7-10 minutes
later — without per-event visibility the failure mode is
indistinguishable between 'watch never fires', 'reconcile silently
errors', and 'API list returns empty until reflector reconnect'.

Bumps the chart-manifest digest, the controller-image digest, and the
cozystack-side image-pin assertion in lockstep. The bats deadline for
the coredns-custom rewrite check rises from 180s to 900s — even on
the broken-watch path the reflector eventually reconnects within
that budget, so the test stops failing on the symptom while we keep
the diagnostic on to learn what the symptom actually is.

cozystack tenants force controller.logLevel: debug via the
defaultOuroborosValues helper; operators that want quieter logs can
override through addons.ouroboros.valuesOverride. Two helm-unittest
fixtures pin both the default and the override path.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-11 02:20:34 +03:00
Aleksei Sviridkin d6c24dd078 fix(ouroboros): pin tenant clusterDomain to cluster.local 
The cozystack tenant ouroboros wrapper deliberately left
`controller.clusterDomain` empty so the upstream chart's per-pod
`/etc/resolv.conf` auto-detect could pick up whatever the tenant CoreDNS
actually serves. In a cozystack tenant that auto-detect resolves to the
host platform's `cozy.local` rather than the tenant's `cluster.local`:
the cozystack tenant kubelet injects the host's cluster-domain into pod
search lists, but Kamaji's TenantControlPlane is configured with
`networkProfile.clusterDomain: cluster.local` and the tenant CoreDNS
Corefile serves only that domain. The mismatch makes the proxy compose
its `BackendHost` as
`ingress-nginx-controller.cozy-ingress-nginx.svc.cozy.local`, which
NXDOMAINs against tenant CoreDNS — `/readyz` returns 503 forever, the
proxy Deployment never reaches Ready, helm install times out at 10
minutes, and the e2e job hits the 2h cancel without `ouroboros.bats`
ever progressing past the first kubectl wait.

Pin `controller.clusterDomain: cluster.local` in `defaultOuroborosValues`
so the chart bakes the resolved BackendHost against the domain tenant
CoreDNS actually serves. This is the contract Kamaji TCP defines for
every cozystack tenant; operators on a non-default tenant CoreDNS can
still override via
`addons.ouroboros.valuesOverride.ouroboros.controller.clusterDomain`.

The underlying tenant kubelet injecting the wrong cluster-domain into
pod resolv.conf is a separate bootstrap concern that should be addressed
on its own track; pinning the chart input here keeps ouroboros
operational on the default cozystack tenant topology in the meantime.

Update the corresponding helm-unittest to assert the new contract.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-08 10:49:51 +03:00
Andrei Kvapil 45e21a330f feat(networking): add ouroboros for hairpin-NAT under PROXY-protocol (#2561) 
## What this PR does

Adds
[`lexfrei/ouroboros@v0.7.0`](https://github.com/lexfrei/ouroboros/releases/tag/v0.7.0)
— a Go reimplementation of
[`compumike/hairpin-proxy`](https://github.com/compumike/hairpin-proxy)
— and wires it into cozystack as a *conditional default*: a single
`publishing.proxyProtocol: true` switch turns PROXY-protocol on at the
host ingress-nginx **and** auto-deploys ouroboros to fix the resulting
hairpin-NAT problem (intra-cluster traffic to the cluster's own public
hostnames arriving at ingress-nginx without the PROXY header it now
requires). Default stays `false` — clusters that do not run
PROXY-protocol get zero new resources, zero RBAC delta, zero behaviour
change.

The chart is platform-agnostic at v0.7.0: the controller and proxy
binaries auto-detect the cluster's DNS domain from `/etc/resolv.conf`
inside the pod and compose the rewrite target FQDN and the backend host
at runtime as `<svc>.<ns>.svc.<cluster-domain>`. Cozystack tenants on
`cozy.local`, kubeadm-bootstrapped clusters on `cluster.local`,
federations on `k8s.example.com`, custom-domain RKE2/k3s — all work
without operator override. Verified end-to-end on a `cozy.local`
cluster: controller logs `cluster-domain=cozy.local` at startup, chart
emits `--proxy-service-name + --proxy-service-namespace` (no chart-baked
`--proxy-fqdn`), Corefile picks up the rewrite block between BEGIN/END
markers, in-pod `dig <ingress-host>` returns the ouroboros-proxy
ClusterIP. Operators that prefer a chart-baked FQDN can still set
`controller.clusterDomain` explicitly — the explicit-override path is
preserved for non-Service backends and for split-horizon DNS topologies.

The chart runs in `coredns` mode for the host (the host CoreDNS is
Talos-managed and does not carry an `import` directive, so the
alternative would silently no-op there) and in `coredns-import` mode for
tenant deployments (tenants run cozystack-coredns, which carries the
import directive and ships an empty `kube-system/coredns-custom`
ConfigMap; ouroboros writes plugin-only `rewrite name` snippets into the
`ouroboros.override` data key without ever touching the chart-rendered
Corefile, so it does not race the chart that re-templates
`data.Corefile` on every Flux reconcile).

### Layout

* `packages/system/ouroboros/` — vendored chart from
`oci://ghcr.io/lexfrei/charts/ouroboros@0.7.0`. Cozystack-side defaults
pinned in `values.yaml`: `controller.mode=coredns` (host scenario),
`proxy.target.namespace=cozy-ingress-nginx` (override the chart-default
`ingress-nginx` namespace; chart-default
`serviceName=ingress-nginx-controller` matches as-is), Gateway-API
watching left at upstream's default of `false`. `proxy.target.host` is
left empty so the proxy composes the backend host at runtime. Image is
digest-pinned at
`0.7.0@sha256:478665e05dd0ffc4c1f7764320ea24b52251781884ddfd668d93a03e39d9094c`
and pulled directly from `ghcr.io/lexfrei/ouroboros` — this package is
intentionally not mirrored under `ghcr.io/cozystack/*`. The chart
manifest itself is also digest-pinned via
`OUROBOROS_CHART_DIGEST=sha256:f28bfac9fd7070b1f3357983ff5aad28c16115e7de1cfc16ee568a3b4cfc9d7e`
in the package Makefile, with a Chart.yaml-version vs. Makefile-version
assert that aborts the update target on lockstep drift. Helm-unittest in
`tests/` covers controller args, RBAC narrowing per mode, Gateway-API
gating, runtime-composition flag emission in both `coredns` and
`coredns-import` modes, image pin drift.
* `packages/system/coredns/templates/coredns-custom.yaml` +
`values.yaml` — empty `coredns-custom` ConfigMap as the chart-managed
`import` prerequisite, plus the `import /etc/coredns/custom/*.override`
plugin in the `.:53` server block + `extraVolumes`/`extraVolumeMounts`
mount at `/etc/coredns/custom/`. Same package serves host and tenant
CoreDNS, so this covers both layers without duplication. Helm-unittest
pins the rendered Corefile via a snapshot so future `make update` does
not silently drop the import directive. The empty rendered ConfigMap
deliberately omits `data:` — Helm three-way merge has no template-side
keys to compare against, so apiserver-side runtime writes (the
`ouroboros.override` data key) survive every chart upgrade. The
`notExists: data` assertion in `tests/coredns_custom_test.yaml` pins the
invariant.
* `packages/core/platform/sources/ouroboros.yaml` — new
`cozystack.ouroboros` PackageSource for the host install (`dependsOn:
cozystack.networking, cozystack.ingress-application`).
* `packages/core/platform/sources/kubernetes-application.yaml` —
`kubernetes-ouroboros` component registered against `system/ouroboros`
so the tenant HelmRelease can reference it.
* `packages/apps/kubernetes/templates/helmreleases/ouroboros.yaml` —
tenant-side HelmRelease deployed via the tenant admin-kubeconfig,
mirroring the cert-manager pattern. A `cozystack.defaultOuroborosValues`
helper flips `mode` back to `coredns-import` for tenants and mirrors
per-tenant `addons.gatewayAPI.enabled` into the controller (forcing
`--gateway-api` on a tenant without Gateway-API CRDs CrashLoops the
controller on `WaitForCacheSync`). `controller.clusterDomain` is
intentionally NOT pre-set — chart 0.7.0 auto-detects per-pod from
`/etc/resolv.conf` and composes the rewrite target at runtime, so the
same wrapper renders correctly on `cluster.local`, `cozy.local`, or any
custom tenant cluster-domain. Operators on a non-default cluster-domain
can still override via
`valuesOverride.ouroboros.controller.clusterDomain`. `dependsOn` grows a
conditional `gateway-api-crds` entry on the same gate. If
`addons.ouroboros.enabled` is true while `addons.ingressNginx.enabled`
is false, the HelmRelease fails the render with a clear error rather
than silently skipping — the hairpin fix has nothing to fix against
without ingress-nginx, and a no-op addon that quietly does nothing is
harder to debug than an explicit render-time error. The valuesOverride
gateway-api guard accepts every truthy spelling Helm/YAML emit (bool
`true`, string `"true"`/`"True"`/`"TRUE"`, numeric `1`, YAML
`yes`/`Yes`/`YES`/`on`/`On`/`ON`/`y`/`Y`) via
`toString`-then-`lower`-then-string-compare; direct bool `eq` would
error on type mismatch when an operator passes the value as a string
from `--set`, and case-insensitive folding catches `--set ...=On` /
`--set ...=True` that pure-lowercase comparison would miss. Tenant
disable is single-step: flipping `addons.ouroboros.enabled: false`
deletes the rendered HelmRelease, helm-controller runs `helm uninstall`
on the tenant, and the chart's pre-delete cleanup hook
(`coredns-cleanup-hook.yaml`, vendored at 0.7.0) nulls the
`ouroboros.override` key on the tenant's `kube-system/coredns-custom`
ConfigMap automatically. No render-time guard is wired on the tenant
side — gating on a `lookup` for the leftover HelmRelease would deadlock
the routine disable (the parent render runs before helm-controller
applies the missing-child diff, so the lookup would always find the
leftover at the moment of the flag flip). If the chart's hook fails to
land (controller pod stuck CrashLooping, ConfigMap RBAC drift, Job
timeout), operators recover manually via the documented tenant cleanup
recipe.
* `packages/apps/kubernetes/values.yaml` — new `OuroborosAddon` typedef
with `enabled` + `valuesOverride`. `make generate` re-emitted README,
JSON schema, the Go struct + deepcopy, and the kubernetes-rd
ApplicationDefinition openAPISchema in lockstep.
* `packages/core/platform/values.yaml` — new `publishing.proxyProtocol`
flag with extensive comment covering the upstream-LB precondition (the
L4 LB in front of ingress-nginx must already be injecting PROXY-v1
headers before the flag flips on, otherwise external traffic breaks),
the host-vs-tenant fan-out (the flag does not cascade into tenants), the
Talos re-render flap window, and the asymmetric disable path. The host
disable path: flipping back to `false` reverts ingress-nginx
PROXY-protocol config (flip the LB off first), but the host ouroboros
itself stays installed because its Package CR carries
`helm.sh/resource-policy: keep` — an explicit `kubectl delete
package.cozystack.io cozystack.ouroboros` (or `bundles.disabledPackages`
entry) actually triggers helm uninstall, at which point the chart's
pre-delete cleanup hook fires and patches the host Corefile
automatically. The values comment ships a manual `sed`-based cleanup
recipe as a fallback for the rare case where the chart's hook fails to
land (controller stuck in CrashLoop, ConfigMap RBAC drift, hook
timeout); the `publishing.proxyProtocolAcknowledgeUnclean` gate keys off
a `lookup` against the leftover Package CR to catch operators who flip
the flag back without confirming the cleanup landed.
* `packages/core/platform/templates/bundles/system.yaml` — flips between
the default-form and explicit-form `cozystack.ingress-application`
Package emission so the host ingress-nginx gets `use-proxy-protocol` +
`real-ip-header=proxy_protocol` injected only when the flag is true
(`use-forwarded-headers` and `compute-full-forwarded-for` are
deliberately *not* on the list — they would let any upstream proxy spoof
`X-Forwarded-For`); the same conditional emits `cozystack.ouroboros`.
Host Gateway-API watching is left at the upstream chart's default of
`false` because cozystack does not install Gateway-API CRDs cluster-wide
on the host.
* `hack/e2e-apps/ouroboros.bats` — auto-discovered smoke test for the
tenant addon (HelmRelease ready, `coredns-custom` ConfigMap present in
tenant, controller pod running, and after creating a TLS Ingress the
rewrite snippet shows up in the import ConfigMap; an in-tenant `dig`
from a netshoot pod also confirms CoreDNS actually serves the rewrite —
pod-readiness alone passes vacuously since the upstream chart ships no
readiness probes).

### Why not KEP-1860 `ipMode=Proxy`

Cilium's `kubeProxyReplacement: true` (cozystack default) drops the LB
frontend entirely when `ipMode != VIP`, breaking Service ingress for
L2/BGP-announced IPs. KEP-1860's Proxy mode is reserved for clusters
fronted by an external CCM-managed proxy LB. ouroboros is the correct
fix for L2/BGP-announce topologies that cozystack ships by default.

### Coverage

Covered by helm-unittest (host chart, coredns wrapper, tenant addon —
including the `kubernetesProvider` mock that exercises the lookup-driven
`proxyProtocolAcknowledgeUnclean` gate, the lookup-returns-nil
CI/dry-run path, the gateway-api truthy-spelling acceptance across
`true`/`"true"`/`"True"`/`"TRUE"`/`1`/`yes`/`on`, the assertion that
`--proxy-service-name` + `--proxy-service-namespace` are emitted instead
of a chart-baked `--proxy-fqdn` in both `coredns` and `coredns-import`
modes, the assertion that `--target-service-name` +
`--target-service-namespace` are emitted for the proxy backend, and the
`_namespace.etcd`-empty render gate) and a tenant e2e bats test that
creates a real Ingress, asserts the rewrite snippet lands in the import
ConfigMap, and confirms in-tenant DNS serves it (with an in-pod retry
loop tolerating CoreDNS reload latency). The host-level
`publishing.proxyProtocol: true` path is not e2e-tested in this PR — it
requires a PROXY-protocol-aware upstream LB, which the cozystack e2e
harness does not stand up. That path is exercised manually and by docs
review.

End-to-end runtime auto-detect was verified manually on a `cozy.local`
cluster (dev17): chart deployed at v0.7.0, controller logged
`cluster-domain=cozy.local`, the rewrite block landed in
`kube-system/coredns` Corefile pointing at
`test-ouroboros-proxy.test-ouroboros.svc.cozy.local.`, an in-pod `dig
<ingress-host>` resolved to the ouroboros-proxy ClusterIP on the first
attempt, and the chart's pre-delete hook restored the Corefile cleanly
on `helm uninstall`.

### Supply-chain note

Image and chart are pulled from a contributor's personal namespace
(`ghcr.io/lexfrei/...`). Existing precedent
(`packages/system/etcd-operator/Makefile` pulls from
`ghcr.io/aenix-io/...`) is similar but `aenix-io` is the
cozystack-adjacent org while `lexfrei` is one human. Worth a maintainer
call.

Pairs with cozystack/website#527 (operator docs).

### Release note

```release-note
feat(networking): add `publishing.proxyProtocol: true` to enable PROXY-protocol on host ingress-nginx and auto-deploy ouroboros (a Go reimplementation of compumike/hairpin-proxy) to solve the resulting hairpin-NAT problem; tenants can opt in per-cluster via `addons.ouroboros.enabled` (requires `addons.ingressNginx.enabled`). Default behaviour unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Ouroboros addon to address hairpin‑NAT for ingress‑nginx with
configurable enablement and tenant overrides.
* Added a packaged Ouroboros chart providing proxy, controller, CoreDNS
integration, external‑DNS and etc‑hosts modes.
* Introduced PROXY‑protocol gating with an explicit acknowledgement
flag.

* **Documentation**
* Expanded addon reference and platform docs with detailed
Ouroboros/PROXY‑protocol configuration and defaults.

* **Tests**
* Added extensive unit and e2e tests covering rendering, RBAC, uninstall
cleanup, and DNS behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-07 18:41:13 +02:00
Andrey Kolkov 8da1e7a40a feat(platform): implement backup strategy for postgres (#2552) 
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
Implemented CNPG-strategy controller for postgres backups with supporting in-place and to-copy restores.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* CNPG-based backup strategy for PostgreSQL with S3-backed object store
and controller support
* Option to reference existing Kubernetes Secret for S3 credentials
(configurable key names)
* Controller supports scheduled/ad-hoc backups and both in-place and
to-copy restores; safer credential handling (no inline keys)

* **Examples**
* Complete manifests: bucket, credentials, CNPG strategy, BackupClass,
Plan, BackupJob, RestoreJob, Postgres source/target

* **Security/RBAC**
* Controller RBAC expanded for CNPG resources; tests enforce no
cluster-wide Secret read

* **Documentation**
* README, chart values/schema and UI schema updated to document
secret-reference workflow and restore guidance

* **Tests**
* New end-to-end and unit tests covering CNPG flows, restore semantics,
cleanup behavior and RBAC constraints
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-07 18:32:07 +04:00
Andrey Kolkov ed5ae16e36 feat(platform): implement backup strategy for postgres application 
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
 2026-05-07 13:12:54 +04:00
IvanHunters 418e3da527 fix(kafka): create LoadBalancer service when external flag is enabled 
When `external: true` was set, the Kafka chart relied on Strimzi to
create a LoadBalancer service via listener configuration, but the
listener was unconditionally defined (as `type: internal` when disabled).
Additionally, the dashboard RBAC did not include the Strimzi-managed
external bootstrap service, making it invisible to tenant users.

- Make the external listener conditional: only add it to the Kafka CR
  when `external: true` is set
- Add `<release>-kafka-external-bootstrap` to dashboard RBAC rules
  when `external: true`, so the LB service is accessible from the
  tenant dashboard
- Add helm-unittest tests for both kafka listeners and dashboard RBAC
  covering the external/internal paths

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-05-06 12:17:30 +03:00
Aleksei Sviridkin 7ab940f309 feat(kubernetes): expose ouroboros as a per-tenant addon 
Adds the OuroborosAddon typedef, addons.ouroboros knob, and tenant
HelmRelease that deploys the cozy-ouroboros chart inside a tenant
Kubernetes cluster via the host's flux-helm-controller and the tenant
admin-kubeconfig — same shape as the cert-manager addon.

The HelmRelease overrides controller.mode back to coredns-import
because tenants run the cozystack-coredns wrapper (which carries the
`import /etc/coredns/custom/*.override` directive); the parent chart's
default of mode=coredns is the right shape only for the Talos-managed
host CoreDNS. controller.gatewayApi.enabled is mirrored from the
per-tenant addons.gatewayAPI.enabled — forcing --gateway-api on a
tenant without Gateway-API CRDs CrashLoops the controller on
WaitForCacheSync against missing GVRs. dependsOn grows a conditional
gateway-api-crds entry on the same gate.

controller.clusterDomain is intentionally NOT pre-set by the wrapper.
The chart auto-detects it inside the controller pod by parsing
/etc/resolv.conf — the search list of which is `*.<ns>.svc.<domain>`
and reflects exactly the zone the tenant CoreDNS actually serves
(governed by KamajiControlPlane.spec.networkProfile.clusterDomain
defaulting to `cluster.local`, and the cozystack-coredns wrapper
kubernetes-plugin zone, also `cluster.local`). Pre-setting from a
host-side fact like `_cluster.cluster-domain` would override the
auto-detect with a value that does not match what the tenant kube-dns
serves, silently emitting NXDOMAIN-bound rewrites. Operators that run
a tenant on a non-default cluster-domain can still override via
addons.ouroboros.valuesOverride.ouroboros.controller.clusterDomain.

If addons.ouroboros.enabled is true but ingressNginx is off, the
HelmRelease fails the render with a clear error rather than silently
skipping — the hairpin fix has nothing to fix against without
ingress-nginx, and a no-op addon that quietly does nothing is harder
to debug than an explicit render-time error.

The valuesOverride gateway-api guard accepts every truthy spelling
Helm/YAML emit (bool true, string "true"/"True"/"TRUE", numeric 1,
YAML "yes"/"Yes"/"YES"/"on"/"On"/"ON"/"y"/"Y") via
toString-then-lower-then-string-compare; direct bool eq would error
on type mismatch when an operator passes the value as a string from
--set, and case-insensitive folding catches `--set ...=On` /
`--set ...=True` that pure-lowercase comparison would miss.

The acknowledgeUncleanRemoval gate is defence-in-depth for the rare
edge case where helm uninstall fails to land cleanly (tenant
unreachable, hook timeout, manual kubectl delete bypassing helm) —
in the normal path the upstream chart's pre-delete cleanup hook
(coredns-cleanup-hook.yaml, vendored at 0.6.0) nulls the
ouroboros.override key automatically when the HelmRelease is
uninstalled. The gate only blocks render when a leftover HelmRelease
on the host kube-apiserver shows the previous install is still
sitting on the cluster.

The component is also registered in the kubernetes-application
PackageSource (as kubernetes-ouroboros at path system/ouroboros) so
the tenant HelmRelease's ExternalArtifact resolves.

make generate re-emitted README.md, values.schema.json, the Go struct
+ deepcopy, and the kubernetes-rd ApplicationDefinition openAPISchema
in lockstep.

helm-unittest covers: render gating on enabled flag, hard-fail when
ingress-nginx is off, namespaces and labels, chartRef, dependsOn
cilium/coredns/ingress-nginx + conditional gateway-api-crds, default
coredns-import mode, valuesOverride precedence, gateway-api flag
mirroring in both directions, gateway-api truthy-spelling acceptance
(including case-insensitive on/On/True/TRUE), the lookup-driven
acknowledge gate (kubernetesProvider mock asserts the failedTemplate
path when a prior HelmRelease still exists on the host), the
lookup-returns-nil CI/dry-run path (render proceeds when the cluster
mock has no objects), the _namespace.etcd-empty render gate, and
that controller.clusterDomain is NOT pre-set in the rendered
HelmRelease (so chart auto-detect runs).

Signed-off-by: Aleksei Sviridkin <f@lex.la>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-05-06 06:48:06 +03:00
Andrei Kvapil 90b3719f6a fix(mongodb): expose replica-set members per pod for external access (#2514) (#2522) 
## What this PR does

Fixes #2514.

When `external: true` is set on a non-sharded `MongoDB`, Cozystack
rendered a single
`LoadBalancer` Service whose selector matched **all** replica-set
members
(`app.kubernetes.io/replset: rs0`). Round-robin balancing sent ~⅔ of
writes to
secondaries and they were rejected with `MongoServerError: not primary`.

Drivers couldn't recover on their own either: connecting through the LB
returned
the in-cluster DNS names of all members from `isMaster`, none of which
resolve
from outside the cluster, so automatic primary discovery failed.

This PR replaces the manual `external-svc.yaml` (for replica-set mode)
with the
Percona MongoDB operator's native `replsets[].expose` feature:

```yaml
replsets:
  - name: rs0
    expose:
      enabled: true
      type: LoadBalancer
      externalTrafficPolicy: Local
```

The operator then:
- creates one `LoadBalancer` Service **per pod** (`<release>-rs0-0`,
`-1`, `-2`)
  with `statefulset.kubernetes.io/pod-name` selectors,
- watches `Service.status.loadBalancer.ingress` for each pod and
rewrites the
  member `host` in `rs.conf` to the external IPs.

External clients use a standard replica-set URI
(`mongodb://...:27017,...:27017,...:27017/?replicaSet=rs0`) and the
driver
discovers the primary correctly from outside the cluster.

For **sharded** mode the existing single `LoadBalancer` is preserved —
it
front-ends stateless `mongos`, which can route writes to the right shard
primary itself. `external-svc.yaml` is now scoped to that case only.

### Tested

- `helm unittest .` in `packages/apps/mongodb/`: 91/91 pass.
- Applied on a dev stand:
  - `replsets[0].expose` is set as expected in the rendered PSMDB CR.
- The Percona operator (`v1.21.1`) accepts the spec and creates 3
per-pod
    `LoadBalancer` Services with `statefulset.kubernetes.io/pod-name`
selectors. The previous single round-robin Service is no longer
rendered.
- With an LB IP pool present, the operator promotes the allocated LB IPs
into
    the PSMDB `status.host` field and into `rs.conf`.

End-to-end mongosh write loop wasn't run on the dev stand because the
underlying network does not advertise/route allocated LoadBalancer IPs
(no
working L2/BGP path), so the rewritten `rs.conf` members are unreachable
in
that environment. The structural change is otherwise fully verified and
the
issue's reproduction scenario expects a real LB-capable network.

### Release note

```release-note
fix(mongodb): expose each replica-set member through its own LoadBalancer when `external: true`, so external clients can discover the primary via the standard replica-set connection string and writes no longer fail with "not primary" errors.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* External LoadBalancer services now only render when both external and
sharding are enabled.
* Replica set external exposure is now conditional based on the external
setting.
* Service configuration now uses LoadBalancer type with Local traffic
policy for external access.
* Service selectors refined for proper mongos targeting in sharded
deployments.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-04 20:40:58 +02:00
Andrei Kvapil 0b70197bf1 fix(kafka): bump default resourcesPreset to medium (#2537) 
## What this PR does

Changes the default `resourcesPreset` for both `kafka` and `zookeeper`
in the Kafka managed application from `small` (1 CPU / 512Mi) to
`medium` (1 CPU / 1Gi).

The `small` preset is no longer sufficient for the current Strimzi 0.45
/ Kafka 3.9 image baseline:

- Strimzi sets JVM heap to 50% of the container memory limit for Kafka
and 75% for ZooKeeper. With a 512Mi limit that leaves only 256Mi (Kafka)
or 128Mi (ZooKeeper) for the JVM non-heap.
- Actual non-heap usage in the Kafka 3.9 image (Metaspace, code cache,
direct buffers, JMX javaagent, embedded Jetty admin server for
`/v1/ready` and KRaft migration endpoints, thread stacks) systematically
exceeds those budgets at startup, triggering a cgroup OOM kill (exit
137) before the broker can connect to ZooKeeper.

Result on a fresh deployment with the previous default: every Kafka
broker and every ZooKeeper pod ends up in `CrashLoopBackOff`, with
`lastState.terminated.reason: OOMKilled`. ZooKeeper occasionally hits
`OOMKilled` once quorum traffic ramps up, breaking the quorum and
causing Kafka brokers to fail with `ZooKeeperClientTimeoutException`.

The `medium` preset (1Gi) is the smallest preset that gives both Kafka
(heap 512Mi + ~500Mi non-heap budget) and ZooKeeper (heap 768Mi + 256Mi
non-heap budget) enough headroom to start cleanly. Users who need more
can still pick `large`, `xlarge`, or `2xlarge`, or override `resources`
explicitly.

The `small` preset is intentionally kept in the enum so existing
deployments that pin `resourcesPreset: small` are not broken; only the
default changes.

### Release note

```release-note
fix(kafka): default `resourcesPreset` for both Kafka and ZooKeeper is now `medium` (1 CPU / 1Gi). The previous default (`small`, 512Mi) is no longer sufficient under the current Strimzi 0.45 / Kafka 3.9 image and caused OOMKilled CrashLoopBackOff on fresh deployments. Existing deployments that explicitly set `resourcesPreset` are unaffected.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated default resource allocation preset from "small" to "medium"
for Kafka and ZooKeeper components across CRD type definitions, Helm
chart values, and JSON schema configurations.

* **Documentation**
* Updated parameter reference tables to reflect the new default resource
preset values for both Kafka and ZooKeeper.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-04 20:40:13 +02:00
myasnikovdaniil 8fbbef3f02 fix(harbor): drive bucket-secret.yaml from values, gate HR on BucketInfo (#2528) 
## Summary
- `cozy-harbor/templates/bucket-secret.yaml` no longer relies on
`lookup` against the BucketAccess credentials Secret. It is now driven
by `.Values.bucket.bucketInfo` (a JSON string) with `dig`-based safe
accessors, so a missing/empty/partial value renders nothing rather than
crashing with `index of untyped nil` (the failure mode that broke harbor
in CI #25049445125).
- The downstream `<release>-system` HelmRelease now sources `BucketInfo`
via `valuesFrom` (`valuesKey: BucketInfo`, `targetPath:
bucket.bucketInfo`). With the default `optional: false`, helm-controller
refuses to compose values until the COSI BucketAccess controller has
populated the Secret — both gating initial reconciliation and forcing a
config-digest change (and thus a helm upgrade) when the credentials
arrive. Flux HR `dependsOn` cannot reference COSI resources directly, so
`valuesFrom` is the correct primitive here.
- `bucket.secretName` was the only consumer of the old `lookup` and is
removed from both the system chart's `values.yaml` default and the apps
chart's `values:` block.

## Why this shape
helm-controller's upgrade trigger is digest-based over composed values +
chart artifact; a `lookup` returning new data on a later reconcile is
not enough to force an upgrade on its own. Routing `BucketInfo` through
`valuesFrom` puts the bucket data on the digest path, so the chart
re-applies the moment credentials become available.

## Test plan
- [x] `helm template` of `packages/system/harbor` with
`bucket.bucketInfo` unset, empty string, empty JSON object `{}`, and
fully populated — only the populated case renders the `*-registry-s3`
Secret; the others render nothing without erroring.
- [x] `helm template` of `packages/apps/harbor` produces the expected
`<release>-system` HR with the new `valuesFrom` entry and the cleaned-up
`values:` block.
- [ ] Re-run the harbor E2E on a fresh stand and confirm 15/15.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Harbor bucket config now comes from a centralized Secret merged into
chart values; in-values bucket entry removed.
* **Bug Fixes**
* Deployment synchronization and Helm upgrade behavior improved by
depending on the Secret’s readiness/contents.
* **Tests**
* Added unit tests covering when the bucket secret is rendered (or not)
and a make target to run those tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-04 17:21:17 +05:00
Aleksei Sviridkin 7eaafe0b40 fix(mongodb): reclaim PVCs and operator secrets on uninstall (#2525) 
## What this PR does

Adds the upstream Percona finalizer pair to the `PerconaServerMongoDB`
custom resource so the operator performs an ordered shutdown and
reclaims persistent storage when a MongoDB release is uninstalled.
Previously, deleting a MongoDB application left PVCs and
operator-managed secrets orphaned in the tenant namespace, requiring
manual cleanup before the same release name could be reused.

The finalizers added on the CR:

- `percona.com/delete-psmdb-pods-in-order` — ensures the primary is shut
down last for graceful cluster termination.
- `percona.com/delete-psmdb-pvc` — instructs the operator to delete PVCs
and operator-managed secrets when the CR is removed.

Coverage applies to all replica sets in both replicaset and sharded
modes, including the config server in sharded deployments.

### Behavior change

Users who previously relied on orphaned PVCs as an implicit "soft
delete" for data recovery should be aware that uninstalling a MongoDB
release now removes the underlying storage and operator-managed
credentials. The README's new "Data lifecycle" section enumerates
exactly which resources are reclaimed by the operator versus by the
standard `helm uninstall`. Take a backup before deletion if data
retention is required.

This resolves cozystack/cozystack#2513.

### Screenshots

Not applicable — no UI changes.

### Release note

```release-note
fix(mongodb): reclaim PVCs and operator-managed secrets on uninstall via Percona finalizers.

WARNING: upgrading scopes the admin credentials secret per release instead of the namespace-shared default. This triggers an in-place rotation of the five operator-managed admin passwords on first reconcile. Pods are not restarted; application user passwords are unaffected. Workloads consuming `<release>-credentials` must be restarted to pick up the new admin passwords.
```

### Out of scope / follow-up

- Bats integration test would require a live cluster with the Percona
MongoDB operator and belongs in the `hack/e2e-*.bats` family — left as a
follow-up.
- Issue #2513 mentions considering similar treatment for
PostgreSQL/MariaDB/Redis/ClickHouse. Each operator has a different
cleanup mechanism; that work is intentionally out of scope here and
should be tracked per-app.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Expanded uninstall and upgrade guidance: which secrets/pvc remain,
cert-manager TLS secrets must be deleted manually, remediation for
orphaned legacy secret, and recovery steps for stuck deletions when the
operator is removed before CRs.

* **New Features**
  * Added deletion finalizers to enforce controlled teardown.
* User credentials secret now uses a deterministic, release-scoped name.

* **Tests**
* Added chart tests for finalizer behavior (sharded/non‑sharded) and
release-scoped users secret.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-04 13:18:02 +03:00
Arsolitt 5786afe8bf feat(kubernetes): add kubelet resource reservations for worker nodes (#2420) 
**What this PR does**

Adds configurable kubelet resource reservations and eviction thresholds
for tenant cluster worker nodes. Without explicit reservations and
cgroup enforcement, the scheduler treats nearly all node memory and CPU
as allocatable. Under pressure, the OOM killer can target kubelet itself
because no cgroup boundaries protect it, leading to "Kubelet stopped
posting node status" failures and unrecoverable NotReady nodes.

This PR auto-computes per-node-group kubelet reservations from the
effective node capacity (instanceType or explicit `resources`):

- **Memory**: `systemReservedMemory` and `kubeReservedMemory` each
default to 5% of effective memory, clamped 256Mi-1Gi
- **CPU**: `systemReservedCpu` and `kubeReservedCpu` each default to 5%
of effective CPU, clamped 50m-500m
- **Eviction**: configurable `evictionHardMemory` (default 7%) and
`evictionSoftMemory` (default 10%) with validation that hard < soft
- **Disk eviction**: `nodefs.available`, `imagefs.available`, and
`nodefs.inodesFree` thresholds are preserved alongside memory signals
- **Capacity annotations**: cluster-autoscaler annotations now report
allocatable memory and CPU (total minus all reservations and eviction
threshold) instead of raw totals
- **Cgroup enforcement**: `enforce-node-allocatable=pods` configures
advisory-only reservations — the scheduler accounts for reserved
resources when computing allocatable capacity, but no cgroup boundaries
are enforced for system or kube reserved resources
- **Validation**: format checks, zero-value rejection, eviction ordering
(hard < soft), same-unit-type requirement for eviction thresholds, and
total reservation sanity check against effective node capacity

All reservation fields are optional and overridable per node group via
`.nodeGroups[].kubelet`.

**Default rendered kubeletExtraArgs:**

```yaml
kubeletExtraArgs:
  system-reserved: "memory=256Mi,cpu=50m"
  kube-reserved: "memory=256Mi,cpu=50m"
  eviction-hard: "memory.available<7%,nodefs.available<10%,imagefs.available<15%,nodefs.inodesFree<5%"
  eviction-soft: "memory.available<10%,nodefs.available<15%,imagefs.available<20%"
  eviction-soft-grace-period: "memory.available=1m30s,nodefs.available=1m30s,imagefs.available=1m30s"
  eviction-minimum-reclaim: "memory.available=256Mi"
  enforce-node-allocatable: "pods"
```

**New values schema:**

```yaml
nodeGroups:
  md0:
    kubelet:
      systemReservedMemory: ""   # auto-computed (5% of effective memory, 256Mi-1Gi)
      kubeReservedMemory: ""     # auto-computed (5% of effective memory, 256Mi-1Gi)
      systemReservedCpu: ""      # auto-computed (5% of effective CPU, 50m-500m)
      kubeReservedCpu: ""        # auto-computed (5% of effective CPU, 50m-500m)
      evictionHardMemory: "7%"   # absolute (e.g. 200Mi) or percentage
      evictionSoftMemory: "10%"  # must be > evictionHardMemory, same unit type
```

**Release note:**

```
feat(kubernetes): add per-node-group kubelet resource reservations with auto-computed system/kube reserved memory (5% of effective memory, clamped 256Mi-1Gi) and CPU (5% of effective CPU, clamped 50m-500m), configurable memory eviction thresholds, preserved disk eviction signals (nodefs, imagefs, inodesFree), pods-only cgroup enforcement, and reservation-aware allocatable capacity annotations for cluster-autoscaler.
```
 2026-05-01 14:45:30 +03:00
Andrei Kvapil 70e507ee5c fix(security): bump Go dependencies to clear Dependabot CVE alerts (#2494) 
## What this PR does

Closes the 25 open Dependabot Go-ecosystem alerts on this repository (2
critical, 11 high, 12 moderate) by bumping the affected dependencies in
five Go modules to their latest stable versions. Splits the work into
six signed commits, one per logical block, so the diff in each commit
can be reviewed in isolation.

### Full advisory coverage

| # | Module | Severity | Package | Advisory | Title | Vulnerable |
Shipped |
|---|---|---|---|---|---|---|---|
| 29 | `go.mod` | critical | `google.golang.org/grpc` |
[GHSA-p77j-4mvh-x3m3](https://github.com/advisories/GHSA-p77j-4mvh-x3m3)
| gRPC-Go authorization bypass via missing leading slash in `:path` | `<
1.79.3` | `1.80.0` |
| 28 | `kubevirt-csi-driver/go.mod` | critical |
`google.golang.org/grpc` |
[GHSA-p77j-4mvh-x3m3](https://github.com/advisories/GHSA-p77j-4mvh-x3m3)
| gRPC-Go authorization bypass via missing leading slash in `:path` | `<
1.79.3` | `1.80.0` |
| 32 | `go.mod` | high | `go.opentelemetry.io/otel` |
[GHSA-mh2q-q3fh-2475](https://github.com/advisories/GHSA-mh2q-q3fh-2475)
| Multi-value `baggage` header extraction causes excessive allocations
(remote DoS amplification) | `>= 1.36.0, <= 1.40.0` | `1.43.0` |
| 31 | `go.mod` | high | `github.com/moby/spdystream` |
[GHSA-pc3f-x583-g7j2](https://github.com/advisories/GHSA-pc3f-x583-g7j2)
| SpdyStream: DoS on CRI | `<= 0.5.0` | `0.5.1` |
| 30 | `go.mod` | high | `go.opentelemetry.io/otel/sdk` |
[GHSA-hfvc-g4fc-pqhx](https://github.com/advisories/GHSA-hfvc-g4fc-pqhx)
| BSD `kenv` command not using absolute path enables PATH hijacking |
`>= 1.15.0, <= 1.42.0` | `1.43.0` |
| 27 | `go.mod` | high | `go.opentelemetry.io/otel/sdk` |
[GHSA-9h8m-3fm2-qjrq](https://github.com/advisories/GHSA-9h8m-3fm2-qjrq)
| OpenTelemetry Go SDK vulnerable to arbitrary code execution via PATH
hijacking | `>= 1.21.0, < 1.40.0` | `1.43.0` |
| 26 | `kubevirt-csi-driver/go.mod` | high |
`github.com/opencontainers/runc` |
[GHSA-cgrx-mc8f-2prm](https://github.com/advisories/GHSA-cgrx-mc8f-2prm)
| runc container escape and DoS due to arbitrary write gadgets and
procfs write redirects | `<= 1.2.7` | `1.4.2` |
| 25 | `kubevirt-csi-driver/go.mod` | high |
`github.com/opencontainers/runc` |
[GHSA-qw9x-cqr3-wc7r](https://github.com/advisories/GHSA-qw9x-cqr3-wc7r)
| runc container escape with malicious config due to `/dev/console`
mount and related races | `>= 1.0.0-rc3, <= 1.2.7` | `1.4.2` |
| 24 | `kubevirt-csi-driver/go.mod` | high |
`github.com/opencontainers/runc` |
[GHSA-9493-h29p-rfm2](https://github.com/advisories/GHSA-9493-h29p-rfm2)
| runc container escape via "masked path" abuse due to mount race
conditions | `<= 1.2.7` | `1.4.2` |
| 23 | `kubevirt-csi-driver/go.mod` | high | `golang.org/x/oauth2` |
[GHSA-6v2p-p543-phr9](https://github.com/advisories/GHSA-6v2p-p543-phr9)
| Improper Validation of Syntactic Correctness of Input | `< 0.27.0` |
`0.36.0` |
| 16 | `kubeovn-webhook/go.mod` | high | `golang.org/x/oauth2` |
[GHSA-6v2p-p543-phr9](https://github.com/advisories/GHSA-6v2p-p543-phr9)
| Improper Validation of Syntactic Correctness of Input | `< 0.27.0` |
`0.36.0` |
| 4 | `kubeovn-webhook/go.mod` | high | `golang.org/x/net` |
[GHSA-4374-p667-p6c8](https://github.com/advisories/GHSA-4374-p667-p6c8)
| HTTP/2 rapid reset can cause excessive work in `net/http` | `< 0.17.0`
| `0.53.0` |
| 3 | `kubeovn-webhook/go.mod` | high | `golang.org/x/net` |
[GHSA-vvpx-j8f3-3w6h](https://github.com/advisories/GHSA-vvpx-j8f3-3w6h)
| `golang.org/x/net` vulnerable to Uncontrolled Resource Consumption |
`< 0.7.0` | `0.53.0` |
| 22 | `kubevirt-csi-driver/go.mod` | medium | `golang.org/x/net` |
[GHSA-vvgc-356p-c3xw](https://github.com/advisories/GHSA-vvgc-356p-c3xw)
| `golang.org/x/net` vulnerable to Cross-site Scripting | `< 0.38.0` |
`0.53.0` |
| 21 | `kubevirt-csi-driver/go.mod` | medium | `golang.org/x/net` |
[GHSA-qxp5-gwg8-xv66](https://github.com/advisories/GHSA-qxp5-gwg8-xv66)
| HTTP Proxy bypass using IPv6 Zone IDs | `< 0.36.0` | `0.53.0` |
| 20 | `kubevirt-csi-driver/go.mod` | medium |
`github.com/opencontainers/runc` |
[GHSA-jfvp-7x6p-h2pv](https://github.com/advisories/GHSA-jfvp-7x6p-h2pv)
| runc can be confused to create empty files/directories on the host |
`< 1.1.14` | `1.4.2` |
| 19 | `go.mod` | medium | `golang.org/x/crypto` |
[GHSA-f6x5-jh6r-wrfv](https://github.com/advisories/GHSA-f6x5-jh6r-wrfv)
| `golang.org/x/crypto/ssh/agent` vulnerable to panic if message is
malformed (OOB read) | `< 0.45.0` | `0.50.0` |
| 18 | `go.mod` | medium | `golang.org/x/crypto` |
[GHSA-j5w8-q4qc-rx2x](https://github.com/advisories/GHSA-j5w8-q4qc-rx2x)
| `golang.org/x/crypto/ssh` allows an attacker to cause unbounded memory
consumption | `< 0.45.0` | `0.50.0` |
| 14 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-vvgc-356p-c3xw](https://github.com/advisories/GHSA-vvgc-356p-c3xw)
| `golang.org/x/net` vulnerable to Cross-site Scripting | `< 0.38.0` |
`0.53.0` |
| 11 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-qxp5-gwg8-xv66](https://github.com/advisories/GHSA-qxp5-gwg8-xv66)
| HTTP Proxy bypass using IPv6 Zone IDs | `< 0.36.0` | `0.53.0` |
| 9 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-xrjj-mj9h-534m](https://github.com/advisories/GHSA-xrjj-mj9h-534m)
| `golang.org/x/net/http2` vulnerable to possible excessive memory
growth | `< 0.4.0` | `0.53.0` |
| 8 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-4v7x-pqxf-cx7m](https://github.com/advisories/GHSA-4v7x-pqxf-cx7m)
| `net/http`, `x/net/http2`: close connections when receiving too many
headers | `< 0.23.0` | `0.53.0` |
| 7 | `kubeovn-webhook/go.mod` | medium | `google.golang.org/protobuf` |
[GHSA-8r3f-844c-mc37](https://github.com/advisories/GHSA-8r3f-844c-mc37)
| `protojson.Unmarshal` infinite loop on certain forms of invalid JSON |
`< 1.33.0` | `1.36.11` |
| 6 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-qppj-fm5r-hxr3](https://github.com/advisories/GHSA-qppj-fm5r-hxr3)
| HTTP/2 Stream Cancellation Attack | `< 0.17.0` | `0.53.0` |
| 5 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-2wrh-6pvc-2jm9](https://github.com/advisories/GHSA-2wrh-6pvc-2jm9)
| Improper rendering of text nodes in `golang.org/x/net/html` | `<
0.13.0` | `0.53.0` |

Sorted by severity (critical → high → medium), then by alert number. The
`Module` column lists the manifest path inside the repo (`go.mod` is the
root); `Vulnerable` is what GitHub flagged in the original advisory;
`Shipped` is the version this PR pins to.

### Modules bumped

1. Root `go.mod` (`github.com/cozystack/cozystack`) — purely transitive
bumps for `grpc`, `otel`, `otel/sdk`, `x/crypto`, `spdystream`.
2. `api/apps/v1alpha1/go.mod` — bumps `k8s.io/apimachinery` and
`golang.org/x/net`/`x/text` for consistency with the root module's `go
1.26.2` directive.
3. `packages/apps/kubernetes/images/kubevirt-csi-driver/go.mod` — bumps
the upstream `kubevirt.io/csi-driver` pin to its latest commit, raises
the `k8s.io/*` replace block from `v0.31.4` to `v0.36.0`, bumps `runc`
to `v1.4.2`, and adapts `main.go` to the upstream's new builder-style
API for `service.NewKubevirtCSIDriver` (the constructor was split into
`WithIdentityService` / `WithControllerService` / `WithNodeService`).
4. `packages/system/kubeovn-webhook/images/kubeovn-webhook/go.mod` —
direct bumps to `golang.org/x/net`, `golang.org/x/oauth2`,
`google.golang.org/protobuf`.
5. `packages/system/dashboard/images/token-proxy/go.mod` — bumps `jwx`,
`httprc`, `x/crypto`, `x/sys` for consistency with the root module's
directive.

All five modules now declare `go 1.26.2`. The three vendored-image
Dockerfiles (`kubevirt-csi-driver`, `kubeovn-webhook`, `token-proxy`)
move to `golang:1.26{,-alpine}`.

### First-party Dockerfile bumps

The root `go.mod` directive jump to `1.26.2` means every Dockerfile that
copies the root `go.mod` and builds a cozystack-owned binary needs a
matching base image, otherwise the in-container toolchain has to pull
`go 1.26` on the fly via `GOTOOLCHAIN=auto` at every build. Eight images
are bumped from `golang:1.25-alpine` to `golang:1.26-alpine`:
`cozystack-operator`, `cozystack-api`, `cozystack-controller`,
`backup-controller`, `backupstrategy-controller`, `flux-plunger`,
`kubeovn-plunger`, `lineage-controller-webhook`.

Vendored Dockerfiles (`kubevirt-cloud-provider`, `redis-operator`,
`keycloak-operator`, `linstor-csi`, `multus-cni`, `objectstorage`,
`s3manager`, `kamaji`) are intentionally untouched: each one fetches its
source from upstream at build time and pins its `golang:` base image to
whatever the upstream commit was tested against.

### Why bump past the advisory minimums

Bumping each affected dependency to its latest stable rather than the
lowest patched version means we pick up subsequent (non-CVE) bug fixes
for free and reduce the number of follow-up Dependabot bumps the next
reviewer has to triage. Where this required a cascading bump (e.g.
`runc` v1.4.2 only compiles together with `mount-utils` v0.32+, which
dropped the deprecated `libcontainer/userns` import path), the cascade
was followed through.

### Build verification

`go build ./...` and `go vet ./...` are clean across all five modules
locally. Two `go vet` warnings in the root module
(`pkg/lineage/lineage_test.go:76` unkeyed struct fields,
`cmd/cozypkg/cmd/add.go:521` non-constant format string) and three
`randfill.Continue` test panics in `pkg/apis/{apps,core}/install` and
`pkg/apiserver` also reproduce on `main` and are out of scope for this
PR.

### Out of scope

The `k8s.io/*` libraries in `kubeovn-webhook/go.mod` are still pinned at
`v0.26.0`; they are not flagged by Dependabot and bumping them would be
a separate, much larger maintenance change.

### Release note

```release-note
fix(security): bump Go dependencies in root, api/apps/v1alpha1, kubevirt-csi-driver, kubeovn-webhook, and token-proxy modules to clear all open Dependabot CVE alerts
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
  * Upgraded Go toolchain across components and build stages to 1.26.x.
* Updated numerous dependencies for compatibility, stability and
security (Kubernetes APIs, gRPC, OpenTelemetry, golang.org/x/*, and
related libs).
* Several containers now run as a non-root user (UID/GID 65532) to
improve runtime security.

* **Refactor**
* CSI driver initialization refactored to a builder-style pattern
enabling conditional service wiring.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-01 13:16:03 +02:00
Arsolitt 59c547eacf docs(mongodb): document password rotation behavior on upgrade 
Describe the in-place password rotation performed by the Percona operator
when upgrading to releases that scope the system users secret per CR.
Clarify that consumers of the credentials secret need a pod restart to
pick up the new password, custom user secrets are unaffected, and the
legacy namespace-shared secret is left orphaned and must be cleaned up
manually.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-04-30 16:41:00 +03:00
Arsolitt 22d2a224bb test(kubernetes): rename ephemeralStorage to diskSize in kubelet tests 
PR #2454 renamed the nodeGroups field ephemeralStorage to diskSize and
added a migration guard that fails template rendering when the old name
is used. Update kubelet reservation tests to use the new field name.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-04-30 14:14:23 +03:00
Arsolitt d4808b9674 merge: resolve conflicts with main after PR #2454 
Merge origin/main into feat/kubernetes-kubelet-reserved-resources.

PR #2454 (persistent storage for worker nodes) landed in main and
touched the same nodeGroups schema. Conflicts resolved by keeping
both feature sets (kubelet reservations + persistent storage):

- api/apps/v1alpha1/kubernetes/types.go: merged kubebuilder default
  marker with all keys from both features
- packages/apps/kubernetes/README.md: merged Parameters table with
  diskSize/storageClass + kubelet* rows
- packages/system/kubernetes-rd/cozyrds/kubernetes.yaml: merged
  openAPISchema and keysOrder, dropped stale ephemeralStorage path

Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-04-30 13:51:28 +03:00
Aleksei Sviridkin dff5f8d19a feat(kubernetes)!: replace ephemeral emptyDisk with persistent storage for worker nodes (#2454) 
## What this PR does

Replace the ephemeral `emptyDisk` volume on tenant Kubernetes worker VMs
with persistent PVC-backed storage via KubeVirt `dataVolumeTemplates`.
Previously, kubelet certificates, kubeconfig, and containerd state were
lost on VM reboot, causing the node to fall out of the tenant cluster.

**Changes:**

- Rename `ephemeralStorage` field to `diskSize` in NodeGroup
configuration
- Replace `emptyDisk` volume with `dataVolumeTemplates` using CDI blank
source
- Add `cdi.kubevirt.io/storage.usePopulator: "false"` annotation to work
around CDI volume population issues with LINSTOR CSI
- Add optional per-nodeGroup `storageClass` field (defaults to cluster
default)
- Set fixed 2Gi ephemeral-storage resource for virt-launcher pod
- Rename internal mount paths from `/ephemeral` to `/persistent`
- Add helm unit tests (`cluster_test.yaml`) covering dataVolumeTemplates
defaults, custom disk size, storageClass presence/absence, fixed
ephemeral-storage, disk name consistency, and CDI populator annotation
- Quote storage values in templates (`diskSize`, `storageClassName`) for
Helm correctness
- Guard bind-mount commands with `mountpoint -q` checks and create
target directories to handle fresh images and cloud-init retries
- Set `blockSize.matchVolume` on worker node disks for automatic block
size detection, fixing DRBD 4Kn compatibility with QEMU

**Known gotchas:**

- CDI volume population (prime PVC rebind) causes `ClaimMisbound` errors
with LINSTOR CSI driver, regardless of StorageClass (`replicated` or
`local`). The `usePopulator: false` annotation switches to the legacy
import path which works correctly. This is a known CDI issue documented
in
[kubevirt/containerized-data-importer#3146](https://github.com/kubevirt/containerized-data-importer/issues/3146).
- The `replicated` StorageClass with DRBD `suspend-io` quorum policy can
cause transient VM IO errors during LINSTOR satellite restarts (quorum
loss). The `local` StorageClass is recommended for worker node disks.
When `storageClass` is not specified in a nodeGroup, the cluster default
StorageClass is used.
- Existing clusters will trigger a rolling replacement of all worker
nodes on upgrade due to changed KubevirtMachineTemplate hash.
- DRBD volumes use 4096-byte logical sectors (`block-size 4096`).
Without explicit block size configuration, QEMU defaults to 512-byte
sectors, causing immediate IO errors and VM pause. The
`blockSize.matchVolume` setting in the disk spec ensures KubeVirt
auto-detects the correct sector size from the volume.

**Tested on a live cluster:**

- Verified DataVolume provisioning with `local` StorageClass
- Verified nodes join the tenant cluster and reach Ready state
- Verified node survives VM reboot (pod deletion) and returns to Ready
with the same Node object, reusing kubelet certificates from persistent
disk

### Screenshots

N/A — no UI changes.

### Release note

```release-note
feat(kubernetes)!: replace ephemeral emptyDisk with persistent storage for worker nodes

Worker node VMs now use persistent PVC-backed disks instead of ephemeral emptyDisk volumes. Kubelet certificates, kubeconfig, and containerd state survive VM reboots, preventing nodes from falling out of the tenant cluster.

BREAKING CHANGE: The `ephemeralStorage` field in nodeGroups has been renamed to `diskSize`. A new optional `storageClass` field allows per-nodeGroup storage class selection. Users must update their values overrides accordingly.

Worker node disks now use `blockSize.matchVolume` to automatically detect the underlying volume's sector size, ensuring compatibility with DRBD-backed storage classes that use 4096-byte logical sectors.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Optional storageClass parameter to select storage class for worker
node persistent disks (defaults to cluster default).
  * Makefile test target added to run Helm unit tests for the chart.

* **Configuration Changes**
* Worker node storage parameter renamed from ephemeralStorage to
diskSize (default: 20Gi).
* Kubelet/containerd state moved to persistent disk paths and node
ephemeral-storage accounting fixed to 2Gi; disk provisioning aligns
block sizing with backing volumes.

* **Documentation**
* Updated docs and chart/schema to reflect the persistent disk model and
new fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-30 12:41:34 +03:00
Aleksei Sviridkin 4dbd4b0b85 fix(harbor): mark credentials Secret with user-secret label (#2541) 
## What this PR does

- Stamps the Helm-managed `{{ .Release.Name }}-credentials` Secret with
`apps.cozystack.io/user-secret: "true"`.
- Adds a `matchLabels` rule to `secrets.include` in the Harbor
`ApplicationDefinition`, alongside the existing `resourceNames` entry.

The lineage webhook decides whether a Secret is tenant-visible by
matching its name against the `secrets.include[].resourceNames` template
in the corresponding `ApplicationDefinition`. For Harbor the template
`{{ .name }}-credentials` renders to `{name}-credentials` and does not
account for the `release.prefix: "harbor-"` set in the same definition,
while the Helm chart actually creates `harbor-{name}-credentials`. The
names never match, so the Secret ends up labelled
`internal.cozystack.io/tenantresource=false` and never appears in the
TenantSecrets API.

This change follows the existing `apps.cozystack.io/user-secret: "true"`
pattern already used by Bucket
(`packages/system/bucket/templates/user-credentials.yaml`) and RabbitMQ
(`packages/apps/rabbitmq/templates/rabbitmq.yaml`). The webhook then
matches via labels and is independent of how Helm names the Secret.

### Screenshots

Not applicable — no UI changes in this PR.

### Release note

```release-note
fix(harbor): expose Harbor credentials Secret to tenant-aware clients by marking it with the `apps.cozystack.io/user-secret` label, mirroring the pattern already used by Bucket and RabbitMQ.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated Harbor configuration to support label-based secret discovery
alongside existing name-based selection, improving secret management
reliability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-30 12:40:48 +03:00
IvanHunters 53caea3e08 chore(postgres): regenerate after denylist update 
Update description in values.yaml with corrected denylist and re-run
cozyvalues-gen to propagate changes to values.schema.json, README.md,
and types.go.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 10:20:57 +03:00
IvanHunters 95f7be7a38 docs(postgres): update denylist documentation to match implementation 
Update values.yaml comments and README to reflect the corrected denylist:
remove log_destination and event_source (not code-execution vectors),
add dynamic_library_path and *_preload_libraries (arbitrary .so loading).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 10:18:29 +03:00
IvanHunters 97d8f84be4 fix(postgres): update security denylist for parameter blocking 
Remove log_destination and event_source (not code-execution vectors).
Add dynamic_library_path, local_preload_libraries, session_preload_libraries,
shared_preload_libraries which allow loading arbitrary .so files.
Improve error message accuracy.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 10:14:20 +03:00
IvanHunters 6ec761fc1f fix(postgres): accept integer values in parameters schema for backward compat 
Allow integer values in postgresql.parameters via anyOf schema, so existing
HelmReleases with max_connections: 100 (integer) are not broken on upgrade.
Template coerces all values to string via toString before quoting.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 10:12:47 +03:00
Myasnikov Daniil 2555a9ac08 fix(harbor): merge BucketInfo at values root, drop targetPath 
The previous fix (a21c18f9) sourced BucketInfo via Flux valuesFrom with
`targetPath: bucket.bucketInfo`. Flux runs values with `targetPath`
through Helm's `strvals.ParseInto`, which splits the value on commas as
list separators. The COSI BucketInfo JSON is comma-rich, so values
resolution bailed:

  could not resolve Secret chart values reference
  'tenant-X/harbor-X-registry-bucket' with key 'BucketInfo':
  key "\"spec\":{\"bucketName\":\"bucket-1785...\""
  has no value (cannot end with ,)

Drop `targetPath`. With only `valuesKey: BucketInfo`, helm-controller
unmarshals the value as YAML and merges at the chart's values root, so
JSON commas stay nested instead of being split. The system chart's
bucket-secret.yaml now reads `.Values.spec.bucketName` /
`.Values.spec.secretS3.{accessKeyID,accessSecretKey,endpoint}`,
guarded by nested `with` blocks so the registry-s3 Secret renders only
when secretS3 has been populated.

Gating semantics from the previous fix are preserved: with the default
`optional: false`, helm-controller still refuses to compose values
until the COSI BucketAccess controller writes `BucketInfo` into the
Secret, and the value's content still drives the HR config-digest.

Verified via `helm template` against the system chart with three
scenarios: missing `spec` and `spec` without `secretS3` render
nothing; full BucketInfo renders all five S3 env keys correctly.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
 2026-04-30 09:35:14 +05:00
IvanHunters fac0622499 fix(postgres): make security denylist case-insensitive 
PostgreSQL parameter names are case-insensitive, but the denylist
check was case-sensitive, allowing bypass via mixed-case names
(e.g., ARCHIVE_COMMAND instead of archive_command).

Normalize parameter names to lowercase before checking against the
denylist to prevent security bypass.

Thanks to CodeRabbit for catching this security issue.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 01:43:43 +03:00
IvanHunters 96c7c16c21 fix(postgres): correct merge argument order for parameter precedence 
Fix regression where defaults were overriding user-supplied parameters.
Sprig merge gives precedence to first argument, so user values must be
merged before defaults to maintain correct override behavior.

Before: merge (dict) $defaults .Values.postgresql.parameters
After:  merge (dict) .Values.postgresql.parameters $defaults

Now max_wal_senders and other user parameters correctly override defaults.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 01:43:42 +03:00
ohotnikov.ivan f8001e23b6 fix(postgres): address review feedback 
- Regenerate deepcopy artifacts to remove orphaned PostgreSQLParameters methods
- Update e2e test to use string value for max_connections parameter
- Fix template to avoid .Values mutation and use quote function for safe YAML escaping

Signed-off-by: ohotnikov.ivan <ohotnikov.ivan@e-queo.net>
 2026-04-30 01:43:42 +03:00
IvanHunters fb0ed08565 feat(postgres): support all PostgreSQL configuration parameters 
Allow users to configure any PostgreSQL server parameter via the
postgresql.parameters field. All parameter values must be provided
as strings (numeric values should be quoted).

Security measures:
- Block parameters that allow shell command execution (archive_command,
  restore_command, ssl_passphrase_command, log_destination, event_source)
- Validation implemented in templates to prevent security vulnerabilities

Generated files updated:
- types.go: Add Parameters field with map[string]string type
- README.md: Add parameter documentation and warnings
- values.schema.json: Update schema for new field
- CRD: Update OpenAPI schema with new field and reordered properties

CloudNativePG manages certain parameters automatically (archive_mode,
primary_conninfo, wal_level). Users should not override these as it
may break backup/replication functionality.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-04-30 01:43:41 +03:00
IvanHunters fdbb6fdfd7 fix(harbor): mark credentials Secret with user-secret label 
The Helm-managed harbor-{name}-credentials Secret was not visible
through the TenantSecrets API because the lineage webhook compared the
real Secret name against the rendered template "{{ .name }}-credentials"
(which becomes "{name}-credentials" without the "harbor-" HelmRelease
prefix), failed to match, and labelled the Secret as
internal.cozystack.io/tenantresource=false.

Adopt the same pattern Bucket and RabbitMQ already use: stamp the
user-facing Secret with apps.cozystack.io/user-secret=true and add a
matchLabels rule alongside the existing resourceNames entry in the
ApplicationDefinition. The webhook now matches via labels instead of
relying on Secret-name templating, so the Secret is properly marked
tenantresource=true and surfaces in tenant-aware clients.

Signed-off-by: IvanHunters <ivan.okhotnikov@aenix.io>
 2026-04-29 20:47:09 +03:00
Ivan Okhotnikov c2884eac17 fix(kafka): bump default resourcesPreset to medium 
The `small` preset (512Mi) is not sufficient for the Strimzi 0.45 /
Kafka 3.9 image baseline. With the default Strimzi heap formula
(Kafka 50%, ZooKeeper 75% of container memory limit), the remaining
non-heap budget (Metaspace, code cache, direct buffers, JMX
javaagent, embedded Jetty admin server, KRaft migration endpoints,
thread stacks) consistently overflows the 512Mi cgroup limit at
startup, leading to OOMKilled (exit 137) and CrashLoopBackOff for
both Kafka and ZooKeeper pods.

Bump the default `resourcesPreset` for kafka and zookeeper from
`small` (512Mi) to `medium` (1Gi). The `small` preset stays in the
enum so existing deployments that pin it are unaffected; only the
default changes.

Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
 2026-04-29 17:45:09 +03:00
Arsolitt 998cc77635 chore(kubernetes): merge main to resolve CRD schema conflict 
Integrate the HAMi GPU virtualization addon from main while
keeping the persistent-storage diskSize and storageClass fields
introduced by this branch.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-04-29 16:54:58 +03:00
Arsolitt d5caffcc8d fix(kubernetes): gate HAMi HelmRelease on _namespace.etcd 
Match the pattern used by every other tenant-cluster addon HelmRelease
(gpu-operator, cilium, fluxcd, ...): only render when the upstream etcd
namespace is ready, not just when the addon is enabled. Add values-ci.yaml
to the HAMi test suite so the new gate has the fixture it needs.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-04-29 11:13:42 +03:00
Myasnikov Daniil 8daf495286 fix(harbor): drive bucket-secret.yaml from values, gate HR on BucketInfo 
Two bugs caused the harbor E2E to fail (CI #25049445125):

1. cozy-harbor's bucket-secret.yaml called `index $existingSecret.data
   "BucketInfo"` unconditionally. During first-time install the COSI
   BucketAccess controller may create the credentials Secret as a
   placeholder before populating it, so `.data` is nil and the chart
   render crashes with `index of untyped nil`.

2. The downstream `<release>-system` HelmRelease starts reconciling
   immediately, in parallel with bucket provisioning, hitting bug #1
   on every retry within the test's 5-minute window.

The previous shape relied on `lookup` to read the Secret at render
time. helm-controller's upgrade trigger is digest-based over composed
values + chart artifact, so a `lookup` returning new data on a later
reconcile is not enough on its own to force an upgrade — the rendered
output may diverge but the digest does not.

Switch to a values-driven shape:

- `bucket-secret.yaml` now reads `.Values.bucket.bucketInfo` (a JSON
  string) and uses `dig` for safe access; if the value is empty or the
  expected nested fields are missing, no `*-registry-s3` Secret is
  rendered.
- The `<release>-system` HR sources `BucketInfo` via `valuesFrom` with
  `targetPath: bucket.bucketInfo`. With the default `optional: false`,
  helm-controller will refuse to compose values until the key exists,
  which both gates initial reconciliation on the BucketAccess Secret
  being populated and forces a config-digest change (and thus a helm
  upgrade) when its contents change.
- `bucket.secretName` is removed from the system chart's values and
  from the apps chart's `values:` block; the only consumer (the
  `lookup` call) is gone.

Verified four scenarios via `helm template` against the system chart:
unset bucketInfo, empty string, empty JSON object `{}`, and a
fully-populated BucketInfo — only the last renders the registry-s3
Secret; the others render nothing without erroring.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-04-29 11:37:10 +05:00
Arsolitt 805b3b17cb fix(kubernetes): add CI values to GPU Operator HAMi tests 
Tests need _namespace.etcd from values-ci.yaml after the merge
introduced an etcd-namespace guard on the gpu-operator HelmRelease
condition.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
 2026-04-28 19:11:26 +03:00