## What this PR does
Reverts the `publishing.exposure=loadBalancer` mode for ingress-nginx
added in #2468.
The mode rendered the ingress-nginx Service as `type: LoadBalancer` and
a `CiliumLoadBalancerIPPool` over `publishing.externalIPs`, as a
migration path off `Service.spec.externalIPs` (deprecated upstream in
Kubernetes v1.36 per KEP-5707). In default Cozystack the implementation
produces two failure modes:
- `publishing.externalIPs` is the operator-supplied list of node public
IPs (addresses the cluster nodes already hold and listen on, see [docs
section "4b. Node Public IP
Setup"](https://cozystack.io/docs/v1.3/install/cozystack/platform/#4b-node-public-ip-setup)),
not a virtual-VIP pool. Treating it as one and DNATing those IPs via
Cilium LB-IPAM collides with the existing ingress-nginx DNAT on the same
IPs (`Service.spec.externalIPs` path), so traffic is forwarded
non-deterministically.
- Cilium in default Cozystack runs with `ipam.mode: kubernetes` and no
L2/BGP announcer, so a pool over addresses outside the node range is
allocated but never advertised — the Service stays unreachable.
The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side `IPPool` baked into the tenant chart. That follow-up belongs
in its own PR.
### Removed
- `publishing.exposure` value (and the inline comment) from
`core/platform/values.yaml`.
- `expose-mode` plumbing line in `core/platform/templates/apps.yaml`.
- `CiliumLoadBalancerIPPool` template at
`apps/tenant/templates/cilium-lb-pool.yaml`.
- `type: LoadBalancer + externalTrafficPolicy: Local` branch in
`extra/ingress/templates/nginx-ingress.yaml`. The Service now renders as
`ClusterIP + spec.externalIPs` when externalIPs are set, falling through
to `type: LoadBalancer` when they are not — pre-#2468 shape.
- Exposure mode section from `extra/ingress/README.md`.
- `exposure_test.yaml` fixtures in both charts.
### Compatibility
Operators who explicitly set `publishing.exposure=loadBalancer` will see
their ingress Service revert to `ClusterIP + spec.externalIPs` on
upgrade. Anyone on the default value (`externalIPs`) is unaffected.
### Release note
```release-note
revert(ingress): remove the publishing.exposure=loadBalancer mode added in #2468. The default externalIPs path is unchanged; operators who explicitly opted into loadBalancer will have their ingress Service revert to ClusterIP + spec.externalIPs on upgrade. Migration off Service.spec.externalIPs (deprecated in KEP-5707) will be reintroduced via a mechanism-agnostic API knob in a follow-up.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Refactor**
* Removed conditional generation of the Cilium load‑balancer IP pool and
simplified service exposure to rely on explicit external IP values.
* Streamlined configuration for service exposure keys.
* **Documentation**
* Removed the exposure-mode guidance from ingress documentation.
* **Tests**
* Deleted the exposure-mode-related test suites covering various
external IP and loadBalancer scenarios.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2603)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:
nano -> t1.nano micro -> t1.micro
small -> t1.small medium -> c1.small
large -> c1.medium xlarge -> c1.large
2xlarge -> c1.xlarge
Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.
Three hardcoded legacy literals in chart templates migrated:
- packages/apps/harbor/templates/harbor.yaml:121 (portal)
- packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
- packages/system/keycloak/templates/sts.yaml:115 (heap fallback)
packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.
Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Reverts the ingress loadBalancer exposure mode added in #2468.
The mode rendered the ingress-nginx Service as type: LoadBalancer and a
CiliumLoadBalancerIPPool over publishing.externalIPs, as a migration path
off Service.spec.externalIPs (deprecated upstream in Kubernetes v1.36 per
KEP-5707). In default Cozystack the implementation produces two failure
modes:
- publishing.externalIPs is the operator-supplied list of node public IPs
(addresses the cluster nodes already hold and listen on), not a
virtual-VIP pool. Treating it as one and DNATing those IPs via Cilium
LB-IPAM collides with the existing ingress-nginx DNAT on the same IPs
(Service.spec.externalIPs path), so traffic is forwarded
non-deterministically.
- Cilium in default Cozystack runs with ipam.mode: kubernetes and no
L2/BGP announcer, so a pool over addresses outside the node range is
allocated but never advertised — the Service stays unreachable.
The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side IPPool baked into the tenant chart. That follow-up belongs in
its own PR.
Removed:
- publishing.exposure value (and the inline comment) from
core/platform/values.yaml
- expose-mode plumbing line in core/platform/templates/apps.yaml
- CiliumLoadBalancerIPPool template at
apps/tenant/templates/cilium-lb-pool.yaml
- type: LoadBalancer + externalTrafficPolicy: Local branch in
extra/ingress/templates/nginx-ingress.yaml (Service renders back as
ClusterIP + spec.externalIPs when externalIPs are set, falling through
to type: LoadBalancer when they are not — pre-#2468 shape)
- Exposure mode section from extra/ingress/README.md
- exposure_test.yaml fixtures in both charts
Operators who explicitly set publishing.exposure=loadBalancer will see
their ingress Service revert to ClusterIP + spec.externalIPs on upgrade.
Anyone on the default value (externalIPs) is unaffected.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Stacks on top of #2258. Adds `dependsOn: ingress-nginx-system` to the
inner `seaweedfs-system` HelmRelease so flux blocks its install until
the ingress chart is Ready.
## Why
In #2258's E2E run (and the `cozyreport-2258` bundle from this morning),
`seaweedfs-system` failed to install with:
```
Helm install failed for release tenant-root/seaweedfs-system: 1 error occurred:
* Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io":
failed to call webhook: Post "https://root-ingress-controller-admission.tenant-root.svc:443/networking/v1/ingresses?timeout=10s":
dial tcp 10.96.33.42:443: connect: operation not permitted
```
### Root cause class
The outer tenant HelmReleases (`apps/tenant/templates/*.yaml`) render
one inner HR per subsystem and let flux reconcile them. Both `seaweedfs`
and `ingress` outer HRs get applied in parallel, and each of them spawns
its own inner HR — which flux then installs in parallel too.
When `seaweedfs-system` gets to the `creating 39 resource(s)` step, it
tries to create the tenant Ingress. The apiserver calls the
ingress-nginx `ValidatingWebhookConfiguration` (which uses
`failurePolicy: Fail` by default). If the nginx controller
pod/Service/endpoint state hasn't finished reconciling on the CNI yet,
the connect to the ClusterIP fails with `EPERM`, the webhook errors,
Helm aborts, and flux rolls back.
E2E recovers via the `flux reconcile --force` retry loop in
`hack/e2e-install-cozystack.bats`, but it's flaky and adds minutes.
### Fix
Introduce an explicit ordering dependency at the inner HR layer:
```yaml
{{- if $ingress }}
dependsOn:
- name: ingress-nginx-system
{{- end }}
```
Flux waits for `ingress-nginx-system` to reach `Ready=true` before
installing seaweedfs — which, under Flux's wait semantics, means the
ingress-nginx Deployment is Available and its webhook endpoint is
routable. The race disappears.
Guarded by `{{- if $ingress }}` so this only kicks in for tenants that
actually have ingress enabled.
## Test plan
- [ ] Rerun E2E on #2258 — `seaweedfs-system` should install on the
first try without a `Retry install` iteration.
- [ ] Verify rendered template still matches expectations when
`$ingress` is unset (no `dependsOn` block).
### Release note
```release-note
[seaweedfs] Make tenant seaweedfs wait for ingress-nginx to be Ready before creating its Ingress resource, avoiding a webhook race that intermittently failed install on fresh clusters.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved deployment sequencing so the system now waits for the Ingress
controller to be ready before creating or updating resources that
include Ingress. This prevents premature deployments and related
failures, reduces rollout errors, and provides more reliable,
predictable deployments for applications relying on Ingress resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review blocker on #2406: gate the dependsOn on
`eq $ingress .Release.Namespace` so sub-tenants inheriting ingress
from a parent namespace don't deadlock waiting for an HR that doesn't
exist locally. Also add explicit `namespace:` to match repo convention.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Without this, the outer tenant HelmReleases apply both seaweedfs and
ingress in parallel, which in turn creates their inner HelmReleases
(seaweedfs-system and ingress-nginx-system) in parallel too. When
seaweedfs-system tries to create its Ingress before the nginx admission
webhook's Service/endpoint state has finished reconciling on the CNI,
the apiserver's webhook call fails with:
failed calling webhook "validate.nginx.ingress.kubernetes.io":
dial tcp ...:443: connect: operation not permitted
and flux rolls back the install. E2E recovers via the "Retry install"
loop but it's flaky and adds minutes to every run.
Adding dependsOn: ingress-nginx-system on the inner seaweedfs HR makes
flux block the install until the ingress chart is Ready, which means
the controller Deployment is Available and its webhook endpoint is
routable before the first Ingress is created.
Guarded by {{- if \$ingress }} so this only kicks in for tenants that
have ingress enabled (i.e. tenant-root in the default topology).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Bumps the vendored `etcd-operator` chart from `v0.4.2` to `v0.4.3` and
wires the new backup machinery through the cozystack `Etcd` app API.
Upstream v0.4.3 ships two new CRDs — `EtcdBackup` (one-shot snapshot)
and `EtcdBackupSchedule` (periodic snapshots) — along with the
`EtcdCluster.spec.bootstrap` restore field, a new `OPERATOR_IMAGE` env
on the operator deployment, and a replacement of the deprecated
`gcr.io/kubebuilder/kube-rbac-proxy` image. The chart portion of this PR
is a clean re-vendor via `make update` — no local patches.
On top of the bump, `packages/extra/etcd` exposes a new optional
`backup.*` values block. When `backup.enabled=true`, the chart renders
an `EtcdBackupSchedule` referencing the platform `EtcdCluster/etcd` and
a `Secret/etcd-s3-creds` with the S3 credentials expected by the
backup-agent. The values field names (`destinationPath`, `endpointURL`,
`s3AccessKey`, `s3SecretKey`, `schedule`) mirror the existing `postgres`
backup pattern for consistency. The chart `fail`s at template render if
any of the four required S3 fields are empty or if `destinationPath` is
malformed (missing `s3://` prefix, `//` segments, no bucket).
Test coverage: 15 `helm-unittest` cases across
`tests/backup-secret_test.yaml` and
`tests/etcd-backup-schedule_test.yaml` (wired into `make test`, picked
up by `hack/helm-unit-tests.sh`), plus three bats cases in
`hack/e2e-apps/etcd.bats` covering the baseline `Etcd` app, the `backup:
{}` default-off path, and the full `backup.enabled=true` case that
verifies the `EtcdBackupSchedule` materializes and the operator
generates the downstream `CronJob`.
`packages/core/platform/sources/etcd-operator.yaml` now sets
`upgradeCRDs: CreateReplace` on the etcd-operator component, so Flux
applies the new CRDs on upgrade of the chart instead of silently
skipping them.
Depends on #2427.
### Scope deferred to follow-ups
- The one-shot `EtcdBackup` CRD is vendored but not exposed through this
chart — users can still hand-apply the CRD directly.
- `EtcdCluster.spec.bootstrap` (restore from an existing snapshot) is
likewise vendored but not exposed. Both gaps are called out in
`packages/extra/etcd/README.md`.
### Release note
```release-note
feat(etcd): update etcd-operator to v0.4.3 and expose S3 backup schedules on the Etcd app. Set `backup.enabled=true` along with `backup.s3AccessKey`, `backup.s3SecretKey`, `backup.destinationPath`, and `backup.endpointURL` to schedule periodic snapshots.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Scheduled S3 backups for etcd clusters (configurable cron, generated
S3 credentials) and restore/init from S3 or PVC via new backup
resources.
* **Tests**
* New end-to-end tests for etcd lifecycle and backup scenarios; added
Helm unit tests for backup templates and validations.
* **Documentation**
* Backup configuration guide and parameter reference added.
* **Chores**
* Operator chart version bump; RBAC, webhooks, and deployment wiring
updated; added chart test target.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The `filer` and `s3` blocks in the seaweedfs HelmRelease values omitted
the `replicas:` field, so user-supplied `.Values.filer.replicas` and
`.Values.s3.replicas` were silently ignored and the upstream chart
defaults took over. As a second-order effect, the `WorkloadMonitor`
resources for `filer` and `s3` (which read the same values) diverged
from the actual replica count and could trigger spurious alerts when
users set non-default values.
This PR mirrors the existing `master` / `volume` / `db` pattern and
interpolates `replicas:` into both blocks.
Fixes#2531
### Release note
```release-note
fix(seaweedfs): pass filer.replicas and s3.replicas to HelmRelease so user-supplied values take effect
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* SeaweedFS deployment configuration now supports configurable replica
counts for the filer and s3 components, providing flexible cluster
scaling through deployment parameters.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Limit single PUT request body size to 5GB for SeaweedFS S3 endpoints to
enforce multipart upload usage for large files.
## Motivation
Large monolithic file uploads via single PUT requests (files >5GB)
frequently cause issues in production environments:
- Connection timeouts during long uploads
- Memory pressure on proxy/gateway components
- Incomplete uploads resulting in corrupted data
- High support load from users experiencing these failures
AWS S3 and other object storage providers recommend using multipart
uploads for files larger than 100MB, with multipart being mandatory for
files >5GB.
## Changes
- Set `nginx.ingress.kubernetes.io/proxy-body-size: "5g"` in system
SeaweedFS configuration
- Set `nginx.ingress.kubernetes.io/proxy-body-size: "5g"` in tenant
SeaweedFS template (hardcoded, non-overridable)
- Add explanatory comments documenting the limitation and rationale
## Impact
**For users:**
- Single PUT requests are limited to 5GB maximum
- Files >5GB must be uploaded using S3 multipart upload API
- All standard S3 clients (aws-cli, boto3, MinIO client, etc.)
automatically use multipart for large files
- No breaking changes for properly configured S3 clients
**Platform benefits:**
- Reduced support load from large file upload failures
- Better resource utilization (memory, connections)
- More reliable upload experience for end users
- Aligns with industry best practices (AWS, MinIO, etc.)
## Testing
- Verify small files (<5GB) upload successfully via PUT
- Verify large files (>5GB) via PUT return HTTP 413
- Verify multipart upload works for any file size
- Test with common S3 clients (aws-cli, s3cmd, etc.)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated SeaweedFS S3 ingress configuration to enforce a 5GB maximum
for single requests; files larger than 5GB should use multipart upload.
* Disabled response/request buffering and extended NGINX timeouts, and
enabled upstream service handling to improve reliability of large
uploads and long-lived connections.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The filer and s3 blocks in the seaweedfs HelmRelease values omitted the
replicas: field, so user-supplied .Values.filer.replicas and
.Values.s3.replicas were silently ignored and the upstream chart defaults
took over. This also caused the WorkloadMonitor resources for filer and
s3 (which read the same values) to diverge from the actual replica count
and trigger spurious alerts when users set non-default values.
Mirror the master/volume/db pattern and interpolate replicas: into both
blocks.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds an opt-in `publishing.exposure=loadBalancer` mode for the
ingress-nginx Service as a migration path away from
`Service.spec.externalIPs`, which is deprecated upstream in Kubernetes
v1.36
([KEP-5707](https://github.com/kubernetes/enhancements/issues/5707),
[kubernetes#137293](https://github.com/kubernetes/kubernetes/pull/137293)).
The `AllowServiceExternalIPs` feature gate is expected to default to off
around v1.40 and the implementation to be removed around v1.43.
Stacked on top of #2464 (cilium v1.19.3 bump) — depends on it for the
`CiliumLoadBalancerIPPool` at `cilium.io/v2`.
### Behavior
- New platform value `publishing.exposure` — enum `externalIPs |
loadBalancer`, default `externalIPs` (current behavior unchanged on
upgrade).
- Plumbed through `cozystack-values` into each tenant's ingress
HelmRelease via the new `expose-mode` key.
- When `exposure=loadBalancer` and the current namespace matches
`publishing.ingressName`, the Service becomes `type: LoadBalancer` with
`externalTrafficPolicy: Local` and a `CiliumLoadBalancerIPPool`
announces the addresses from `publishing.externalIPs` via Cilium LB
IPAM.
- The pool uses a namespace-only `serviceSelector`
(`io.kubernetes.service.namespace: <ns>`) — any LoadBalancer Service in
the tenant namespace draws from it. See "Pool ownership" below.
- IPv4 addresses get `/32` CIDRs, IPv6 addresses get `/128`. Mixed
families supported. Pre-CIDR entries (`192.0.2.10/32`) are accepted
without double-suffixing.
- Unknown values, `loadBalancer` with empty externalIPs, and stray empty
entries from `publishing.externalIPs` are rejected at render time with
explicit error messages.
### Pool ownership
The `CiliumLoadBalancerIPPool` is rendered from
`packages/apps/tenant/templates/cilium-lb-pool.yaml`, not from the
ingress chart. The tenant chart is the per-tenant owner of cross-cutting
resources (Namespace, `cozystack-values` Secret, HelmReleases for
ingress and gateway), so one pool per tenant lives there.
Cilium LB IPAM rejects overlapping CIDRs across pools regardless of
`serviceSelector` — the last-added pool gets `cilium.io/PoolConflict`
and stops allocating. Keeping the pool in the ingress chart would
collide with the Gateway-API PR (#2470), which materialises its own
LoadBalancer Service from the same `publishing.externalIPs` range when
`tenant.spec.gateway=true`. Moving the pool to the tenant chart with a
namespace-only selector lets a single pool back both services
(ingress-nginx today, a Cilium Gateway Service once #2470 lands).
Only the ingress-loadBalancer signal is wired here
(`_cluster.expose-mode=loadBalancer` + `.Values.ingress=true` +
publishing tenant). #2470 rebases on top of this PR, drops its own
`packages/extra/gateway/templates/cilium-lb-pool.yaml`, and adds an OR
branch for `.Values.gateway` in the tenant template.
### Scope
Only the ingress-nginx Service is migrated by this setting. Other
cozystack components that still write `Service.spec.externalIPs`
directly (notably the `vpn` app at
`packages/apps/vpn/templates/service.yaml`) need separate follow-up
before Kubernetes v1.40.
### Tests
- `packages/extra/ingress/tests/exposure_test.yaml` — 10 helm-unittest
cases on the ingress-nginx Service: type / externalTrafficPolicy /
externalIPs assertion for both modes, unknown-mode rejection
(case-sensitive enum), empty-IPs failure, empty-entry filtering,
non-publishing-tenant fallback.
- `packages/apps/tenant/tests/exposure_test.yaml` — 9 cases on the pool
itself: IPv4, IPv6, mixed, pre-CIDR input, empty-entry filtering,
`ingress=false` in publishing tenant (no pool), non-publishing tenant
(no pool), empty externalIPs (no pool).
- Both suites are auto-discovered by `hack/helm-unit-tests.sh` via the
`test:` target in each package's Makefile.
### Caveats (copied from the inline `values.yaml` comment)
- `loadBalancer` mode uses `externalTrafficPolicy: Local`. The external
IP must already be routed to a node that hosts an ingress pod (floating
IP / upstream router / podAntiAffinity).
- Cilium does not announce the IP on its own unless L2 announcements or
BGP are enabled in the Cilium values (disabled by default in cozystack).
- Switching the value on a running cluster causes the ingress-nginx
Service to be recreated (`upgrade.force: true` on the HelmRelease +
Service kind change); expect a brief interruption of ingress traffic.
### Release note
```release-note
feat(ingress): add opt-in publishing.exposure=loadBalancer mode that uses type: LoadBalancer + CiliumLoadBalancerIPPool instead of the deprecated Service.spec.externalIPs. Default (externalIPs) preserves existing behavior.
```
## What this PR does
Removes the `post-upgrade` Helm hook in `packages/extra/etcd` that was
`kubectl delete`ing the etcd TLS chain (`etcd-ca-tls`,
`etcd-peer-ca-tls`,
`etcd-client-tls`, `etcd-peer-tls`, `etcd-server-tls`) and then deleting
etcd pods on every chart upgrade.
The hook was gated by a semver compare of a stored
`etcd-deployed-version`
ConfigMap against `2.6.1`. That gate was written when chart versions
looked
like `2.6.0`, `2.6.1`, etc. Since commit `f871fbdb` ("Remove
versions_map
logic") all chart versions are stamped as `0.0.0+<git-hash>`, which per
semver is always `< 2.6.1`, so the gate always evaluates to "regenerate
certs" and the destructive hook fires on every upgrade.
On clusters running Kamaji-managed tenant control planes the consequence
is
severe: wiping the etcd CA triggers cert-manager to re-issue it with a
brand-new CA, but Kamaji's `datastore-certificate` Secrets still carry
the
old CA bundle mounted into tenant `kube-apiserver` pods. Those
apiservers
hit `x509: certificate signed by unknown authority` against
`etcd.<ns>.svc:2379` and go into CrashLoopBackOff until each tenant
`DataStore` is individually force-reconciled and the Deployments rolled.
Commit `47d81f70` ("Disabled private key rotation in CA certs") already
fixed the underlying `rotationPolicy: Always` issue the hook was written
to
paper over in March 2025, so the migration has no remaining use.
Changes:
- Delete `templates/hook/{job,role,rolebinding,serviceaccount}.yaml`
- Delete `templates/version.yaml` (only the hook read
`etcd-deployed-version`)
- Add `tests/no-post-upgrade-hook_test.yaml` as a regression guard — if
`templates/hook/job.yaml` or `templates/version.yaml` is ever brought
back,
`make unit-tests` fails
### Release note
```release-note
fix(etcd): remove destructive post-upgrade cert-regeneration hook. Previously the etcd chart ran a `post-upgrade` hook on every Helm upgrade that deleted etcd TLS Secrets and pods, causing cert-manager to re-issue the etcd CA. On clusters with Kamaji-managed tenant control planes this put every tenant `kube-apiserver` into CrashLoopBackOff until each DataStore was manually re-reconciled. The hook was a one-shot `2.6.0 -> 2.6.1` migration that became a permanent footgun once chart versioning moved to `0.0.0+<git-hash>` and the underlying `rotationPolicy: Always` issue was fixed in commit `47d81f70`. This is behavior removal, not cosmetic cleanup.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added Helm chart test suite verifying absence of post-upgrade hook Job
and version ConfigMap.
* **Chores**
* Removed post-upgrade hook Job and associated RBAC resources (Role,
RoleBinding, ServiceAccount).
* Removed version ConfigMap.
* Added `test` target to Makefile for running Helm chart unit tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The pool is now rendered from packages/apps/tenant/templates/cilium-lb-pool.yaml instead of packages/extra/ingress/templates/cilium-lb-pool.yaml. Cilium LB IPAM forbids overlapping CIDRs across pools regardless of serviceSelector, so both the ingress-loadBalancer path and the upcoming per-tenant Gateway in #2470 cannot each own their own pool on the same publishing.externalIPs range. The tenant chart is the natural per-tenant owner — it already creates the Namespace, the cozystack-values Secret, and the HelmReleases for both ingress and gateway.
The new pool uses a namespace-only serviceSelector (io.kubernetes.service.namespace: <ns>), which matches any LoadBalancer Service in the tenant namespace. The metadata.name changed from <trim>-ingress to <trim>-exposure to reflect that the pool is not ingress-specific.
Only the ingress-loadBalancer signal is wired in this commit (_cluster.expose-mode=loadBalancer plus .Values.ingress=true on the publishing tenant). The gateway branch is added in #2470 on top of this commit — it rebases, drops its own packages/extra/gateway/templates/cilium-lb-pool.yaml, and adds an OR branch for .Values.gateway in the tenant template.
Pool-rendering unit tests moved from packages/extra/ingress/tests/ to packages/apps/tenant/tests/. The ingress chart tests keep the Service-level asserts. packages/apps/tenant/Makefile gains a test target so hack/helm-unit-tests.sh picks up the new suite.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from coderabbitai on
packages/extra/etcd/README.md:9: CRDs are schema definitions; what users
apply to trigger a restore or one-shot backup is a custom resource
instance of the EtcdBackup kind, not the CRD itself. Reword accordingly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on
packages/extra/etcd/templates/etcd-backup-schedule.yaml:20-21: explicitly
coerce successfulJobsHistoryLimit and failedJobsHistoryLimit to int so
string values passed via helm --set do not produce a type mismatch in
the EtcdBackupSchedule spec.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on
packages/extra/ingress/templates/cilium-lb-pool.yaml:19: if the operator
passes an externalIP already in CIDR form (192.0.2.10/32 or 2001:db8::1/128),
the template appended a second /32 or /128 suffix producing an invalid
CiliumLoadBalancerIPPool block. Guard the suffix append on the absence of
"/" in the input.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Service.spec.externalIPs is deprecated upstream in Kubernetes v1.36
(KEP-5707, kubernetes#137293). The AllowServiceExternalIPs feature gate
is expected to default to off around v1.40 and the implementation to
be removed around v1.43. For bare-metal installs that rely on
externalIPs today, cozystack needs a migration path.
This change adds an opt-in 'loadBalancer' exposure mode for the
ingress-nginx Service:
- New platform value 'publishing.exposure' (enum: externalIPs |
loadBalancer, default externalIPs). Plumbed through cozystack-values
into each tenant's ingress HelmRelease via the new 'expose-mode' key.
- Unknown values and loadBalancer with an empty externalIPs list fail
the chart render with explicit error messages, rather than silently
producing a broken Service.
- When exposure=loadBalancer and the current namespace matches
publishing.ingressName, the Service becomes type: LoadBalancer with
externalTrafficPolicy: Local.
- A new template renders a CiliumLoadBalancerIPPool whose blocks come
from publishing.externalIPs (IPv4 addresses get /32, IPv6 addresses
get /128) and whose serviceSelector uses Cilium's synthetic
io.kubernetes.service.namespace key combined with the standard
app.kubernetes.io/name: ingress-nginx label. No custom label is
written to the Service itself, avoiding cross-tenant collisions from
user-defined labels.
Default behaviour is unchanged: without opting in, the Service is
still ClusterIP + spec.externalIPs as today.
Scope: only ingress-nginx is migrated by this setting. Other cozystack
components that still write Service.spec.externalIPs directly (notably
the vpn app) must be migrated separately before the v1.40 feature gate
flip.
Tests: packages/extra/ingress/tests/exposure_test.yaml adds 13
helm-unittest cases covering both modes, IPv4/IPv6, empty-token
filtering, unknown-mode rejection, and the non-matching-namespace
fallback.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The etcd chart shipped a `post-upgrade` Helm hook that `kubectl delete`d the
etcd TLS chain (`etcd-{ca,peer-ca,client,peer,server}-tls`) and then deleted
etcd pods on every chart upgrade, gated by a semver compare of an
`etcd-deployed-version` ConfigMap against `2.6.1`.
The hook was added as a one-shot migration for the chart `2.6.0 -> 2.6.1`
transition. Since commit f871fbdb ("Remove versions_map logic") all chart
versions are stamped as `0.0.0+<git-hash>`, which per semver is always
`< 2.6.1`. The gate therefore always resolves to "update certs", firing the
destructive hook on every etcd upgrade. On clusters running Kamaji-managed
tenant control planes this wipes the etcd CA, cert-manager re-issues it, and
tenant kube-apiservers hit `x509: certificate signed by unknown authority`
against `etcd.<ns>.svc:2379` until each tenant DataStore is manually
re-reconciled.
Commit 47d81f70 ("Disabled private key rotation in CA certs") already fixed
the underlying `rotationPolicy: Always` issue the migration was papering
over, so the hook has no remaining use. Remove the hook Job, its RBAC, the
version ConfigMap it read, and add a helm-unittest suite under
`packages/extra/etcd/tests/` that guards against re-introducing the hook
or the version ConfigMap.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
ClusterIssuer solver referenced IngressClass "nginx" which does not
exist on cozystack clusters — real classes are named after tenant
namespaces (e.g. tenant-root). Cert issuance only worked because every
requesting Ingress overrode the ClusterIssuer via the legacy
acme.cert-manager.io/http01-ingress-class annotation.
Switch both sides to the modern cert-manager API (available since
cert-manager 1.12; cozystack ships 1.19.3):
- ClusterIssuer: http01.ingress.ingressClassName, value parameterized
from _cluster.expose-ingress (default "tenant-root")
- Ingress annotation: http01-ingress-ingressclassname
These must migrate together — mixing ingressClassName (ClusterIssuer)
with the old http01-ingress-class annotation triggers cert-manager's
"fields ingressClassName and class cannot be set at the same time"
validation and breaks issuance.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Run 'make generate' under packages/extra/etcd to refresh the JSON schema
and the corresponding ResourceDefinition YAML after adding backup and
resources fields. Matches cozyvalues-gen v1.3.0 output used by the
pre-commit CI check.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Fourth review round fixes.
Simplify backup validation by dropping placeholder-regex guards and switching default values of s3AccessKey, s3SecretKey, destinationPath, and endpointURL from angle-bracketed hints to empty strings. A single 'required' check on each field is now the whole validation for presence; the regex guard was brittle (silently stopped matching if the hint ever changed) and dropping it makes the values.yaml defaults match the behavior contract the README describes. s3:// prefix and '//' rejection still live in etcd-backup-schedule.yaml as template-time fail branches.
Add helm-unittest suites in packages/extra/etcd/tests/ covering both templates end-to-end: rendered output assertions for the happy path (bucket/key split, region passthrough, conditional rendering of optional fields) and failedTemplate assertions for every fail branch. A 'test' target is wired into the package Makefile so hack/helm-unit-tests.sh picks it up on 'make unit-tests'.
Add type: Opaque to backup-secret.yaml for consistency with the mongodb template.
E2E: wait for HelmRelease, Secret, and EtcdBackupSchedule deletion in setup() so the empty-backup case does not flake against lingering state from the previous test. Replace the fictitious minio-e2e hostname with no-such-endpoint.invalid and explicitly document that the e2e verifies schedule/CronJob materialization, not backup execution.
Restore via EtcdCluster.spec.bootstrap and the one-shot EtcdBackup CRD are still not exposed through this chart; both are explicit follow-ups called out in the README.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Use name='etcd' in all three e2e cases. The chart pins the Helm
release name to 'etcd' via check-release-name.yaml, so any other
name causes template rendering to fail before the HelmRelease
can reconcile. This is why the tenant chart hardcodes HR name
'etcd' in packages/apps/tenant/templates/etcd.yaml.
- Consolidate all backup validation guards into
etcd-backup-schedule.yaml so Helm's alphabetical render order no
longer determines which failure a user sees first. Previously, a
bad destinationPath would be masked by the earlier placeholder
check in backup-secret.yaml.
- Use regex match '^<.*>$' for placeholder detection instead of
exact string equality. The previous check silently stopped
firing if anyone updated the default placeholder string in
values.yaml; the regex form catches any angle-bracketed default.
- Reject destinationPath strings containing '//' after the s3://
prefix. Previously 's3://bucket//subdir' silently produced a key
beginning with '/', confusing S3 clients and retention policies.
- Document the backup contract in README.md: what enabling it
produces, which fields are required, why placeholders are
rejected, and that restore is a known follow-up gap.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Mark backup.enabled as optional in the values typedef so
generated JSON schema no longer lists it in 'required'.
Previously, 'backup: {}' (relying on the enabled:false default)
failed schema validation at CRD admission.
- Validate backup.destinationPath has s3:// prefix. Without this
check, passing a path without the scheme silently used the
first path segment as the bucket name.
- Add tail newline to values.schema.json to match repo convention
(cozyvalues-gen output lacks it — worth fixing in that project
separately).
- Add bats setup() that clears tenant-test etcd resources before
each test so a failure mid-test does not leak into subsequent
tests. Hardcoded chart-level resource names ('etcd',
'etcd-s3-creds') make two Etcd apps in the same namespace
collide; setup() enforces a single owner per test.
- Add dump_diagnostics() helper that dumps operator logs + the
relevant CRD state on wait-for-ready timeouts, so CI failures
surface actionable context instead of bare exit code 124.
- Add a third e2e case 'Create Etcd with empty backup block' that
asserts backup:{} admits and produces no backup resources — a
regression test for the required:['enabled'] bug fixed here.
Known scope: the upstream EtcdCluster.spec.bootstrap restore field
(also new in v0.4.3) is not exposed through this values.yaml yet.
Restoring a cluster from an existing snapshot is a follow-up PR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Review found three issues:
1. The guard in backup-secret.yaml only triggered on empty strings,
not on the placeholder defaults ('<your-access-key>' etc). A user
enabling backup.enabled=true without overriding the values would
produce a Secret containing the literal placeholder, and the
backup CronJob would fail at runtime with an opaque 403 far from
the source. Add explicit template guards that fail rendering when
any of the four required fields still hold their placeholder
default.
2. The backup e2e test waited on .spec.schedule of the newly-created
EtcdBackupSchedule, which is set at apply time — the wait was
tautological and did not prove the etcd-operator reconciled
anything. Replace it with a wait on the EtcdCluster's Ready
condition plus a poll for the CronJob the operator generates
from the schedule (selected via the
etcd.aenix.io/etcdbackupschedule-name label).
3. The backup e2e test also did not wait for EtcdCluster readiness
before tearing down. Add the same Ready wait as the baseline
test, and verify the Secret contains the real access key value
rather than just existing.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds two bats cases: a baseline Etcd managed-app deployment and a
deployment with backup.enabled=true that verifies the
EtcdBackupSchedule resource and the S3 credentials Secret are
materialized. The backup case uses fake S3 credentials — the
operator will not successfully push backups, but the schedule
resource itself is reconciled, which is what this test validates.
Also hardcodes the EtcdBackupSchedule and backup Secret names to
'etcd' / 'etcd-s3-creds' to match the existing hardcoded
EtcdCluster name in the chart (one etcd per tenant namespace).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Expose the EtcdBackupSchedule CRD (etcd.aenix.io/v1alpha1) through
optional values so operators can declaratively configure periodic
S3 backups of the platform etcd cluster. Kamaji stores tenant
control-plane etcd data here, so losing this cluster means losing
every tenant cluster — a backup schedule is critical.
The backup is disabled by default (backup.enabled: false), keeping
existing installations unchanged. When enabled, a Secret is created
with AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY keys referenced by
the new EtcdBackupSchedule resource. Field naming mirrors the
existing postgres backup pattern (destinationPath, endpointURL,
s3AccessKey, s3SecretKey, schedule) for consistency.
Requires etcd-operator v0.4.3+ (shipped in the previous commit).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add nginx ingress annotation to limit single PUT request body size to 5GB
for both system and tenant SeaweedFS S3 endpoints. This enforces the use
of S3 multipart upload API for files larger than 5GB, which prevents
issues commonly seen with large monolithic uploads and reduces support load.
Changes:
- Set proxy-body-size to "5g" in system seaweedfs values.yaml
- Set proxy-body-size to "5g" in tenant seaweedfs template
- Add explanatory comments for the limitation
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Without concurrencyPolicy and job limits, the defrag CronJob can
accumulate hundreds of running/failed pods during cluster upgrades
when etcd is temporarily unavailable. This was observed after
upgrading to v1.1.2 where defrag jobs piled up across tenants.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## Summary
- Adds `inlineScrapeConfig` support to the tenant `VMAgent` resource in
`packages/system/monitoring/templates/vm/vmagent.yaml`
- Adds commented usage example in both
`packages/system/monitoring/values.yaml` and
`packages/extra/monitoring/values.yaml`
## Problem
The tenant VMAgent resource does not support custom scrape
configurations. Using `additionalScrapeConfigs` (which references a
Kubernetes Secret) is not viable because tenant users have no access to
create or read Secrets — they can only manage resources under
`apps.cozystack.io`. Instead, `inlineScrapeConfig` accepts inline YAML
directly through Helm values, which is consistent with tenant
permissions.
Relates to #2194
## Usage
```yaml
vmagent:
inlineScrapeConfig: |
- job_name: "custom"
static_configs:
- targets: ["my-service:9090"]
```
## Test plan
- [ ] `helm template` monitoring with `inlineScrapeConfig` set — block
rendered
- [ ] `helm template` monitoring without it — no `inlineScrapeConfig` in
output
- [ ] Deploy and verify custom scrape targets are picked up by vmagent
```release-note
Add inlineScrapeConfig support to tenant vmagent
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Support for including an inline scrape configuration into VMAgent
monitoring when provided.
* **Documentation**
* Added a commented example showing how to supply an inline scrape job
and targets.
* Reordered remote-write URL entries in monitoring configuration for
clearer ordering.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Add external-dns as a standalone self-managed application in
`packages/extra/external-dns/`, allowing tenants to deploy and configure
their own DNS management directly from the dashboard.
## Motivation
Tenants need the ability to manage their own DNS domains with their own
provider. Following the [developers
guide](https://github.com/cozystack/website/pull/413), this is
implemented as an extra package (like `ingress` and `seaweedfs`) using
the HelmRelease-based pattern, rather than embedding it in the tenant
chart.
This enables multi-tenant scenarios where:
- Tenant A uses Cloudflare for `domain-a.com`
- Tenant B uses AWS Route53 for `domain-b.com`
- Each tenant deploys and manages external-dns independently from the
dashboard
## Changes
- **New package**: `packages/extra/external-dns/` — standalone
HelmRelease-based application
- **New PackageSource**:
`packages/core/platform/sources/external-dns-application.yaml` —
references `system/external-dns` and `extra/external-dns` components
- **Cleaned tenant chart**: removed the previously embedded
`externalDns` block from `packages/apps/tenant/`
## Features
- Support for 9 DNS providers: cloudflare, aws, azure, google,
digitalocean, linode, ovh, exoscale, godaddy
- Per-provider credential configuration with full JSON schema validation
- Domain filtering via `domainFilters`
- Configurable sync policy (`sync` or `upsert-only`)
- Namespaced operation (`namespaced: true`) for tenant isolation
- Unique `txtOwnerId` per namespace to prevent DNS record conflicts
- Resource sizing via presets or explicit CPU/memory
## Usage Example
Deploy from the dashboard, or via values:
```yaml
# Cloudflare
provider: cloudflare
domainFilters:
- example.com
cloudflare:
apiToken: "your-cloudflare-api-token"
```
```yaml
# AWS Route53
provider: aws
domainFilters:
- example.org
aws:
accessKeyId: "AKIAXXXXXXXX"
secretAccessKey: "your-secret-key"
region: "us-east-1"
```
## Test plan
- [ ] `helm template external-dns packages/extra/external-dns/ --set
provider=cloudflare --set cloudflare.apiToken=test` renders correctly
- [ ] `helm template external-dns packages/extra/external-dns/` fails
(provider required)
- [ ] `helm template wrong-name packages/extra/external-dns/ --set
provider=cloudflare` fails (release name check)
- [ ] Deploy external-dns from tenant dashboard
- [ ] Verify HelmRelease is created in tenant namespace with namespaced
RBAC
- [ ] Create an Ingress and verify DNS record is created
- [ ] Verify no conflict with global external-dns instance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an External DNS package for automatic DNS record management.
* Support for 9 DNS providers: Cloudflare, AWS, Azure, Google,
DigitalOcean, Linode, OVH, Exoscale, GoDaddy.
* Helm-based deployment with namespaced/system variants and release
configuration options.
* Configurable synchronization policies, domain filtering, provider
credentials, extra args, and resource presets.
* **Documentation**
* New README and schema-driven values documentation for installation and
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace deprecated single-node VLogs CR with VLCluster (cluster mode)
for reliability and horizontal scalability.
Changes:
- Replace VLogs (v1beta1) with VLCluster (v1) using vlinsert/vlselect/vlstorage
- Update fluent-bit outputs to vlinsert-generic:9481
- Update Grafana datasource to vlselect:9471
- Update ExternalName service from vlogs-generic to vlinsert-generic
- Add VPA for all VLCluster components
- Update WorkloadMonitors for three-component architecture
- Add migration 35 to preserve old VLogs resources with keep annotation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Combines and unifies COSI enhancements across seaweedfs and bucket
charts:
**SeaweedFS (extra + system charts):**
- Rename storage pool BucketClass suffix from `-worm` to `-lock`
- Rename parameter `disk` to `diskType` for consistency with COSI driver
- Reduce default object lock retention from 36500 to 365 days
- Add `-lock` BucketClass (COMPLIANCE mode, 365 days) for client and
system topologies
- Add `-readonly` BucketAccessClass with explicit `accessPolicy` for all
topologies
- Add explicit `accessPolicy: readwrite` on default BucketAccessClass
- Update pool name validation to reject `-lock` suffix (was `-worm`)
**Bucket app:**
- Add `locking` parameter: provisions from `-lock` BucketClass
- Add `storagePool` parameter: selects pool-specific BucketClass
- Replace hardcoded BucketAccess with `users` map — each entry creates a
BucketAccess with optional `readonly` flag
- Update dashboard RBAC to dynamically list user credential secrets
- Update ApplicationDefinition schema with new properties
**Breaking change:** empty `users: {}` (default) produces zero
BucketAccess resources. Existing buckets that relied on the implicit
default BucketAccess will need to define users explicitly.
### Release note
```release-note
[apps] Add locking, storagePool, and users configuration to bucket app; rename COSI BucketClass suffix from -worm to -lock
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Bucket locking with a shorter retention option, storage-pool
selectable bucket classes, and per-user access (per-user BucketAccess
and readonly controls)
* S3 Manager login mode: user login/logout, per-session credentials, and
new login UI
* **Behavior Changes**
* Credential handling changed to per-user secrets/label selection;
previously generated secrets removed; Ingress basic auth annotations
removed
* **Documentation**
* Added parameters: locking, storagePool, users (including per-user
readonly)
* **Updates**
* Updated COSI driver and S3 manager images
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The seaweedfs-cosi-driver v0.3.0 expects the parameter key 'disk',
not 'diskType'. Restore the correct key to match the driver's
paramDisk constant.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add COSI resources for object locking and read-only access to both
client topology and system chart:
- BucketClass with -lock suffix (COMPLIANCE mode, 365 days retention)
- BucketAccessClass with -readonly suffix
- Explicit accessPolicy: readwrite on default BucketAccessClass
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Disabled private key rotation in every CA cert in cozystack system packages to prevent trustchain problems when CA cert reissued
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Disabled private-key rotation (set rotationPolicy: Never) for CA/root
certificates used by multiple system components (ingress-nginx, linstor,
linstor-scheduler, seaweedfs, victoria-metrics-operator,
kubeovn-webhook, lineage-controller-webhook, cozystack-api, etcd,
linstor API/internal, seaweedfs).
* Added patch application steps to relevant update workflows to ensure
the certificate template changes are applied during chart/update
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Redesign storage pools architecture:
- Move storagePools map from top-level into volume.pools (Simple topology)
and volume.zones[name].pools (MultiZone topology)
- Add nodeSelector, storageClass, dataCenter overrides for zones
- Add reserved suffix validation (-worm, -readonly) for pool names
- Block volume.pools usage in MultiZone (must use zone.pools instead)
- Use ternary/hasKey pattern for all optional replicas to handle 0 correctly
- Fix nodeSelector rendering for multiline values using indent
- Use disk: parameter (not diskType:) for COSI driver v0.3.0 BucketClass
- Bump seaweedfs-cosi-driver tag to v0.3.0
- Add minimum: 1 constraint for volume/zone/pool replicas in schema
- Regenerate README, CRD, and openAPISchema
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Andrey Kolkov
