The upload source unconditionally set the
cdi.kubevirt.io/storage.bind.immediate.requested annotation, which
bypasses WaitForFirstConsumer and may provision the DataVolume on a
node that does not satisfy the VM's node affinity constraints.
With the dependencyMappings feature ensuring VMInstance HelmRelease
depends on VMDisk HelmRelease, WFFC binding is triggered by the VM
pod itself — the annotation is no longer needed.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Fixes#2514.
When `external: true` is set on a non-sharded `MongoDB`, Cozystack
rendered a single
`LoadBalancer` Service whose selector matched **all** replica-set
members
(`app.kubernetes.io/replset: rs0`). Round-robin balancing sent ~⅔ of
writes to
secondaries and they were rejected with `MongoServerError: not primary`.
Drivers couldn't recover on their own either: connecting through the LB
returned
the in-cluster DNS names of all members from `isMaster`, none of which
resolve
from outside the cluster, so automatic primary discovery failed.
This PR replaces the manual `external-svc.yaml` (for replica-set mode)
with the
Percona MongoDB operator's native `replsets[].expose` feature:
```yaml
replsets:
- name: rs0
expose:
enabled: true
type: LoadBalancer
externalTrafficPolicy: Local
```
The operator then:
- creates one `LoadBalancer` Service **per pod** (`<release>-rs0-0`,
`-1`, `-2`)
with `statefulset.kubernetes.io/pod-name` selectors,
- watches `Service.status.loadBalancer.ingress` for each pod and
rewrites the
member `host` in `rs.conf` to the external IPs.
External clients use a standard replica-set URI
(`mongodb://...:27017,...:27017,...:27017/?replicaSet=rs0`) and the
driver
discovers the primary correctly from outside the cluster.
For **sharded** mode the existing single `LoadBalancer` is preserved —
it
front-ends stateless `mongos`, which can route writes to the right shard
primary itself. `external-svc.yaml` is now scoped to that case only.
### Tested
- `helm unittest .` in `packages/apps/mongodb/`: 91/91 pass.
- Applied on a dev stand:
- `replsets[0].expose` is set as expected in the rendered PSMDB CR.
- The Percona operator (`v1.21.1`) accepts the spec and creates 3
per-pod
`LoadBalancer` Services with `statefulset.kubernetes.io/pod-name`
selectors. The previous single round-robin Service is no longer
rendered.
- With an LB IP pool present, the operator promotes the allocated LB IPs
into
the PSMDB `status.host` field and into `rs.conf`.
End-to-end mongosh write loop wasn't run on the dev stand because the
underlying network does not advertise/route allocated LoadBalancer IPs
(no
working L2/BGP path), so the rewritten `rs.conf` members are unreachable
in
that environment. The structural change is otherwise fully verified and
the
issue's reproduction scenario expects a real LB-capable network.
### Release note
```release-note
fix(mongodb): expose each replica-set member through its own LoadBalancer when `external: true`, so external clients can discover the primary via the standard replica-set connection string and writes no longer fail with "not primary" errors.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* External LoadBalancer services now only render when both external and
sharding are enabled.
* Replica set external exposure is now conditional based on the external
setting.
* Service configuration now uses LoadBalancer type with Local traffic
policy for external access.
* Service selectors refined for proper mongos targeting in sharded
deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Changes the default `resourcesPreset` for both `kafka` and `zookeeper`
in the Kafka managed application from `small` (1 CPU / 512Mi) to
`medium` (1 CPU / 1Gi).
The `small` preset is no longer sufficient for the current Strimzi 0.45
/ Kafka 3.9 image baseline:
- Strimzi sets JVM heap to 50% of the container memory limit for Kafka
and 75% for ZooKeeper. With a 512Mi limit that leaves only 256Mi (Kafka)
or 128Mi (ZooKeeper) for the JVM non-heap.
- Actual non-heap usage in the Kafka 3.9 image (Metaspace, code cache,
direct buffers, JMX javaagent, embedded Jetty admin server for
`/v1/ready` and KRaft migration endpoints, thread stacks) systematically
exceeds those budgets at startup, triggering a cgroup OOM kill (exit
137) before the broker can connect to ZooKeeper.
Result on a fresh deployment with the previous default: every Kafka
broker and every ZooKeeper pod ends up in `CrashLoopBackOff`, with
`lastState.terminated.reason: OOMKilled`. ZooKeeper occasionally hits
`OOMKilled` once quorum traffic ramps up, breaking the quorum and
causing Kafka brokers to fail with `ZooKeeperClientTimeoutException`.
The `medium` preset (1Gi) is the smallest preset that gives both Kafka
(heap 512Mi + ~500Mi non-heap budget) and ZooKeeper (heap 768Mi + 256Mi
non-heap budget) enough headroom to start cleanly. Users who need more
can still pick `large`, `xlarge`, or `2xlarge`, or override `resources`
explicitly.
The `small` preset is intentionally kept in the enum so existing
deployments that pin `resourcesPreset: small` are not broken; only the
default changes.
### Release note
```release-note
fix(kafka): default `resourcesPreset` for both Kafka and ZooKeeper is now `medium` (1 CPU / 1Gi). The previous default (`small`, 512Mi) is no longer sufficient under the current Strimzi 0.45 / Kafka 3.9 image and caused OOMKilled CrashLoopBackOff on fresh deployments. Existing deployments that explicitly set `resourcesPreset` are unaffected.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated default resource allocation preset from "small" to "medium"
for Kafka and ZooKeeper components across CRD type definitions, Helm
chart values, and JSON schema configurations.
* **Documentation**
* Updated parameter reference tables to reflect the new default resource
preset values for both Kafka and ZooKeeper.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- `cozy-harbor/templates/bucket-secret.yaml` no longer relies on
`lookup` against the BucketAccess credentials Secret. It is now driven
by `.Values.bucket.bucketInfo` (a JSON string) with `dig`-based safe
accessors, so a missing/empty/partial value renders nothing rather than
crashing with `index of untyped nil` (the failure mode that broke harbor
in CI #25049445125).
- The downstream `<release>-system` HelmRelease now sources `BucketInfo`
via `valuesFrom` (`valuesKey: BucketInfo`, `targetPath:
bucket.bucketInfo`). With the default `optional: false`, helm-controller
refuses to compose values until the COSI BucketAccess controller has
populated the Secret — both gating initial reconciliation and forcing a
config-digest change (and thus a helm upgrade) when the credentials
arrive. Flux HR `dependsOn` cannot reference COSI resources directly, so
`valuesFrom` is the correct primitive here.
- `bucket.secretName` was the only consumer of the old `lookup` and is
removed from both the system chart's `values.yaml` default and the apps
chart's `values:` block.
## Why this shape
helm-controller's upgrade trigger is digest-based over composed values +
chart artifact; a `lookup` returning new data on a later reconcile is
not enough to force an upgrade on its own. Routing `BucketInfo` through
`valuesFrom` puts the bucket data on the digest path, so the chart
re-applies the moment credentials become available.
## Test plan
- [x] `helm template` of `packages/system/harbor` with
`bucket.bucketInfo` unset, empty string, empty JSON object `{}`, and
fully populated — only the populated case renders the `*-registry-s3`
Secret; the others render nothing without erroring.
- [x] `helm template` of `packages/apps/harbor` produces the expected
`<release>-system` HR with the new `valuesFrom` entry and the cleaned-up
`values:` block.
- [ ] Re-run the harbor E2E on a fresh stand and confirm 15/15.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Harbor bucket config now comes from a centralized Secret merged into
chart values; in-values bucket entry removed.
* **Bug Fixes**
* Deployment synchronization and Helm upgrade behavior improved by
depending on the Secret’s readiness/contents.
* **Tests**
* Added unit tests covering when the bucket secret is rendered (or not)
and a make target to run those tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds the upstream Percona finalizer pair to the `PerconaServerMongoDB`
custom resource so the operator performs an ordered shutdown and
reclaims persistent storage when a MongoDB release is uninstalled.
Previously, deleting a MongoDB application left PVCs and
operator-managed secrets orphaned in the tenant namespace, requiring
manual cleanup before the same release name could be reused.
The finalizers added on the CR:
- `percona.com/delete-psmdb-pods-in-order` — ensures the primary is shut
down last for graceful cluster termination.
- `percona.com/delete-psmdb-pvc` — instructs the operator to delete PVCs
and operator-managed secrets when the CR is removed.
Coverage applies to all replica sets in both replicaset and sharded
modes, including the config server in sharded deployments.
### Behavior change
Users who previously relied on orphaned PVCs as an implicit "soft
delete" for data recovery should be aware that uninstalling a MongoDB
release now removes the underlying storage and operator-managed
credentials. The README's new "Data lifecycle" section enumerates
exactly which resources are reclaimed by the operator versus by the
standard `helm uninstall`. Take a backup before deletion if data
retention is required.
This resolves cozystack/cozystack#2513.
### Screenshots
Not applicable — no UI changes.
### Release note
```release-note
fix(mongodb): reclaim PVCs and operator-managed secrets on uninstall via Percona finalizers.
WARNING: upgrading scopes the admin credentials secret per release instead of the namespace-shared default. This triggers an in-place rotation of the five operator-managed admin passwords on first reconcile. Pods are not restarted; application user passwords are unaffected. Workloads consuming `<release>-credentials` must be restarted to pick up the new admin passwords.
```
### Out of scope / follow-up
- Bats integration test would require a live cluster with the Percona
MongoDB operator and belongs in the `hack/e2e-*.bats` family — left as a
follow-up.
- Issue #2513 mentions considering similar treatment for
PostgreSQL/MariaDB/Redis/ClickHouse. Each operator has a different
cleanup mechanism; that work is intentionally out of scope here and
should be tracked per-app.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Expanded uninstall and upgrade guidance: which secrets/pvc remain,
cert-manager TLS secrets must be deleted manually, remediation for
orphaned legacy secret, and recovery steps for stuck deletions when the
operator is removed before CRs.
* **New Features**
* Added deletion finalizers to enforce controlled teardown.
* User credentials secret now uses a deterministic, release-scoped name.
* **Tests**
* Added chart tests for finalizer behavior (sharded/non‑sharded) and
release-scoped users secret.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**What this PR does**
Adds configurable kubelet resource reservations and eviction thresholds
for tenant cluster worker nodes. Without explicit reservations and
cgroup enforcement, the scheduler treats nearly all node memory and CPU
as allocatable. Under pressure, the OOM killer can target kubelet itself
because no cgroup boundaries protect it, leading to "Kubelet stopped
posting node status" failures and unrecoverable NotReady nodes.
This PR auto-computes per-node-group kubelet reservations from the
effective node capacity (instanceType or explicit `resources`):
- **Memory**: `systemReservedMemory` and `kubeReservedMemory` each
default to 5% of effective memory, clamped 256Mi-1Gi
- **CPU**: `systemReservedCpu` and `kubeReservedCpu` each default to 5%
of effective CPU, clamped 50m-500m
- **Eviction**: configurable `evictionHardMemory` (default 7%) and
`evictionSoftMemory` (default 10%) with validation that hard < soft
- **Disk eviction**: `nodefs.available`, `imagefs.available`, and
`nodefs.inodesFree` thresholds are preserved alongside memory signals
- **Capacity annotations**: cluster-autoscaler annotations now report
allocatable memory and CPU (total minus all reservations and eviction
threshold) instead of raw totals
- **Cgroup enforcement**: `enforce-node-allocatable=pods` configures
advisory-only reservations — the scheduler accounts for reserved
resources when computing allocatable capacity, but no cgroup boundaries
are enforced for system or kube reserved resources
- **Validation**: format checks, zero-value rejection, eviction ordering
(hard < soft), same-unit-type requirement for eviction thresholds, and
total reservation sanity check against effective node capacity
All reservation fields are optional and overridable per node group via
`.nodeGroups[].kubelet`.
**Default rendered kubeletExtraArgs:**
```yaml
kubeletExtraArgs:
system-reserved: "memory=256Mi,cpu=50m"
kube-reserved: "memory=256Mi,cpu=50m"
eviction-hard: "memory.available<7%,nodefs.available<10%,imagefs.available<15%,nodefs.inodesFree<5%"
eviction-soft: "memory.available<10%,nodefs.available<15%,imagefs.available<20%"
eviction-soft-grace-period: "memory.available=1m30s,nodefs.available=1m30s,imagefs.available=1m30s"
eviction-minimum-reclaim: "memory.available=256Mi"
enforce-node-allocatable: "pods"
```
**New values schema:**
```yaml
nodeGroups:
md0:
kubelet:
systemReservedMemory: "" # auto-computed (5% of effective memory, 256Mi-1Gi)
kubeReservedMemory: "" # auto-computed (5% of effective memory, 256Mi-1Gi)
systemReservedCpu: "" # auto-computed (5% of effective CPU, 50m-500m)
kubeReservedCpu: "" # auto-computed (5% of effective CPU, 50m-500m)
evictionHardMemory: "7%" # absolute (e.g. 200Mi) or percentage
evictionSoftMemory: "10%" # must be > evictionHardMemory, same unit type
```
**Release note:**
```
feat(kubernetes): add per-node-group kubelet resource reservations with auto-computed system/kube reserved memory (5% of effective memory, clamped 256Mi-1Gi) and CPU (5% of effective CPU, clamped 50m-500m), configurable memory eviction thresholds, preserved disk eviction signals (nodefs, imagefs, inodesFree), pods-only cgroup enforcement, and reservation-aware allocatable capacity annotations for cluster-autoscaler.
```
## What this PR does
Closes the 25 open Dependabot Go-ecosystem alerts on this repository (2
critical, 11 high, 12 moderate) by bumping the affected dependencies in
five Go modules to their latest stable versions. Splits the work into
six signed commits, one per logical block, so the diff in each commit
can be reviewed in isolation.
### Full advisory coverage
| # | Module | Severity | Package | Advisory | Title | Vulnerable |
Shipped |
|---|---|---|---|---|---|---|---|
| 29 | `go.mod` | critical | `google.golang.org/grpc` |
[GHSA-p77j-4mvh-x3m3](https://github.com/advisories/GHSA-p77j-4mvh-x3m3)
| gRPC-Go authorization bypass via missing leading slash in `:path` | `<
1.79.3` | `1.80.0` |
| 28 | `kubevirt-csi-driver/go.mod` | critical |
`google.golang.org/grpc` |
[GHSA-p77j-4mvh-x3m3](https://github.com/advisories/GHSA-p77j-4mvh-x3m3)
| gRPC-Go authorization bypass via missing leading slash in `:path` | `<
1.79.3` | `1.80.0` |
| 32 | `go.mod` | high | `go.opentelemetry.io/otel` |
[GHSA-mh2q-q3fh-2475](https://github.com/advisories/GHSA-mh2q-q3fh-2475)
| Multi-value `baggage` header extraction causes excessive allocations
(remote DoS amplification) | `>= 1.36.0, <= 1.40.0` | `1.43.0` |
| 31 | `go.mod` | high | `github.com/moby/spdystream` |
[GHSA-pc3f-x583-g7j2](https://github.com/advisories/GHSA-pc3f-x583-g7j2)
| SpdyStream: DoS on CRI | `<= 0.5.0` | `0.5.1` |
| 30 | `go.mod` | high | `go.opentelemetry.io/otel/sdk` |
[GHSA-hfvc-g4fc-pqhx](https://github.com/advisories/GHSA-hfvc-g4fc-pqhx)
| BSD `kenv` command not using absolute path enables PATH hijacking |
`>= 1.15.0, <= 1.42.0` | `1.43.0` |
| 27 | `go.mod` | high | `go.opentelemetry.io/otel/sdk` |
[GHSA-9h8m-3fm2-qjrq](https://github.com/advisories/GHSA-9h8m-3fm2-qjrq)
| OpenTelemetry Go SDK vulnerable to arbitrary code execution via PATH
hijacking | `>= 1.21.0, < 1.40.0` | `1.43.0` |
| 26 | `kubevirt-csi-driver/go.mod` | high |
`github.com/opencontainers/runc` |
[GHSA-cgrx-mc8f-2prm](https://github.com/advisories/GHSA-cgrx-mc8f-2prm)
| runc container escape and DoS due to arbitrary write gadgets and
procfs write redirects | `<= 1.2.7` | `1.4.2` |
| 25 | `kubevirt-csi-driver/go.mod` | high |
`github.com/opencontainers/runc` |
[GHSA-qw9x-cqr3-wc7r](https://github.com/advisories/GHSA-qw9x-cqr3-wc7r)
| runc container escape with malicious config due to `/dev/console`
mount and related races | `>= 1.0.0-rc3, <= 1.2.7` | `1.4.2` |
| 24 | `kubevirt-csi-driver/go.mod` | high |
`github.com/opencontainers/runc` |
[GHSA-9493-h29p-rfm2](https://github.com/advisories/GHSA-9493-h29p-rfm2)
| runc container escape via "masked path" abuse due to mount race
conditions | `<= 1.2.7` | `1.4.2` |
| 23 | `kubevirt-csi-driver/go.mod` | high | `golang.org/x/oauth2` |
[GHSA-6v2p-p543-phr9](https://github.com/advisories/GHSA-6v2p-p543-phr9)
| Improper Validation of Syntactic Correctness of Input | `< 0.27.0` |
`0.36.0` |
| 16 | `kubeovn-webhook/go.mod` | high | `golang.org/x/oauth2` |
[GHSA-6v2p-p543-phr9](https://github.com/advisories/GHSA-6v2p-p543-phr9)
| Improper Validation of Syntactic Correctness of Input | `< 0.27.0` |
`0.36.0` |
| 4 | `kubeovn-webhook/go.mod` | high | `golang.org/x/net` |
[GHSA-4374-p667-p6c8](https://github.com/advisories/GHSA-4374-p667-p6c8)
| HTTP/2 rapid reset can cause excessive work in `net/http` | `< 0.17.0`
| `0.53.0` |
| 3 | `kubeovn-webhook/go.mod` | high | `golang.org/x/net` |
[GHSA-vvpx-j8f3-3w6h](https://github.com/advisories/GHSA-vvpx-j8f3-3w6h)
| `golang.org/x/net` vulnerable to Uncontrolled Resource Consumption |
`< 0.7.0` | `0.53.0` |
| 22 | `kubevirt-csi-driver/go.mod` | medium | `golang.org/x/net` |
[GHSA-vvgc-356p-c3xw](https://github.com/advisories/GHSA-vvgc-356p-c3xw)
| `golang.org/x/net` vulnerable to Cross-site Scripting | `< 0.38.0` |
`0.53.0` |
| 21 | `kubevirt-csi-driver/go.mod` | medium | `golang.org/x/net` |
[GHSA-qxp5-gwg8-xv66](https://github.com/advisories/GHSA-qxp5-gwg8-xv66)
| HTTP Proxy bypass using IPv6 Zone IDs | `< 0.36.0` | `0.53.0` |
| 20 | `kubevirt-csi-driver/go.mod` | medium |
`github.com/opencontainers/runc` |
[GHSA-jfvp-7x6p-h2pv](https://github.com/advisories/GHSA-jfvp-7x6p-h2pv)
| runc can be confused to create empty files/directories on the host |
`< 1.1.14` | `1.4.2` |
| 19 | `go.mod` | medium | `golang.org/x/crypto` |
[GHSA-f6x5-jh6r-wrfv](https://github.com/advisories/GHSA-f6x5-jh6r-wrfv)
| `golang.org/x/crypto/ssh/agent` vulnerable to panic if message is
malformed (OOB read) | `< 0.45.0` | `0.50.0` |
| 18 | `go.mod` | medium | `golang.org/x/crypto` |
[GHSA-j5w8-q4qc-rx2x](https://github.com/advisories/GHSA-j5w8-q4qc-rx2x)
| `golang.org/x/crypto/ssh` allows an attacker to cause unbounded memory
consumption | `< 0.45.0` | `0.50.0` |
| 14 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-vvgc-356p-c3xw](https://github.com/advisories/GHSA-vvgc-356p-c3xw)
| `golang.org/x/net` vulnerable to Cross-site Scripting | `< 0.38.0` |
`0.53.0` |
| 11 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-qxp5-gwg8-xv66](https://github.com/advisories/GHSA-qxp5-gwg8-xv66)
| HTTP Proxy bypass using IPv6 Zone IDs | `< 0.36.0` | `0.53.0` |
| 9 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-xrjj-mj9h-534m](https://github.com/advisories/GHSA-xrjj-mj9h-534m)
| `golang.org/x/net/http2` vulnerable to possible excessive memory
growth | `< 0.4.0` | `0.53.0` |
| 8 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-4v7x-pqxf-cx7m](https://github.com/advisories/GHSA-4v7x-pqxf-cx7m)
| `net/http`, `x/net/http2`: close connections when receiving too many
headers | `< 0.23.0` | `0.53.0` |
| 7 | `kubeovn-webhook/go.mod` | medium | `google.golang.org/protobuf` |
[GHSA-8r3f-844c-mc37](https://github.com/advisories/GHSA-8r3f-844c-mc37)
| `protojson.Unmarshal` infinite loop on certain forms of invalid JSON |
`< 1.33.0` | `1.36.11` |
| 6 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-qppj-fm5r-hxr3](https://github.com/advisories/GHSA-qppj-fm5r-hxr3)
| HTTP/2 Stream Cancellation Attack | `< 0.17.0` | `0.53.0` |
| 5 | `kubeovn-webhook/go.mod` | medium | `golang.org/x/net` |
[GHSA-2wrh-6pvc-2jm9](https://github.com/advisories/GHSA-2wrh-6pvc-2jm9)
| Improper rendering of text nodes in `golang.org/x/net/html` | `<
0.13.0` | `0.53.0` |
Sorted by severity (critical → high → medium), then by alert number. The
`Module` column lists the manifest path inside the repo (`go.mod` is the
root); `Vulnerable` is what GitHub flagged in the original advisory;
`Shipped` is the version this PR pins to.
### Modules bumped
1. Root `go.mod` (`github.com/cozystack/cozystack`) — purely transitive
bumps for `grpc`, `otel`, `otel/sdk`, `x/crypto`, `spdystream`.
2. `api/apps/v1alpha1/go.mod` — bumps `k8s.io/apimachinery` and
`golang.org/x/net`/`x/text` for consistency with the root module's `go
1.26.2` directive.
3. `packages/apps/kubernetes/images/kubevirt-csi-driver/go.mod` — bumps
the upstream `kubevirt.io/csi-driver` pin to its latest commit, raises
the `k8s.io/*` replace block from `v0.31.4` to `v0.36.0`, bumps `runc`
to `v1.4.2`, and adapts `main.go` to the upstream's new builder-style
API for `service.NewKubevirtCSIDriver` (the constructor was split into
`WithIdentityService` / `WithControllerService` / `WithNodeService`).
4. `packages/system/kubeovn-webhook/images/kubeovn-webhook/go.mod` —
direct bumps to `golang.org/x/net`, `golang.org/x/oauth2`,
`google.golang.org/protobuf`.
5. `packages/system/dashboard/images/token-proxy/go.mod` — bumps `jwx`,
`httprc`, `x/crypto`, `x/sys` for consistency with the root module's
directive.
All five modules now declare `go 1.26.2`. The three vendored-image
Dockerfiles (`kubevirt-csi-driver`, `kubeovn-webhook`, `token-proxy`)
move to `golang:1.26{,-alpine}`.
### First-party Dockerfile bumps
The root `go.mod` directive jump to `1.26.2` means every Dockerfile that
copies the root `go.mod` and builds a cozystack-owned binary needs a
matching base image, otherwise the in-container toolchain has to pull
`go 1.26` on the fly via `GOTOOLCHAIN=auto` at every build. Eight images
are bumped from `golang:1.25-alpine` to `golang:1.26-alpine`:
`cozystack-operator`, `cozystack-api`, `cozystack-controller`,
`backup-controller`, `backupstrategy-controller`, `flux-plunger`,
`kubeovn-plunger`, `lineage-controller-webhook`.
Vendored Dockerfiles (`kubevirt-cloud-provider`, `redis-operator`,
`keycloak-operator`, `linstor-csi`, `multus-cni`, `objectstorage`,
`s3manager`, `kamaji`) are intentionally untouched: each one fetches its
source from upstream at build time and pins its `golang:` base image to
whatever the upstream commit was tested against.
### Why bump past the advisory minimums
Bumping each affected dependency to its latest stable rather than the
lowest patched version means we pick up subsequent (non-CVE) bug fixes
for free and reduce the number of follow-up Dependabot bumps the next
reviewer has to triage. Where this required a cascading bump (e.g.
`runc` v1.4.2 only compiles together with `mount-utils` v0.32+, which
dropped the deprecated `libcontainer/userns` import path), the cascade
was followed through.
### Build verification
`go build ./...` and `go vet ./...` are clean across all five modules
locally. Two `go vet` warnings in the root module
(`pkg/lineage/lineage_test.go:76` unkeyed struct fields,
`cmd/cozypkg/cmd/add.go:521` non-constant format string) and three
`randfill.Continue` test panics in `pkg/apis/{apps,core}/install` and
`pkg/apiserver` also reproduce on `main` and are out of scope for this
PR.
### Out of scope
The `k8s.io/*` libraries in `kubeovn-webhook/go.mod` are still pinned at
`v0.26.0`; they are not flagged by Dependabot and bumping them would be
a separate, much larger maintenance change.
### Release note
```release-note
fix(security): bump Go dependencies in root, api/apps/v1alpha1, kubevirt-csi-driver, kubeovn-webhook, and token-proxy modules to clear all open Dependabot CVE alerts
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded Go toolchain across components and build stages to 1.26.x.
* Updated numerous dependencies for compatibility, stability and
security (Kubernetes APIs, gRPC, OpenTelemetry, golang.org/x/*, and
related libs).
* Several containers now run as a non-root user (UID/GID 65532) to
improve runtime security.
* **Refactor**
* CSI driver initialization refactored to a builder-style pattern
enabling conditional service wiring.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Describe the in-place password rotation performed by the Percona operator
when upgrading to releases that scope the system users secret per CR.
Clarify that consumers of the credentials secret need a pod restart to
pick up the new password, custom user secrets are unaffected, and the
legacy namespace-shared secret is left orphaned and must be cleaned up
manually.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
PR #2454 renamed the nodeGroups field ephemeralStorage to diskSize and
added a migration guard that fails template rendering when the old name
is used. Update kubelet reservation tests to use the new field name.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Merge origin/main into feat/kubernetes-kubelet-reserved-resources.
PR #2454 (persistent storage for worker nodes) landed in main and
touched the same nodeGroups schema. Conflicts resolved by keeping
both feature sets (kubelet reservations + persistent storage):
- api/apps/v1alpha1/kubernetes/types.go: merged kubebuilder default
marker with all keys from both features
- packages/apps/kubernetes/README.md: merged Parameters table with
diskSize/storageClass + kubelet* rows
- packages/system/kubernetes-rd/cozyrds/kubernetes.yaml: merged
openAPISchema and keysOrder, dropped stale ephemeralStorage path
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Replace the ephemeral `emptyDisk` volume on tenant Kubernetes worker VMs
with persistent PVC-backed storage via KubeVirt `dataVolumeTemplates`.
Previously, kubelet certificates, kubeconfig, and containerd state were
lost on VM reboot, causing the node to fall out of the tenant cluster.
**Changes:**
- Rename `ephemeralStorage` field to `diskSize` in NodeGroup
configuration
- Replace `emptyDisk` volume with `dataVolumeTemplates` using CDI blank
source
- Add `cdi.kubevirt.io/storage.usePopulator: "false"` annotation to work
around CDI volume population issues with LINSTOR CSI
- Add optional per-nodeGroup `storageClass` field (defaults to cluster
default)
- Set fixed 2Gi ephemeral-storage resource for virt-launcher pod
- Rename internal mount paths from `/ephemeral` to `/persistent`
- Add helm unit tests (`cluster_test.yaml`) covering dataVolumeTemplates
defaults, custom disk size, storageClass presence/absence, fixed
ephemeral-storage, disk name consistency, and CDI populator annotation
- Quote storage values in templates (`diskSize`, `storageClassName`) for
Helm correctness
- Guard bind-mount commands with `mountpoint -q` checks and create
target directories to handle fresh images and cloud-init retries
- Set `blockSize.matchVolume` on worker node disks for automatic block
size detection, fixing DRBD 4Kn compatibility with QEMU
**Known gotchas:**
- CDI volume population (prime PVC rebind) causes `ClaimMisbound` errors
with LINSTOR CSI driver, regardless of StorageClass (`replicated` or
`local`). The `usePopulator: false` annotation switches to the legacy
import path which works correctly. This is a known CDI issue documented
in
[kubevirt/containerized-data-importer#3146](https://github.com/kubevirt/containerized-data-importer/issues/3146).
- The `replicated` StorageClass with DRBD `suspend-io` quorum policy can
cause transient VM IO errors during LINSTOR satellite restarts (quorum
loss). The `local` StorageClass is recommended for worker node disks.
When `storageClass` is not specified in a nodeGroup, the cluster default
StorageClass is used.
- Existing clusters will trigger a rolling replacement of all worker
nodes on upgrade due to changed KubevirtMachineTemplate hash.
- DRBD volumes use 4096-byte logical sectors (`block-size 4096`).
Without explicit block size configuration, QEMU defaults to 512-byte
sectors, causing immediate IO errors and VM pause. The
`blockSize.matchVolume` setting in the disk spec ensures KubeVirt
auto-detects the correct sector size from the volume.
**Tested on a live cluster:**
- Verified DataVolume provisioning with `local` StorageClass
- Verified nodes join the tenant cluster and reach Ready state
- Verified node survives VM reboot (pod deletion) and returns to Ready
with the same Node object, reusing kubelet certificates from persistent
disk
### Screenshots
N/A — no UI changes.
### Release note
```release-note
feat(kubernetes)!: replace ephemeral emptyDisk with persistent storage for worker nodes
Worker node VMs now use persistent PVC-backed disks instead of ephemeral emptyDisk volumes. Kubelet certificates, kubeconfig, and containerd state survive VM reboots, preventing nodes from falling out of the tenant cluster.
BREAKING CHANGE: The `ephemeralStorage` field in nodeGroups has been renamed to `diskSize`. A new optional `storageClass` field allows per-nodeGroup storage class selection. Users must update their values overrides accordingly.
Worker node disks now use `blockSize.matchVolume` to automatically detect the underlying volume's sector size, ensuring compatibility with DRBD-backed storage classes that use 4096-byte logical sectors.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional storageClass parameter to select storage class for worker
node persistent disks (defaults to cluster default).
* Makefile test target added to run Helm unit tests for the chart.
* **Configuration Changes**
* Worker node storage parameter renamed from ephemeralStorage to
diskSize (default: 20Gi).
* Kubelet/containerd state moved to persistent disk paths and node
ephemeral-storage accounting fixed to 2Gi; disk provisioning aligns
block sizing with backing volumes.
* **Documentation**
* Updated docs and chart/schema to reflect the persistent disk model and
new fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
- Stamps the Helm-managed `{{ .Release.Name }}-credentials` Secret with
`apps.cozystack.io/user-secret: "true"`.
- Adds a `matchLabels` rule to `secrets.include` in the Harbor
`ApplicationDefinition`, alongside the existing `resourceNames` entry.
The lineage webhook decides whether a Secret is tenant-visible by
matching its name against the `secrets.include[].resourceNames` template
in the corresponding `ApplicationDefinition`. For Harbor the template
`{{ .name }}-credentials` renders to `{name}-credentials` and does not
account for the `release.prefix: "harbor-"` set in the same definition,
while the Helm chart actually creates `harbor-{name}-credentials`. The
names never match, so the Secret ends up labelled
`internal.cozystack.io/tenantresource=false` and never appears in the
TenantSecrets API.
This change follows the existing `apps.cozystack.io/user-secret: "true"`
pattern already used by Bucket
(`packages/system/bucket/templates/user-credentials.yaml`) and RabbitMQ
(`packages/apps/rabbitmq/templates/rabbitmq.yaml`). The webhook then
matches via labels and is independent of how Helm names the Secret.
### Screenshots
Not applicable — no UI changes in this PR.
### Release note
```release-note
fix(harbor): expose Harbor credentials Secret to tenant-aware clients by marking it with the `apps.cozystack.io/user-secret` label, mirroring the pattern already used by Bucket and RabbitMQ.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Harbor configuration to support label-based secret discovery
alongside existing name-based selection, improving secret management
reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Update description in values.yaml with corrected denylist and re-run
cozyvalues-gen to propagate changes to values.schema.json, README.md,
and types.go.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Allow integer values in postgresql.parameters via anyOf schema, so existing
HelmReleases with max_connections: 100 (integer) are not broken on upgrade.
Template coerces all values to string via toString before quoting.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The previous fix (a21c18f9) sourced BucketInfo via Flux valuesFrom with
`targetPath: bucket.bucketInfo`. Flux runs values with `targetPath`
through Helm's `strvals.ParseInto`, which splits the value on commas as
list separators. The COSI BucketInfo JSON is comma-rich, so values
resolution bailed:
could not resolve Secret chart values reference
'tenant-X/harbor-X-registry-bucket' with key 'BucketInfo':
key "\"spec\":{\"bucketName\":\"bucket-1785...\""
has no value (cannot end with ,)
Drop `targetPath`. With only `valuesKey: BucketInfo`, helm-controller
unmarshals the value as YAML and merges at the chart's values root, so
JSON commas stay nested instead of being split. The system chart's
bucket-secret.yaml now reads `.Values.spec.bucketName` /
`.Values.spec.secretS3.{accessKeyID,accessSecretKey,endpoint}`,
guarded by nested `with` blocks so the registry-s3 Secret renders only
when secretS3 has been populated.
Gating semantics from the previous fix are preserved: with the default
`optional: false`, helm-controller still refuses to compose values
until the COSI BucketAccess controller writes `BucketInfo` into the
Secret, and the value's content still drives the HR config-digest.
Verified via `helm template` against the system chart with three
scenarios: missing `spec` and `spec` without `secretS3` render
nothing; full BucketInfo renders all five S3 env keys correctly.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
PostgreSQL parameter names are case-insensitive, but the denylist
check was case-sensitive, allowing bypass via mixed-case names
(e.g., ARCHIVE_COMMAND instead of archive_command).
Normalize parameter names to lowercase before checking against the
denylist to prevent security bypass.
Thanks to CodeRabbit for catching this security issue.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Fix regression where defaults were overriding user-supplied parameters.
Sprig merge gives precedence to first argument, so user values must be
merged before defaults to maintain correct override behavior.
Before: merge (dict) $defaults .Values.postgresql.parameters
After: merge (dict) .Values.postgresql.parameters $defaults
Now max_wal_senders and other user parameters correctly override defaults.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
- Regenerate deepcopy artifacts to remove orphaned PostgreSQLParameters methods
- Update e2e test to use string value for max_connections parameter
- Fix template to avoid .Values mutation and use quote function for safe YAML escaping
Signed-off-by: ohotnikov.ivan <ohotnikov.ivan@e-queo.net>
Allow users to configure any PostgreSQL server parameter via the
postgresql.parameters field. All parameter values must be provided
as strings (numeric values should be quoted).
Security measures:
- Block parameters that allow shell command execution (archive_command,
restore_command, ssl_passphrase_command, log_destination, event_source)
- Validation implemented in templates to prevent security vulnerabilities
Generated files updated:
- types.go: Add Parameters field with map[string]string type
- README.md: Add parameter documentation and warnings
- values.schema.json: Update schema for new field
- CRD: Update OpenAPI schema with new field and reordered properties
CloudNativePG manages certain parameters automatically (archive_mode,
primary_conninfo, wal_level). Users should not override these as it
may break backup/replication functionality.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The Helm-managed harbor-{name}-credentials Secret was not visible
through the TenantSecrets API because the lineage webhook compared the
real Secret name against the rendered template "{{ .name }}-credentials"
(which becomes "{name}-credentials" without the "harbor-" HelmRelease
prefix), failed to match, and labelled the Secret as
internal.cozystack.io/tenantresource=false.
Adopt the same pattern Bucket and RabbitMQ already use: stamp the
user-facing Secret with apps.cozystack.io/user-secret=true and add a
matchLabels rule alongside the existing resourceNames entry in the
ApplicationDefinition. The webhook now matches via labels instead of
relying on Secret-name templating, so the Secret is properly marked
tenantresource=true and surfaces in tenant-aware clients.
Signed-off-by: IvanHunters <ivan.okhotnikov@aenix.io>
The `small` preset (512Mi) is not sufficient for the Strimzi 0.45 /
Kafka 3.9 image baseline. With the default Strimzi heap formula
(Kafka 50%, ZooKeeper 75% of container memory limit), the remaining
non-heap budget (Metaspace, code cache, direct buffers, JMX
javaagent, embedded Jetty admin server, KRaft migration endpoints,
thread stacks) consistently overflows the 512Mi cgroup limit at
startup, leading to OOMKilled (exit 137) and CrashLoopBackOff for
both Kafka and ZooKeeper pods.
Bump the default `resourcesPreset` for kafka and zookeeper from
`small` (512Mi) to `medium` (1Gi). The `small` preset stays in the
enum so existing deployments that pin it are unaffected; only the
default changes.
Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
Integrate the HAMi GPU virtualization addon from main while
keeping the persistent-storage diskSize and storageClass fields
introduced by this branch.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Match the pattern used by every other tenant-cluster addon HelmRelease
(gpu-operator, cilium, fluxcd, ...): only render when the upstream etcd
namespace is ready, not just when the addon is enabled. Add values-ci.yaml
to the HAMi test suite so the new gate has the fixture it needs.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Two bugs caused the harbor E2E to fail (CI #25049445125):
1. cozy-harbor's bucket-secret.yaml called `index $existingSecret.data
"BucketInfo"` unconditionally. During first-time install the COSI
BucketAccess controller may create the credentials Secret as a
placeholder before populating it, so `.data` is nil and the chart
render crashes with `index of untyped nil`.
2. The downstream `<release>-system` HelmRelease starts reconciling
immediately, in parallel with bucket provisioning, hitting bug #1
on every retry within the test's 5-minute window.
The previous shape relied on `lookup` to read the Secret at render
time. helm-controller's upgrade trigger is digest-based over composed
values + chart artifact, so a `lookup` returning new data on a later
reconcile is not enough on its own to force an upgrade — the rendered
output may diverge but the digest does not.
Switch to a values-driven shape:
- `bucket-secret.yaml` now reads `.Values.bucket.bucketInfo` (a JSON
string) and uses `dig` for safe access; if the value is empty or the
expected nested fields are missing, no `*-registry-s3` Secret is
rendered.
- The `<release>-system` HR sources `BucketInfo` via `valuesFrom` with
`targetPath: bucket.bucketInfo`. With the default `optional: false`,
helm-controller will refuse to compose values until the key exists,
which both gates initial reconciliation on the BucketAccess Secret
being populated and forces a config-digest change (and thus a helm
upgrade) when its contents change.
- `bucket.secretName` is removed from the system chart's values and
from the apps chart's `values:` block; the only consumer (the
`lookup` call) is gone.
Verified four scenarios via `helm template` against the system chart:
unset bucketInfo, empty string, empty JSON object `{}`, and a
fully-populated BucketInfo — only the last renders the registry-s3
Secret; the others render nothing without erroring.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Tests need _namespace.etcd from values-ci.yaml after the merge
introduced an etcd-namespace guard on the gpu-operator HelmRelease
condition.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Update Data lifecycle section to address review feedback:
- Reflect the release-scoped users secret naming.
- Distinguish operator-driven PVC release from actual PV/data
deletion (depends on StorageClass reclaimPolicy).
- Document that TLS secrets (<release>-ssl, <release>-ssl-internal)
are not reclaimed automatically.
- Add a recovery note for the case where the operator is removed
before the CR, leaving finalizers unable to run.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Test descriptions said 'percona.com/delete-psmdb-pvc finalizer'
while the assertions verified both delete-psmdb-pvc and
delete-psmdb-pods-in-order. Use 'Percona finalizers' to match
what the tests actually check.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The operator deletes spec.secrets.users (defaulted to the global
percona-server-mongodb-users) as part of the delete-psmdb-pvc
finalizer flow. With the default value, deleting one MongoDB
release in a namespace would remove the secret shared with any
other MongoDB release there.
Override spec.secrets.users with a release-scoped name so each
release owns its own secret and uninstalls do not collide.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The previous wording listed only one operator-managed secret with
'e.g.' prefix. The Percona Operator finalizer flow actually deletes
four specific secrets (internal-<release>, internal-<release>-users,
<release>-mongodb-encryption-key, and the global default
percona-server-mongodb-users). Document each by name so operators
understand the full deletion blast radius, especially the global
default which can collide with externally-created secrets sharing
the same name.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Distinguish between operator-managed secrets reclaimed via the
finalizer flow (e.g. internal-*-users) and Helm-managed secrets
(credentials, user secrets, s3 backup creds) removed by helm
uninstall. The previous wording was ambiguous and could mislead
users about which cleanup path applies to which secret.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
PerconaServerMongoDB CR now carries the upstream-default finalizer
pair: percona.com/delete-psmdb-pods-in-order ensures primary-last
graceful shutdown before percona.com/delete-psmdb-pvc reclaims
storage. This matches deploy/cr.yaml defaults in operator v1.21.1
and prevents potential data loss on deletion under load.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Upstream Percona Operator v1.21.1 ships both delete-psmdb-pvc and
delete-psmdb-pods-in-order in deploy/cr.yaml by default. The
pods-in-order finalizer ensures primary-last graceful shutdown,
preventing data loss on cluster deletion under load.
Currently fails — establishes red phase before adding the finalizer.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Replace the equal assertion (which would block adding more finalizers)
with a sharded-mode test case ensuring the finalizer is rendered
regardless of replicaset/sharded configuration.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Document that uninstalling the release reclaims PVCs and
operator-managed user secrets via the percona.com/delete-psmdb-pvc
finalizer. Pre-existing users who relied on orphaned PVCs to
recover data should take backups before uninstalling.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Rename test description to accurately reflect what's verified
(template renders the finalizer, not runtime cleanup behavior).
Add equal assertion locking the exact finalizer list to catch
regressions where a stray finalizer is appended.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add helm-unittest assertion verifying that the rendered
PerconaServerMongoDB CR carries percona.com/delete-psmdb-pvc finalizer.
Currently fails — establishes red phase before adding the fix.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## Summary
Add `serverName` field to bootstrap configuration to support backup
recovery when the Barman server name in `backup.info` differs from the
Kubernetes cluster name.
This fixes "no target backup found" errors during CloudNativePG recovery
operations.
## Problem
CloudNativePG forms the backup path as `destinationPath + "postgres-" +
oldName`, but searches for backups using the `server_name` field from
`backup.info`. When these values don't match (e.g., cluster name is
`grafana` but `server_name` is `cloud`), recovery fails with "no target
backup found".
## Solution
- Add optional `serverName` parameter to `bootstrap` configuration
- When specified, CloudNativePG uses this value to search for backups in
S3
- Falls back to `oldName` when `serverName` is not provided (backwards
compatible)
## Changes
- Add `ServerName` field to PostgreSQL CRD type definition
- Add conditional `serverName` to Cluster `externalClusters` template
- Update `values.yaml` and README.md with `serverName` documentation
- Regenerate `values.schema.json` and `postgres.yaml` CRD
## Test plan
- [ ] Deploy postgres cluster with backups enabled
- [ ] Create backup
- [ ] Create new postgres instance with `bootstrap.enabled=true`,
`bootstrap.oldName=<original-name>`, and
`bootstrap.serverName=<value-from-backup.info>`
- [ ] Verify recovery completes successfully
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an optional bootstrap.serverName setting to specify the Barman
server name from the original cluster’s backup.info (used when that
server name differs from the Kubernetes cluster name).
* **Documentation**
* Clarified that bootstrap.oldName must match the serverName value
recorded in backup.info; updated docs and examples to reflect the new
bootstrap.serverName option.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds an opt-in `publishing.exposure=loadBalancer` mode for the
ingress-nginx Service as a migration path away from
`Service.spec.externalIPs`, which is deprecated upstream in Kubernetes
v1.36
([KEP-5707](https://github.com/kubernetes/enhancements/issues/5707),
[kubernetes#137293](https://github.com/kubernetes/kubernetes/pull/137293)).
The `AllowServiceExternalIPs` feature gate is expected to default to off
around v1.40 and the implementation to be removed around v1.43.
Stacked on top of #2464 (cilium v1.19.3 bump) — depends on it for the
`CiliumLoadBalancerIPPool` at `cilium.io/v2`.
### Behavior
- New platform value `publishing.exposure` — enum `externalIPs |
loadBalancer`, default `externalIPs` (current behavior unchanged on
upgrade).
- Plumbed through `cozystack-values` into each tenant's ingress
HelmRelease via the new `expose-mode` key.
- When `exposure=loadBalancer` and the current namespace matches
`publishing.ingressName`, the Service becomes `type: LoadBalancer` with
`externalTrafficPolicy: Local` and a `CiliumLoadBalancerIPPool`
announces the addresses from `publishing.externalIPs` via Cilium LB
IPAM.
- The pool uses a namespace-only `serviceSelector`
(`io.kubernetes.service.namespace: <ns>`) — any LoadBalancer Service in
the tenant namespace draws from it. See "Pool ownership" below.
- IPv4 addresses get `/32` CIDRs, IPv6 addresses get `/128`. Mixed
families supported. Pre-CIDR entries (`192.0.2.10/32`) are accepted
without double-suffixing.
- Unknown values, `loadBalancer` with empty externalIPs, and stray empty
entries from `publishing.externalIPs` are rejected at render time with
explicit error messages.
### Pool ownership
The `CiliumLoadBalancerIPPool` is rendered from
`packages/apps/tenant/templates/cilium-lb-pool.yaml`, not from the
ingress chart. The tenant chart is the per-tenant owner of cross-cutting
resources (Namespace, `cozystack-values` Secret, HelmReleases for
ingress and gateway), so one pool per tenant lives there.
Cilium LB IPAM rejects overlapping CIDRs across pools regardless of
`serviceSelector` — the last-added pool gets `cilium.io/PoolConflict`
and stops allocating. Keeping the pool in the ingress chart would
collide with the Gateway-API PR (#2470), which materialises its own
LoadBalancer Service from the same `publishing.externalIPs` range when
`tenant.spec.gateway=true`. Moving the pool to the tenant chart with a
namespace-only selector lets a single pool back both services
(ingress-nginx today, a Cilium Gateway Service once #2470 lands).
Only the ingress-loadBalancer signal is wired here
(`_cluster.expose-mode=loadBalancer` + `.Values.ingress=true` +
publishing tenant). #2470 rebases on top of this PR, drops its own
`packages/extra/gateway/templates/cilium-lb-pool.yaml`, and adds an OR
branch for `.Values.gateway` in the tenant template.
### Scope
Only the ingress-nginx Service is migrated by this setting. Other
cozystack components that still write `Service.spec.externalIPs`
directly (notably the `vpn` app at
`packages/apps/vpn/templates/service.yaml`) need separate follow-up
before Kubernetes v1.40.
### Tests
- `packages/extra/ingress/tests/exposure_test.yaml` — 10 helm-unittest
cases on the ingress-nginx Service: type / externalTrafficPolicy /
externalIPs assertion for both modes, unknown-mode rejection
(case-sensitive enum), empty-IPs failure, empty-entry filtering,
non-publishing-tenant fallback.
- `packages/apps/tenant/tests/exposure_test.yaml` — 9 cases on the pool
itself: IPv4, IPv6, mixed, pre-CIDR input, empty-entry filtering,
`ingress=false` in publishing tenant (no pool), non-publishing tenant
(no pool), empty externalIPs (no pool).
- Both suites are auto-discovered by `hack/helm-unit-tests.sh` via the
`test:` target in each package's Makefile.
### Caveats (copied from the inline `values.yaml` comment)
- `loadBalancer` mode uses `externalTrafficPolicy: Local`. The external
IP must already be routed to a node that hosts an ingress pod (floating
IP / upstream router / podAntiAffinity).
- Cilium does not announce the IP on its own unless L2 announcements or
BGP are enabled in the Cilium values (disabled by default in cozystack).
- Switching the value on a running cluster causes the ingress-nginx
Service to be recreated (`upgrade.force: true` on the HelmRelease +
Service kind change); expect a brief interruption of ingress traffic.
### Release note
```release-note
feat(ingress): add opt-in publishing.exposure=loadBalancer mode that uses type: LoadBalancer + CiliumLoadBalancerIPPool instead of the deprecated Service.spec.externalIPs. Default (externalIPs) preserves existing behavior.
```
A single LoadBalancer selecting all replset members round-robined writes
across primary and secondaries, so ~⅔ of writes failed with "not primary".
Drivers couldn't auto-recover either: `isMaster` returned in-cluster DNS
names that don't resolve from outside.
Switch to the Percona operator's native `replsets[].expose` so it creates
one LoadBalancer per pod and rewrites `rs.conf` with the external IPs;
the driver then discovers the primary correctly via the standard replica
set connection string. The manual `external-svc.yaml` is now scoped to
sharded mode (mongos front-end) only.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
- Add `externalAllowICMP` value (default true) propagated as
`networking.cozystack.io/allowICMP` annotation on the rendered Service
when `externalMethod: PortList`. The cozy-proxy companion (released as
part of cozystack/cozy-proxy#11 + #12) drops ICMP by default in
port-filter mode, which breaks ping and PMTU discovery; defaulting the
chart to "true" preserves user expectations while still allowing
operators to opt out by setting `externalAllowICMP: false`.
- Remove the v1.3.1.md changelog entry. Project convention is to add
changelogs in a dedicated "docs: add changelog for vX.Y.Z" commit at
release time, not as part of feature/fix PRs.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Render `networking.cozystack.io/wholeIP: "false"` on the Service when
`externalMethod: PortList` is configured (was always `"true"` before).
Combined with cozy-proxy v0.3.0+ which adds per-port filtering for
"false"-annotated services, this makes `externalMethod: PortList`
behave as documented: only ports listed in `externalPorts` are
reachable from the LoadBalancer IP.
Backward-compatible: existing services with `externalMethod: WholeIP`
continue to set `wholeIP: "true"` and behave identically. cozy-proxy
versions older than v0.3.0 ignore Services with `wholeIP: "false"`,
which means PortList Services on older cozy-proxy will lose their
egress IP preservation — but that path was already non-functional
ingress-wise, so this is not a regression for users actually relying
on PortList.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Incorporate images.waitForKubeconfig support from main while
keeping the persistent-storage diskSize and storageClass fields
introduced by this branch.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Merge origin/main into feat/kubernetes-kubelet-reserved-resources.
Conflicts resolved:
- packages/apps/kubernetes/Makefile: kept .PHONY declaration from PR
- packages/system/kubernetes-rd/cozyrds/kubernetes.yaml: kept both
kubelet schema from PR and images property from main
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Closes#2412. On a cold tenant-Kubernetes bootstrap, the parent
HelmRelease raced the admin-kubeconfig Secret that Kamaji provisions
asynchronously. Three CP-side Deployments (cluster-autoscaler, kccm,
kcsi-controller) mounted that Secret as a hard volume, flux
helm-controller's default wait budget was too short for Kamaji cold
start, and `install.remediation { retries: -1 }` then uninstalled the
Cluster CR and restarted the cycle forever.
Implements a defense-in-depth fix:
- `optional: true` on the admin-kubeconfig Secret volume in all three
Deployments so kubelet no longer FailedMounts while Kamaji is still
bootstrapping.
- A shared `wait-for-kubeconfig` init container (in
`templates/_helpers.tpl`) that polls for `super-admin.svc` with a 10m
deadline, strictly below the HelmRelease Install.Timeout so a broken
tenant falls into CrashLoopBackOff visibly instead of hanging forever.
- Per-Application HelmRelease Install/Upgrade timeout, driven by a new
`release.cozystack.io/helm-install-timeout` annotation on
ApplicationDefinition. Kubernetes-rd sets it to `15m`; other kinds leave
it unset and keep flux defaults, so their failed installs remediate on
the normal cadence. Parser rejects ns/us/µs (accepted by
`time.ParseDuration`, rejected by Flux's CRD pattern) at startup.
- Soft-skip when `_namespace.etcd` is empty: the CP-side Deployments,
the Cluster/KamajiControlPlane/KubevirtCluster/WorkloadMonitor CRs, and
every child HelmRelease that references admin-kubeconfig now render only
when an etcd DataStore exists for this tenant. An `awaiting-etcd`
ConfigMap is emitted as a user-visible status beacon so `helm install`
still succeeds and flux retries on its 5m interval until the Tenant
chart catches up.
- e2e remediation guard built on `.status.history[].status` (the
Snapshot shape), not on `.status.installFailures` - `ClearFailures()`
zeroes the latter on every successful reconciliation, which made the
previous guard vacuous.
Tests:
- Go unit tests for the annotation parser (accepted/rejected units) and
the HR builder (table-driven across kinds).
- helm unittest for the per-template structure (optional volume, init
container, dataStoreName, awaiting-etcd beacon).
- bats unit tests for the shell guard (every combination of
empty/zero/positive history entries, plus pinned HR v2 shape).
- Chart-wide bats invariants: every Deployment mounting admin-kubeconfig
has the guards; zero such Deployments and zero HelmReleases render when
etcd is empty.
All wired into the existing `make unit-tests` target (`go-unit-tests`
added alongside `helm-unit-tests` and `bats-unit-tests`).
Option 2 from the ticket (separate HelmRelease with `dependsOn`) was
intentionally not taken: the combination above closes the same race
without restructuring the chart's HelmRelease topology.
### Release note
```release-note
fix(kubernetes): close admin-kubeconfig race on tenant Kubernetes bootstrap. The parent HelmRelease no longer enters an uninstall/retry cycle when Kamaji control-plane cold start exceeds flux's default wait budget. A Kubernetes tenant created before the parent Tenant application has etcd enabled now renders only an awaiting-etcd beacon ConfigMap and waits quietly for the DataStore to appear, instead of producing half-installed Deployments that CrashLoopBackOff forever.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Per-application Helm install/upgrade timeout via metadata annotation.
* Init-container guards that wait for admin kubeconfig before workloads
start.
* Chart resources now render conditionally based on etcd presence.
* **Tests**
* Helm-template tests for admin-kubeconfig invariants and
remediation-cycle detection.
* New Go unit tests and CI Helm/unittest coverage plus test value files.
* **Chores**
* Added BusyBox image pin and new Makefile test targets (including Go
unit-tests).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozystack.io/status-beacon: "true" annotation had no consumer in
the chart, no documented contract, and no convention defined for other
charts to follow. It would have become accidental precedent for
contributors copying the pattern without understanding it.
The ConfigMap itself is self-explanatory: the name <release>-awaiting-etcd,
data.status: "awaiting-etcd", and the human-readable message in
data.message all surface the same operator signal via kubectl get cm.
Drop the annotation; keep the ConfigMap.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Operators in air-gapped or rate-limited environments cannot reach
docker.io and the bundled busybox digest pin gives them no escape
hatch. Add an optional images.waitForKubeconfig chart value that, when
set, replaces the helper's image reference with any registry path
kubelet can pull. Empty value falls back to images/busybox.tag, so the
prior digest-pinned default is preserved.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Arsolitt