Commit Graph
5128 Commits
Author SHA1 Message Date
Myasnikov Daniil bc19eae311 docs: add changelog for v1.3.3
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-07 16:21:11 +05:00
myasnikovdaniil c22d6661ad fix(kafka): create LoadBalancer service when external flag is enabled (#2578)
## What this PR does

When `external: true` is set in the Kafka chart, Strimzi creates a
LoadBalancer service named `<release>-kafka-external-bootstrap` for
external client access. However, two issues prevented this from working
correctly:

1. **Unconditional listener definition**: the `external` listener was
   always present in the Kafka CR (as `type: internal` when disabled),
   instead of being omitted entirely when `external: false`.

2. **Missing dashboard RBAC**: the dashboard role only granted access
   to `<release>-kafka-bootstrap` (internal ClusterIP). The
   Strimzi-managed external bootstrap service was not included, making
   it invisible to tenant users even when it was created.

**Changes:**
- Make the external listener conditional — added only when `external:
true`
- Add `<release>-kafka-external-bootstrap` to dashboard RBAC rules
  when `external: true`
- Add helm-unittest tests covering listener and RBAC behavior for both
  `external: true` and `external: false`

### Screenshots

N/A — no UI changes.

### Release note

```release-note
fix(kafka): external listener is now correctly created as a LoadBalancer and exposed in the tenant dashboard when `external: true` is set
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* External Kafka listener and external bootstrap service in dashboard
resource mapping are now conditionally enabled based on configuration
settings.

* **Tests**
* Added comprehensive test suites validating Kafka configuration
rendering and dashboard resource mapping across internal and external
deployment variations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-07 15:15:57 +05:00
Andrey Kolkov ed5ae16e36 feat(platform): implement backup strategy for postgres application
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-05-07 13:12:54 +04:00
Aleksei Sviridkin a9c1298b42 [gpu-operator] Update to v26.3.1 and add experimental vGPU variant (#2323)
## What this PR does

Updates the NVIDIA GPU Operator from v25.3.0 to v26.3.1 and adds an
**experimental** `vgpu` variant for SR-IOV vGPU on Ada Lovelace and
newer GPUs.

Changes:
- Update vendored upstream chart to v26.3.1 (CDI enabled by default,
component version bumps).
- Pin chart version in Makefile for reproducibility.
- Add `values-vgpu.yaml` for the vGPU variant with `defaultWorkload:
vm-vgpu` so the chart activates the vGPU Manager DaemonSet without
per-node labelling.
- Add `vgpu` variant to PackageSource alongside the existing `default`
(passthrough).
- Rename `values-talos.yaml` → `values-passthrough.yaml` and pin
`defaultWorkload: vm-passthrough` for symmetry with the vgpu variant.
The overrides apply to any host OS using vfio-pci, not Talos
specifically.
- Refresh `hack/dcgm-default-counters.csv` to match the new
dcgm-exporter image tag (`4.5.1-4.8.0`) — adds `DCGM_FI_DEV_FB_RESERVED`
and other fields introduced upstream.
`hack/check-gpu-recording-rules.bats` passes against the new fixture.
- Add `hack/check-gpu-operator-variants.bats` that renders both variants
via `helm template` and asserts the pinned ClusterPolicy values, so the
next chart bump cannot silently un-pin them.
- Add internal documentation for the vGPU bring-up.

The `vgpu` variant assumes the user supplies a proprietary vGPU Manager
image via Package CR component values — the .run cannot be
redistributed.

### `vgpu` variant is experimental

NVIDIA's `vgpu-device-manager` walks `/sys/class/mdev_bus/`, which does
not exist on Ada Lovelace and Blackwell — the DaemonSet errors with "no
parent devices found" and is disabled by default. Until an SR-IOV-aware
controller is shipped, profile assignment is an out-of-band step (`echo
<id> > /sys/.../current_vgpu_type` per VF) that must be re-applied after
every node reboot. Without it, `permittedHostDevices.pciHostDevices`
reports zero allocatable resources. The variant ships so the chart bump
and the underlying skeleton (DaemonSet + image plumbing + docs) land
together; production deployments need an automated profile-assignment
mechanism in addition to this PR.

### KubeVirt requirement

KubeVirt advertises VFs via `permittedHostDevices.pciHostDevices` rather
than `mediatedDevices`. This requires kubevirt/kubevirt#16890 ("vGPU:
SRIOV support", merged 2026-04-10) which is `main`-only — released tags
`v1.6.x` / `v1.7.x` / `v1.8.x` do not include it. The patch is targeted
at the next minor KubeVirt release (v1.9.0); track the PR for the actual
release tag. Backports to released branches are not planned.

### Behavior changes inherited from upstream chart v25 → v26

The upstream chart flipped two defaults that are pinned back to `false`
here, so `default` (passthrough) deployments do not silently change
footprint on upgrade:

- `ccManager.enabled` was flipped to `true` upstream — would auto-enable
Confidential Computing on Hopper hardware (H100/H200/B100) after
upgrade.
- `vgpuDeviceManager.enabled` was flipped to `true` upstream — DaemonSet
would render even in passthrough mode.

Both stay opt-in via Package CR overrides. The new bats test guards the
pin.

Other inherited changes that are user-visible and worth pinning down
before upgrade:

- `driver.licensingConfig.configMapName` is deprecated upstream in
favour of `driver.licensingConfig.secretName`. The deprecation marker
lives in the CRD's `description` field (visible via `kubectl explain`) —
there is no helm-render-time warning, the chart template silently
prefers `secretName` and falls back to `configMapName`. Operators with
existing licensing overrides should migrate the key on the next chart
upgrade.
- `driver.version` upstream default jumped from `570.x` to `580.x`.
Invisible on the `default` variant (which sets `driver.enabled: false`),
user-visible for anyone running the upstream Ubuntu driver path.
- `vfioManager.image` switched from
`nvcr.io/nvidia/cuda:12.8.1-base-ubi9` to
`nvcr.io/nvidia/cloud-native/k8s-driver-manager:v0.10.0`. Affects every
`default` (passthrough) deployment on upgrade — different image,
different layer, different image-pull surface. Worth pre-mirroring for
air-gapped clusters.
- `dcgmExporter.image` switched to the `-distroless` flavour. Smaller
surface, fewer base-image CVEs, but no shell — operators who used to
`kubectl exec` into the exporter for ad-hoc debugging will need to
switch to `kubectl debug` or container ephemeral images.
- `CDI` (Container Device Interface) is enabled by default in v26.3.1;
this is upstream's recommended setting and is left as-is.
- `node-feature-discovery` subchart source switched from the GitHub
Pages Helm repo to `oci://registry.k8s.io/nfd/charts`. Air-gapped
installations need to mirror the new OCI URL.

### Talos

Talos is **not recommended** for the vGPU variant. NVIDIA does not
publicly distribute the vGPU guest driver — it requires NVIDIA
Enterprise Portal access — and Sidero closed siderolabs/extensions#461
noting that they cannot support vGPU "unless NVIDIA changes their
licensing terms or provides us a way to obtain, test, and distribute the
software". Building a Talos system extension that bakes in the vGPU
driver is therefore not feasible without a private fork that violates
the EULA. Passthrough (`default` variant) on Talos is unaffected.

### Release note

```release-note
[gpu-operator] Update NVIDIA GPU Operator to v26.3.1 and add experimental `vgpu` variant for SR-IOV vGPU on Ada Lovelace / Blackwell (L4, L40, L40S, B100; Ubuntu host; requires KubeVirt v1.9.0+ for `pciHostDevices` advertising). The variant ships skeleton support — profile assignment (`current_vgpu_type` per VF) is currently out of band and must be automated by the operator before production use. On the existing `default` variant: `ccManager` and `vgpuDeviceManager` upstream defaults are pinned back to `false` to preserve prior behavior on chart upgrade; `defaultWorkload` is now pinned to `vm-passthrough` (was unset, falling back to `container`); `driver.licensingConfig.configMapName` is deprecated by upstream in favour of `secretName`; `driver.version` upstream default moves 570 → 580; `vfioManager.image` switches from CUDA UBI9 to k8s-driver-manager (relevant for air-gapped mirrors); `dcgmExporter` image switched to the `-distroless` flavour. NFD subchart source switched to `oci://registry.k8s.io/nfd/charts`.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Introduced vGPU support variant with end-to-end configuration guide,
validation procedures, and troubleshooting documentation

* **Updates**
* Upgraded GPU Operator to v26.3.1 and Node Feature Discovery to v0.18.3
* Improved component health checks and added PodDisruptionBudget
availability controls

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-07 11:18:36 +03:00
myasnikovdaniil 6cc9013085 docs: add changelog for v1.3.2 (#2550)
This PR adds the changelog for release `v1.3.2`.

✅ Changelog has been automatically generated in
`docs/changelogs/v1.3.2.md`.
untagged-34c5e929a0cb9154bef0
2026-05-07 11:32:45 +05:00
myasnikovdaniil 61de813613 feat(e2e): parallel post-install prep (#2557)
## What this PR does

### Parallel post-install prep

LINSTOR / StorageClass / MetalLB setup, previously sequential `@test`
cases after the 15 m platform HR wait, now run as a background prep that
completes during the platform reconcile. None of them gates platform HR
reconcile, so they can move out of the critical path.

New helper `hack/e2e-post-install-prep.sh` polls for its prerequisites
and is awaited at the end of the `Install Cozystack` test so failures
still surface deterministically.

### T2-race fix

Wait on the LINSTOR HR being `Ready` before polling for
`deploy/linstor-controller`. The old 5-min hard timeout fired during
cold installs where the LINSTOR HR's dependency chain (cert-manager →
piraeus-operator-crds → piraeus-operator → linstor) takes longer than 5
min to resolve.

### E2E uses helmReleaseInterval=30s

E2E install passes `--set cozystackOperator.helmReleaseInterval=30s` to
eliminate the 8–10 min Flux dead zone where dependency-blocked HRs are
requeued only every 5 min. Operator-side flag and chart value landed in
#2509; production defaults are unchanged.

Surfaced from #2500.

### Release note

```
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
- Improved end-to-end test initialization to run concurrent post-install
preparation, wait for cluster release reconciliation, verify component
readiness, and validate automated storage configuration.

* **Chores**
- Added background post-install setup and logging to increase test
reliability and speed up cluster initialization and teardown
diagnostics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-07 10:02:52 +05:00
Aleksei Sviridkin 5096e3bf9c fix(ci): stop backport workflow from cascading on its own PRs (#2584)
## What this PR does

Fix the cascade where merging a backport PR (e.g. #2579) spawns another
backport (e.g. #2581 `[Backport release-1.2] [Backport release-1.3] …`).

Two compounding issues:

1. `pr-labeler.yaml` auto-added the `backport` label to any PR whose
title started with `[Backport release-X.Y]` — exactly the prefix
`korthout/backport-action` puts on every backport PR. Introduced in
`c8ed1c65` (CNCF/k8s label conventions). Fixed by dropping that
`toAdd.add('backport')` line.
2. `dosubot[bot]` independently re-applies `backport-previous` to
backport PRs based on similarity to the parent PR; it cannot be
controlled from this repo. Fixed by gating both jobs in `backport.yaml`
on `pull_request.base.ref == 'main'`, so backport PRs (which target
`release-X.Y`) cannot trigger the workflow regardless of labels.

Verified pre-fix behaviour by inspecting the label-event timeline of
#2580 (dosubot adds `backport-previous` 47s after PR open) and the diff
of `c8ed1c65` adding `pr-labeler.yaml`.

### Release note

```release-note
fix(ci): backport automation no longer creates duplicate backport PRs when a backport PR is merged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Enhanced backport automation workflow with additional safety checks to
ensure operations only process pull requests targeting the main branch.
* Simplified pull request labeling workflow to streamline label
assignment logic.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-06 18:03:28 +03:00
myasnikovdaniil 3e9e393b7f fix(virtual-machine): vminstance disk race + bump VM IP/ready timeouts (#2555)
## What this PR does

Three fixes for `hack/e2e-apps/vminstance.bats`, surfaced once the 3x
retry was dropped from `Run E2E tests`:

1. **Disk delete-recreate race.** The teardown used `kubectl delete
vmdisk … || true` and immediately created a new disk; sometimes the new
disk reconciled before the old finalizer completed, leaving a duplicate
PVC and the new VM stuck. Drop the `|| true`, block until the old disk
is gone, then create.

2. **VM IP wait timeout 20s → 120s.** Under nested-virt + concurrent
runner load, a fresh VM sometimes takes 60–90 s to acquire its IP via
DHCP. The 20 s bound was tripping deterministically.

3. **VM ready timeout 20s → 60s.** Same root cause; the readiness probe
needs the IP to be reachable, so the lower bound has to grow alongside
the IP wait.

Existence backstop on the `vm-instance-$name` HR is added here too (it
touches the same hunk as the timeout fix; cleaner to land them
together).

Surfaced from #2500.

### Release note

```
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
  * Improved end-to-end test reliability for VM instance and disk flows.
* Replaced short sleeps with explicit waits for upstream and downstream
deployment artifacts.
* Extended polling windows and timeouts for IP assignment and readiness
checks.
* Strengthened cleanup to reliably remove prior VM and disk resources
before and after tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-06 19:47:11 +05:00
cozystack-ci[bot] 9d7941b9ec docs: add changelog for v1.3.2
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-05-06 14:38:32 +00:00
myasnikovdaniil 32e75ab183 Merge branch 'main' into daniil/split-vminstance
Signed-off-by: myasnikovdaniil <60174387+myasnikovdaniil@users.noreply.github.com>
2026-05-06 18:15:08 +05:00
myasnikovdaniil 7585b0c42c test(e2e): replace fixed timeouts with event-driven backstops (#2554)
## What this PR does

Replaces fragile `sleep N` / `timeout N until …; do sleep …; done`
patterns in `hack/*.bats` and `hack/*.sh` with `kubectl wait
--for=condition=…` plus a small existence backstop:

```sh
until kubectl get <resource> >/dev/null 2>&1; do sleep 2; done
kubectl wait <resource> --for=condition=Ready --timeout=…
```

### Why the existence backstop matters

`kubectl wait` against a not-yet-created resource sits silently for the
full timeout and then fails with `error: timed out waiting for the
condition` — useless for diagnosis. With the backstop, the test fails
fast with a clear "resource never appeared" error.

Existence backstop added at 14 sites in this sweep across:
- `redis`, `kafka`, `openbao`, `harbor`, `mongodb`, `qdrant`,
`clickhouse`, `mariadb`, `postgres`, `bucket`, `external-dns`,
`foundationdb` bats
- `hack/e2e-install-cozystack.bats`
- `hack/e2e-apps/run-kubernetes.sh`
- `hack/e2e-test-openapi.bats` (port-readiness wait replacing fixed
`sleep 5` after `kubectl proxy`)

The vm-disk hunk in `hack/e2e-apps/vminstance.bats` is included here;
the vm-instance hunks land separately via the `daniil/split-vminstance`
PR.

### Genuine sleeps annotated

The 3 remaining genuine sleeps (LINSTOR node membership propagation,
MetalLB end-to-end HTTP path, HR-count emission heuristic) are annotated
with `# TODO` + rationale so a future reader knows they are intentional.

Same green-path semantics; failures now surface in seconds instead of
after a 5-minute timeout. Surfaced from #2500.

### Release note

```
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Improved e2e stability by replacing fixed sleeps with timeout-based
polling across many integration tests, waiting for operator-created
resources to exist before readiness checks to reduce race conditions and
flakes.

* **Documentation**
* Added clarifying notes for LoadBalancer retry behavior and improved
local proxy readiness checks in e2e scripts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-06 18:13:30 +05:00
Myasnikov Daniil af317f16a4 fix(ci): stop backport workflow from cascading on its own PRs
Two compounding issues caused merging a backport PR to spawn another
backport (e.g. #2581 [Backport release-1.2] [Backport release-1.3] ...
from #2579):

1. pr-labeler auto-added the `backport` label to any PR whose title
   started with `[Backport release-X.Y]` (regression introduced in
   c8ed1c65 when pr-labeler.yaml was first added). korthout/backport-
   action puts that prefix on every backport PR it creates, so the
   label was being re-applied to every backport, satisfying the
   workflow's gate when the backport itself was merged.

2. Even with that fixed, an external labeling bot (dosubot) still
   re-applies `backport-previous` to backport PRs based on similarity
   to the parent. That bot is configured at the org level and cannot
   be controlled from this repo.

Drop the `toAdd.add('backport')` line from pr-labeler so the label
isn't propagated by us, and gate both backport jobs on
`pull_request.base.ref == 'main'` so backport PRs (which target
release-X.Y) cannot trigger the workflow regardless of what labels
end up on them.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-06 18:09:09 +05:00
myasnikovdaniil 8e78536cc3 fix(cert-manager): bump chart to v1.20.2 (#2562)
## What this PR does

Bumps the vendored cert-manager chart from v1.19.3 to v1.20.2. Pure
upstream bump
via `make update` — no cozystack-side template changes.

### Why

[Commit
`2b6e20cc`](https://github.com/cozystack/cozystack/commit/2b6e20cc)
migrated
the platform's ACME HTTP-01 setup to the modern `ingressClassName` API
on both
the ClusterIssuer's solver and the per-tenant override annotation
`acme.cert-manager.io/http01-ingress-ingressclassname`. The override
annotation,
however, was only added to upstream cert-manager in v1.20.0
([cert-manager#8244]) — v1.19.x silently ignores it. Result:
cert-manager always
falls back to the ClusterIssuer's solver class (`tenant-root`), so
HTTP-01 solver
Ingresses for non-`tenant-root` tenants land on the wrong controller and
Let's
Encrypt validation fails for tenant-scoped Ingresses (grafana, harbor,
alerta,
etc. inside child tenants).

Targeting v1.20.2 (not .0):
- [cert-manager#8655] (in v1.20.1) — restores
`clusterissuers/issuers/finalizers`
  RBAC dropped in v1.20.0.
- [cert-manager#8665] (in v1.20.2) — webhook helm-template fix.

### Verified live on dev10

| Stage | `Challenge.spec.solver.http01.ingress.ingressClassName` |
|---|---|
| Pre-bump (v1.19.3) | `"tenant-root"` (bug) |
| Post-bump (v1.20.2) | `"<tenant>"` (fixed) |

Full propagation chain confirmed end-to-end: source Ingress annotation
-> ingress-shim
sets `http01-override-ingress-ingressclassname` on the Certificate ->
acmeorders sets
`spec.solver.http01.ingress.ingressClassName` on the Challenge -> http01
solver creates
the solver Ingress on the tenant's class.

### Upgrade considerations

- **Container UID/GID change**: 1000/0 -> 65532/65532 across controller,
webhook,
cainjector. PSP / SecurityContext rules pinning UID 1000 will need
updating.
- **Helm chart image structure** refactored from `image.repository` to
  `imageRegistry` + `imageNamespace` + `image.name`. Cozystack's wrapper
  `values.yaml` is empty, so rendered image references are unchanged

(`quay.io/jetstack/cert-manager-{controller,webhook,cainjector,acmesolver}:v1.20.2`).
- **NetworkPolicy templates restructured**: legacy
`networkpolicy-egress.yaml`
  removed; per-component `networkpolicy-cainjector.yaml` and
  `networkpolicy-cert-manager.yaml` added. All gated on
  `*.networkPolicy.enabled: false` (default). No behavioral change.
- **CRDs**: `selectableFields` added (Kubernetes >=1.30) and Azure DNS01
`zoneType`
  enum. Both pure additions.
- **RBAC**: gains `clusterissuers/issuers/finalizers` and
  `gateway.networking.k8s.io.listenersets[/finalizers]`.
- `DefaultPrivateKeyRotationPolicyAlways` feature gate moves Beta -> GA,
can no
  longer be disabled. Already default since v1.18; no behavior change.

### Release note

```release-note
fix(cert-manager): bump to v1.20.2 - fixes HTTP-01 solver IngressClass
propagation for tenant-scoped Ingresses (the
`acme.cert-manager.io/http01-ingress-ingressclassname` annotation was a no-op
in prior cert-manager versions). Default container UID/GID changes from
1000/0 to 65532/65532 - review your PSP/SecurityContext rules.
```

[cert-manager#8244]:
https://github.com/cert-manager/cert-manager/pull/8244
[cert-manager#8655]:
https://github.com/cert-manager/cert-manager/pull/8655
[cert-manager#8665]:
https://github.com/cert-manager/cert-manager/pull/8665


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added network policy support for controller, webhook, and CA injector
  * Added Azure DNS zone type selection (public/private)
  * Added webhook client verification configuration options
* Extended pod disruption budget configuration with unhealthy pod
eviction policy

* **Updates**
  * Upgraded cert-manager to v1.20.2
  * Enhanced global image registry and namespace configuration
  * Expanded issuer reference field selectors
  * Added support for extra containers in controller pod
  * Updated issuer terminology and configuration

* **Documentation**
  * Updated Helm chart documentation with new configuration options
  * Updated post-installation guidance

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-06 15:56:23 +05:00
IvanHunters 418e3da527 fix(kafka): create LoadBalancer service when external flag is enabled
When `external: true` was set, the Kafka chart relied on Strimzi to
create a LoadBalancer service via listener configuration, but the
listener was unconditionally defined (as `type: internal` when disabled).
Additionally, the dashboard RBAC did not include the Strimzi-managed
external bootstrap service, making it invisible to tenant users.

- Make the external listener conditional: only add it to the Kafka CR
  when `external: true` is set
- Add `<release>-kafka-external-bootstrap` to dashboard RBAC rules
  when `external: true`, so the LB service is accessible from the
  tenant dashboard
- Add helm-unittest tests for both kafka listeners and dashboard RBAC
  covering the external/internal paths

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-05-06 12:17:30 +03:00
Myasnikov Daniil d8da80bdd2 fix(e2e): propagate failures from parallel LINSTOR pool jobs
Address review feedback on hack/e2e-post-install-prep.sh:31:
POSIX `wait` without arguments returns 0 even if a backgrounded
`kubectl exec` failed, which would mask partial LINSTOR prep failures
and let downstream tests run on a half-provisioned cluster. Collect
each background PID and `wait "$pid"` per job so `set -e` exits the
script on the first failed pool creation.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-06 12:24:01 +05:00
Myasnikov Daniil 209fa08737 test(e2e): decouple vminstance test from vmdisk test
"Create a VM Instance" referenced the VMDisk created by "Create a VM
Disk" via the shared name "test". This implicit ordering meant any
defensive pre-clean of vmdisks in test 2 would destroy the disk test 1
just provisioned, and the vm-instance chart's lookup-or-fail on the
DataVolume would then surface as "Specified disk not exists in cluster:
test".

Make each test self-contained:
- "Create a VM Disk" cleans up its own VMDisk at the end.
- "Create a VM Instance" creates its own VMDisk inline (renamed to
  "vmi-test" to avoid colliding with any in-flight finalizer drain on
  test 1's resources) and waits HR -> DV -> PVC before applying the
  VMInstance.

Replaces the pre-delete approach from the prior followup commit, which
broke the data dependency.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-06 08:59:42 +05:00
Aleksei Sviridkin 585ea7e089 test(e2e): smoke-test the ouroboros tenant addon
Auto-discovered hack/e2e-apps/ouroboros.bats spins up a tenant
Kubernetes CR with addons.ingressNginx + addons.ouroboros enabled
and asserts the cozystack-side wiring delivers a working hairpin
fix end-to-end:

  * the cozy-ouroboros HelmRelease in the tenant's host namespace
    reaches Ready (chartRef + admin-kubeconfig wiring is correct);
  * the cozystack-coredns wrapper rendered the kube-system/coredns-custom
    ConfigMap inside the tenant (the import-glob prerequisite is in
    place);
  * the controller pod is running in cozy-ouroboros inside the tenant;
  * after creating an Ingress with a TLS hostname, the tenant's
    coredns-custom ConfigMap eventually carries a `rewrite name <host>`
    line under the ouroboros.override key — proves the controller is
    reconciling, not merely Pod-Ready (the upstream chart ships no
    readiness probes, so a controller stuck on WaitForCacheSync would
    pass a pod-readiness check vacuously; the rewrite assertion catches
    every cozystack-specific failure mode the unit tests cannot — wrong
    namespace, missing CRDs, narrowed RBAC, gateway-api gate misfires).

End-to-end PROXY-protocol header roundtrip (the actual hairpin fix
in action) is covered by the upstream chart's own e2e on every push;
this file is the cozystack-side integration test that the cozystack
glue (chart vendoring, kubernetes-application registration, tenant
HelmRelease wiring, gateway-api gating, coredns-custom prerequisite)
holds together.

cozystack hack/cozytest.sh is plain shell, not real bats, so kubectl
output is captured directly without the `run` helper.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-06 06:48:07 +03:00
Aleksei Sviridkin 7f01c7eb32 feat(platform): single switch for host PROXY-protocol + ouroboros
Adds publishing.proxyProtocol on the platform values: a single flag
that both turns PROXY-protocol on at the host ingress-nginx (via the
ingress-application Package CR's component values dict) and emits the
new cozystack.ouroboros Package CR for the host install. The two
pieces are coupled by design — turning PROXY-protocol on at
ingress-nginx without ouroboros breaks intra-cluster traffic to the
cluster's own public hostnames (kube-proxy / Cilium-KPR short-circuits
the LB IP, the connection arrives at ingress-nginx without the PROXY
header it now requires, and ingress-nginx closes the connection).

The new sources/ouroboros.yaml PackageSource declares the host chart
with dependsOn: cozystack.networking + cozystack.ingress-application
so install ordering is deterministic. The bundle template emits the
host ouroboros Package via the standard helper — the parent chart's
Gateway-API watching default of false stays in place, because cozystack
does not install Gateway-API CRDs cluster-wide on the host (system/
gateway-api-crds is referenced only from the kubernetes-application
PackageSource, tenant scope). Forcing --gateway-api on a host without
the CRDs would CrashLoop the controller on WaitForCacheSync against
missing GVRs.

The flag default is false. Clusters that do not run PROXY-protocol get
zero new resources, zero RBAC delta, zero behaviour change.

Disabling: flipping publishing.proxyProtocol back to false stops
emitting the cozystack.ouroboros Package CR, but every Package CR
carries helm.sh/resource-policy: keep — Helm leaves the existing
Package on the cluster so ouroboros stays installed and the live
Corefile rewrite block keeps pointing at the still-running
ouroboros-proxy Service. Flag-flip alone is idempotent and safe. To
actually uninstall ouroboros the operator removes the Package CR
explicitly (or adds the package name to bundles.disabledPackages) and
runs the documented Corefile cleanup recipe in the same window —
otherwise stale rewrites point at a deleted Service and in-cluster
DNS for hairpinned hostnames returns connection-refused. The values
comment ships the cleanup recipe (a JSON merge-patch on data.Corefile
that strips the BEGIN/END block while preserving labels, annotations,
and other data keys including any Talos-managed metadata) and
documents the upstream-LB precondition (the L4 LB in front of
ingress-nginx must already be injecting PROXY-v1 headers before the
flag flips on), the host-vs-tenant fan-out (the flag does not cascade
into tenant clusters; tenants opt in per-Kubernetes-CR via
addons.ouroboros.enabled), and the Talos re-render flap window
(machine-config pushes wipe the BEGIN/END markers; the controller
re-applies on next reconcile, ≤10 minutes by default).

Also drops the stale "distro-full" entry from the bundles.system.variant
options comment in core/platform/values.yaml — the variant was removed
in v1.0.1 and was never restored.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-06 06:48:07 +03:00
Aleksei Sviridkin 7ab940f309 feat(kubernetes): expose ouroboros as a per-tenant addon
Adds the OuroborosAddon typedef, addons.ouroboros knob, and tenant
HelmRelease that deploys the cozy-ouroboros chart inside a tenant
Kubernetes cluster via the host's flux-helm-controller and the tenant
admin-kubeconfig — same shape as the cert-manager addon.

The HelmRelease overrides controller.mode back to coredns-import
because tenants run the cozystack-coredns wrapper (which carries the
`import /etc/coredns/custom/*.override` directive); the parent chart's
default of mode=coredns is the right shape only for the Talos-managed
host CoreDNS. controller.gatewayApi.enabled is mirrored from the
per-tenant addons.gatewayAPI.enabled — forcing --gateway-api on a
tenant without Gateway-API CRDs CrashLoops the controller on
WaitForCacheSync against missing GVRs. dependsOn grows a conditional
gateway-api-crds entry on the same gate.

controller.clusterDomain is intentionally NOT pre-set by the wrapper.
The chart auto-detects it inside the controller pod by parsing
/etc/resolv.conf — the search list of which is `*.<ns>.svc.<domain>`
and reflects exactly the zone the tenant CoreDNS actually serves
(governed by KamajiControlPlane.spec.networkProfile.clusterDomain
defaulting to `cluster.local`, and the cozystack-coredns wrapper
kubernetes-plugin zone, also `cluster.local`). Pre-setting from a
host-side fact like `_cluster.cluster-domain` would override the
auto-detect with a value that does not match what the tenant kube-dns
serves, silently emitting NXDOMAIN-bound rewrites. Operators that run
a tenant on a non-default cluster-domain can still override via
addons.ouroboros.valuesOverride.ouroboros.controller.clusterDomain.

If addons.ouroboros.enabled is true but ingressNginx is off, the
HelmRelease fails the render with a clear error rather than silently
skipping — the hairpin fix has nothing to fix against without
ingress-nginx, and a no-op addon that quietly does nothing is harder
to debug than an explicit render-time error.

The valuesOverride gateway-api guard accepts every truthy spelling
Helm/YAML emit (bool true, string "true"/"True"/"TRUE", numeric 1,
YAML "yes"/"Yes"/"YES"/"on"/"On"/"ON"/"y"/"Y") via
toString-then-lower-then-string-compare; direct bool eq would error
on type mismatch when an operator passes the value as a string from
--set, and case-insensitive folding catches `--set ...=On` /
`--set ...=True` that pure-lowercase comparison would miss.

The acknowledgeUncleanRemoval gate is defence-in-depth for the rare
edge case where helm uninstall fails to land cleanly (tenant
unreachable, hook timeout, manual kubectl delete bypassing helm) —
in the normal path the upstream chart's pre-delete cleanup hook
(coredns-cleanup-hook.yaml, vendored at 0.6.0) nulls the
ouroboros.override key automatically when the HelmRelease is
uninstalled. The gate only blocks render when a leftover HelmRelease
on the host kube-apiserver shows the previous install is still
sitting on the cluster.

The component is also registered in the kubernetes-application
PackageSource (as kubernetes-ouroboros at path system/ouroboros) so
the tenant HelmRelease's ExternalArtifact resolves.

make generate re-emitted README.md, values.schema.json, the Go struct
+ deepcopy, and the kubernetes-rd ApplicationDefinition openAPISchema
in lockstep.

helm-unittest covers: render gating on enabled flag, hard-fail when
ingress-nginx is off, namespaces and labels, chartRef, dependsOn
cilium/coredns/ingress-nginx + conditional gateway-api-crds, default
coredns-import mode, valuesOverride precedence, gateway-api flag
mirroring in both directions, gateway-api truthy-spelling acceptance
(including case-insensitive on/On/True/TRUE), the lookup-driven
acknowledge gate (kubernetesProvider mock asserts the failedTemplate
path when a prior HelmRelease still exists on the host), the
lookup-returns-nil CI/dry-run path (render proceeds when the cluster
mock has no objects), the _namespace.etcd-empty render gate, and
that controller.clusterDomain is NOT pre-set in the rendered
HelmRelease (so chart auto-detect runs).

Signed-off-by: Aleksei Sviridkin <f@lex.la>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-06 06:48:06 +03:00
Aleksei Sviridkin 9f8e7f9d4a feat(coredns): import /etc/coredns/custom into the Corefile
Extends the cozystack-coredns wrapper with the standard CoreDNS
`import /etc/coredns/custom/*.override` directive in the .:53 server
block and renders an empty kube-system/coredns-custom ConfigMap as
a chart-managed prerequisite, mounted into the CoreDNS Deployment
at /etc/coredns/custom/ via extraVolumes / extraVolumeMounts. The
ConfigMap carries helm.sh/resource-policy: keep so the chart-managed
empty value cannot strip what later writers add into it.

This exposes a stable extension point on the cozystack CoreDNS for
operators (or other cozystack components) that need to add plugin-only
rewrite snippets without touching the chart-rendered Corefile —
otherwise the chart would re-template data.Corefile on every Flux
reconcile and overwrite the additions.

The same package serves both host and tenant CoreDNS deployments, so
this plumbing covers both layers without per-layer duplication. The
volume mount is `optional: true` for defence-in-depth: a future chart
upgrade that drops the templates/coredns-custom.yaml render still
leaves CoreDNS healthy with no override snippets in play.

helm-unittest fixtures pin the ConfigMap shape (name, namespace, keep
annotation, no data), the `import` directive in the rendered Corefile,
the presence of the `reload` plugin (required for live ConfigMap
updates), and the volume + volumeMount on the CoreDNS Deployment.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-06 06:37:52 +03:00
Aleksei Sviridkin 28e88e9b1d feat(ouroboros): add cozy-ouroboros system chart
New packages/system/ouroboros/ vendors lexfrei/ouroboros 0.6.0 from
oci://ghcr.io/lexfrei/charts/ouroboros and pins cozystack-side defaults
for a hairpin-NAT fix on top of ingress-nginx + PROXY-protocol.

The controller watches Ingress (and optionally Gateway / HTTPRoute)
for public hostnames and rewrites cluster-internal DNS so those
hostnames resolve to a small in-cluster TCP proxy that prepends the
PROXY-protocol v1 header before forwarding to ingress-nginx — without
that, intra-cluster traffic to the cluster's own public hostnames
arrives at ingress-nginx without the PROXY header it requires and
ingress-nginx closes the connection.

cozystack pins:

  * controller.mode=coredns by default (mutates the live
    kube-system/coredns Corefile in place between BEGIN/END markers).
    The host CoreDNS on cozystack is Talos-managed and does not carry
    an `import /etc/coredns/custom/*.override` directive, so the
    alternative coredns-import mode would silently no-op there.
    Tenant deployments override this back to coredns-import via the
    apps/kubernetes wrapper (separate change).
  * proxy.target.host=ingress-nginx-controller.cozy-ingress-nginx.svc.cluster.local —
    cozystack deploys ingress-nginx into cozy-ingress-nginx in both
    the host control plane and tenant clusters; the chart-default
    `ingress-nginx` namespace exists nowhere in cozystack and would
    silently dial a non-existent Service.
  * controller.gatewayApi.enabled defaults to upstream's false at the
    parent level. The bundle template (separate change) flips it on
    for the host scope; the tenant wrapper mirrors per-tenant
    addons.gatewayAPI.enabled. Forcing --gateway-api on a tenant
    without Gateway-API CRDs CrashLoops the controller on
    WaitForCacheSync.
  * image is digest-pinned at
    ghcr.io/lexfrei/ouroboros:0.6.0@sha256:b33c415cd2edf231eb62076297fee1b75657bef2bfcc0be916b84088aa49479d and pulled
    directly from the upstream registry; this package is intentionally
    not mirrored under ghcr.io/cozystack/*. Air-gapped operators
    mirror both the chart and the image separately — bump together
    with OUROBOROS_CHART_VERSION in the Makefile.

The vendored chart, helm-unittest fixtures (mode dispatch, RBAC
narrowing per mode, Gateway-API gating, target FQDN), and chart
layout follow the same shape as packages/system/cilium and
packages/system/metallb.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-06 06:37:52 +03:00
Myasnikov Daniil cc7551cf1b test(e2e): use $name in kafka.bats kubectl wait (review feedback PR #2554)
Address lexfrei's nit on PR #2554: the pre-existing
`kubectl wait kafkas -n tenant-test test ...` on line 47 hardcoded the
resource name, while the existence backstop one line above was already
parametrized with `$name` in 12ccc53d. Switch the wait to `$name` for
consistency so changing the test's resource name in one place keeps the
two lines aligned.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-05 22:10:16 +05:00
Myasnikov Daniil fe138afab7 test(e2e): add --ignore-not-found to vminstance.bats end-of-test deletes
Symmetrize the post-success cleanup on lines 83-84 with the pre-clean
shape on lines 12-13/41. Without --ignore-not-found, a race that already
removed the resource (e.g., a parallel teardown or finalizer-driven GC)
turns a passing test into a failure after assertions have already run.

Review feedback PR #2555 (lexfrei follow-up #1).

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-05 22:06:17 +05:00
Myasnikov Daniil 7ed6444108 feat(installer,e2e): cozystackOperator.helmReleaseInterval value + parallel post-install prep
Two complementary changes that depend on the operator's
`--helmrelease-interval` flag landing via #2509:

1. Installer chart exposes `cozystackOperator.helmReleaseInterval`
   (default empty -> flag not rendered -> operator default 5m
   applies). E2E install passes `30s`. Eliminates the 8-10 min Flux
   dead zone where dependency-blocked HRs (waiting on cert-manager
   webhooks, CRDs) are requeued only every 5 min.

2. LINSTOR / StorageClass / MetalLB setup, previously sequential
   `@test` cases after the 15 m platform HR wait, now run as a
   background prep that completes during the platform reconcile.
   None of them gates platform HR reconcile, so they can move out
   of the critical path. Helper script `hack/e2e-post-install-prep.sh`
   polls for its prerequisites and is awaited at the end of the
   `Install Cozystack` test so failures still surface deterministically.

Includes the T2-race fix: wait on the LINSTOR HR being `Ready`
before polling for `deploy/linstor-controller`. The old 5-min hard
timeout fired during cold installs where the LINSTOR HR's dependency
chain takes longer than 5 min to resolve.

Production defaults are unchanged. The 30 s interval is opt-in per
install command.

Depends on:
- #2509 (operator-side `--helmrelease-interval` flag).

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
2026-05-05 14:41:24 +05:00
myasnikovdaniil 7520967d8d feat(operator): expose HelmRelease generation knobs (interval, retry-interval, timeouts, max-history) (#2509)
## Why

The `cozystack-operator` Package reconciler builds every `HelmRelease`
it generates with a handful of hardcoded values. Three of those have
caused recurring pain across the project:

- **`Spec.Interval = 5m`** (line 220 before this PR) — too long for E2E
install where dependent HRs reconcile out of order; previously addressed
by a manual `flux reconcile --force` workaround in
`hack/e2e-install-cozystack.bats`.
- **`Install.Remediation{Retries: -1}` + `Upgrade.Remediation{Retries:
-1}`** — semantically "retry forever, never remediate", but the
retry-on-failure timing was tied to `Spec.Interval` (5m). Audit of the
last 30 successful PR runs found that **every** run hit `kubectl wait
hr/seaweedfs-system` timeouts at the 2-min mark, recovering only after
the inline `flux reconcile --force` workaround.
- **`Install.Timeout = 10m` / `Upgrade.Timeout = 10m`** — directly
contradicts the per-Application timeout work in `pkg/config/config.go` +
`pkg/registry/apps/application/rest.go` (commit `7b146cbe`).
`packages/apps/tenant/templates/etcd.yaml` hand-rolls its HR YAML to
bypass this for cert rotation; harbor has had multiple commits chasing
the same problem.
- **`Spec.MaxHistory`** unset → Helm default 5. Test clusters bounce HRs
frequently and accumulate release-history Secrets.

This PR exposes all four as operator flags + helm chart values, with
defaults that preserve current behavior.

## What

### New operator flags

| Flag | Default | HR field |
|---|---|---|
| `--helmrelease-interval` | `5m` | `Spec.Interval` |
| `--helmrelease-retry-interval` | `30s` |
`Spec.{Install,Upgrade}.Strategy.RetryInterval` |
| `--helmrelease-install-timeout` | `10m` | `Spec.Install.Timeout` |
| `--helmrelease-upgrade-timeout` | `10m` | `Spec.Upgrade.Timeout` |
| `--helmrelease-max-history` | `5` | `Spec.MaxHistory` |

### Strategy switch

`Install.Strategy.Name` and `Upgrade.Strategy.Name` are now set to
`RetryOnFailure` (instead of leaving the default `RemediateOnFailure`),
with `RetryInterval` set from the new flag. Functionally equivalent to
the previous `Remediation{Retries:-1}` ("retry forever") since
remediation never fired anyway, but **decouples retry-on-failure timing
from `Spec.Interval`** so failed releases recover at 30s without polling
healthy releases at the same fast cadence.

### Chart exposure

`packages/core/installer/values.yaml` gets five new optional values
(`cozystackOperator.helmRelease{Interval,RetryInterval,InstallTimeout,UpgradeTimeout,MaxHistory}`),
each defaulting to empty so the operator's own default applies. The
chart conditionally renders the corresponding flag only when the value
is set.

## Production safety

- All static defaults match the previous hardcoded values (5m / 10m /
10m / 5). No new code path runs until an admin explicitly sets a flag.
- **One intentional behavior change: failed-release retry cadence**
drops from ~5m (previously coupled to `Spec.Interval`) to 30s
(`RetryInterval` default). This is the whole motivation for the strategy
switch — the recurring `kubectl wait hr/seaweedfs-system` flakes
documented above were caused by the previous coupling. Healthy-release
reconcile cadence is unchanged at 5m in production; CI overrides this to
30s via `--set cozystackOperator.helmReleaseInterval=30s` in
`hack/e2e-install-cozystack.bats` for faster convergence on E2E
clusters.
- `RetryOnFailure` is functionally equivalent to `RemediateOnFailure +
Retries:-1` since neither ever triggers remediation.
- Empty value in the chart → flag not rendered → operator falls back to
its built-in default. (`helmReleaseMaxHistory: 0` — unlimited history
per Helm semantics — is also honored; the chart guards on `ne (toString
…) ""` rather than truthiness so int `0` is not silently dropped.)

## Verification

- `go build ./...` clean
- `go test ./internal/operator/...` passes (covers `Interval`,
`MaxHistory`, `Install/Upgrade.Timeout`, `Strategy.Name`,
`RetryInterval`, `ChartRef` propagation, plus a regression test for
`MaxHistory=0`)
- `helm template installer packages/core/installer` renders cleanly with
default and overridden values; `--set
cozystackOperator.helmReleaseMaxHistory=0` correctly emits
`--helmrelease-max-history=0`

## Followups (deliberately not bundled)

- **Per-component timeout overrides** on `ComponentInstall` (mirroring
the `UpgradeCRDs` pattern from #2427). Charts like etcd that need a 30m
timeout would set it on their `Component` rather than tuning a global
flag. Touches the Package CR API + codegen — bigger lift, separate PR.
- **`Remediation.Retries` as a flag.** No current pain; skip.
- **`createOrUpdateHelmRelease` field preservation.** Currently
preserves `Suspend` but not `Test`/`Rollback`/`Uninstall`. Would
silently overwrite an SRE's manual edit. Minor cleanliness, separate PR.

## Test plan

- [ ] CI passes E2E with `helmReleaseInterval=30s` set in install bats
(existing T1 test path)
- [ ] No regression on production install path (default values)
- [ ] HR objects in a healthy cluster render with
`Strategy.Name=RetryOnFailure` and `RetryInterval=30s`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Operator and Helm chart now accept CLI flags and Helm values to
configure HelmRelease reconciliation intervals, retry behavior,
install/upgrade timeouts, and history retention.

* **Bug Fixes**
* Operator now validates HelmRelease timing/history inputs on startup
and will exit if values are invalid or non-positive.

* **Tests**
* Added unit and e2e coverage to verify HelmRelease configuration and a
chart install path using a custom reconciliation interval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-05 14:35:00 +05:00
myasnikovdaniil 086e9bb35b fix(installer): bootstrap cozy-system via --create-namespace + label hook (#2508)
## Why

helm v3 has a chicken-and-egg with charts that ship their own Namespace
resource:

- **With `--create-namespace`** on the install: helm pre-creates the
namespace via plain kubectl-create (no helm meta annotations); the
chart's own Namespace apply then fails with `already exists`.
- **Without `--create-namespace`**: helm fails immediately because it
can't write its release-secret to a non-existent namespace.

The cozy-installer chart shipped a `Namespace cozy-system` resource
since the chart's introduction (December 2025). Every cold install since
has hit this conflict. It was masked by the 3× retry on `Install
Cozystack` in `.github/workflows/pull-requests.yaml`: attempt 1 fails
with the conflict, attempt 2 sees the existing failed release and takes
the upgrade code path which patch-merges instead of strict-create.

Surfaced when retries on the install step were dropped.

## What

- Remove `Namespace cozy-system` from
`packages/core/installer/templates/cozystack-operator.yaml`.
- Add `packages/core/installer/templates/cozy-system-labels.yaml`: a
pre-install/pre-upgrade hook (ServiceAccount + ClusterRole +
ClusterRoleBinding + Job) that patches `cozystack.io/system=true` and
`pod-security.kubernetes.io/enforce=privileged` onto the namespace.
- Hook RBAC is scoped to `verbs: [get, patch]` on `resourceNames:
[cozy-system]` only.
- `hook-delete-policy: before-hook-creation,hook-succeeded` so the hook
surface only exists during install/upgrade.

Install commands now use `helm upgrade --install --namespace cozy-system
--create-namespace …` — standard pattern, matches kube-prometheus-stack
/ argo-cd / cert-manager.

## Verification

End-to-end on a kind cluster:
- Cold install: 3.3s clean, no retries needed
- Upgrade (re-run): 3.0s, idempotent, namespace stays labeled
- Helm uninstall + re-install: clean

## Test plan

- [ ] CI on this PR passes E2E
- [ ] Existing clusters: namespace already labeled correctly, hook
re-asserts on next upgrade — no migration needed

```release-note
fix(installer): cold installs no longer fail on the chart's Namespace resource colliding with `helm --create-namespace`; cozy-system is now bootstrapped via a pre-install hook.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added an automated pre-install/pre-upgrade step that applies system
identity and pod-security labels to the system namespace during
deployment, run in a hardened one-shot task before normal workloads
start.

* **Chores**
* Separated namespace initialization from the operator install flow so
the operator no longer renders or retains the system namespace directly,
improving installation sequencing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-05 14:25:44 +05:00
Myasnikov DaniilandClaude Opus 4.7 25180c58f7 fix(virtual-machine): bump HR readiness wait from 5s to 60s
5s was too tight under runner load — both vm-disk-$name and
vm-instance-$name HR waits could miss the operator's reconcile
window. Aligns with the other timeout bumps in this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-05 13:40:57 +05:00
Myasnikov Daniil 12ccc53d9d test(e2e): tighten event-driven backstops
- kafka.bats: use $name in the kafkas existence backstop, matching the
  variable used in the kubectl wait below it (consistency with the rest
  of this PR's backstops).
- e2e-install-cozystack.bats: the Tenant→HelmRelease materialisation is
  asynchronous (cozystack-controller observes the Tenant CR and creates
  the HR), so guard hr/tenant-test with the same existence backstop
  pattern used elsewhere.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-05 12:58:58 +05:00
Andrei Kvapil 90b3719f6a fix(mongodb): expose replica-set members per pod for external access (#2514) (#2522)
## What this PR does

Fixes #2514.

When `external: true` is set on a non-sharded `MongoDB`, Cozystack
rendered a single
`LoadBalancer` Service whose selector matched **all** replica-set
members
(`app.kubernetes.io/replset: rs0`). Round-robin balancing sent ~⅔ of
writes to
secondaries and they were rejected with `MongoServerError: not primary`.

Drivers couldn't recover on their own either: connecting through the LB
returned
the in-cluster DNS names of all members from `isMaster`, none of which
resolve
from outside the cluster, so automatic primary discovery failed.

This PR replaces the manual `external-svc.yaml` (for replica-set mode)
with the
Percona MongoDB operator's native `replsets[].expose` feature:

```yaml
replsets:
  - name: rs0
    expose:
      enabled: true
      type: LoadBalancer
      externalTrafficPolicy: Local
```

The operator then:
- creates one `LoadBalancer` Service **per pod** (`<release>-rs0-0`,
`-1`, `-2`)
  with `statefulset.kubernetes.io/pod-name` selectors,
- watches `Service.status.loadBalancer.ingress` for each pod and
rewrites the
  member `host` in `rs.conf` to the external IPs.

External clients use a standard replica-set URI
(`mongodb://...:27017,...:27017,...:27017/?replicaSet=rs0`) and the
driver
discovers the primary correctly from outside the cluster.

For **sharded** mode the existing single `LoadBalancer` is preserved —
it
front-ends stateless `mongos`, which can route writes to the right shard
primary itself. `external-svc.yaml` is now scoped to that case only.

### Tested

- `helm unittest .` in `packages/apps/mongodb/`: 91/91 pass.
- Applied on a dev stand:
  - `replsets[0].expose` is set as expected in the rendered PSMDB CR.
- The Percona operator (`v1.21.1`) accepts the spec and creates 3
per-pod
    `LoadBalancer` Services with `statefulset.kubernetes.io/pod-name`
selectors. The previous single round-robin Service is no longer
rendered.
- With an LB IP pool present, the operator promotes the allocated LB IPs
into
    the PSMDB `status.host` field and into `rs.conf`.

End-to-end mongosh write loop wasn't run on the dev stand because the
underlying network does not advertise/route allocated LoadBalancer IPs
(no
working L2/BGP path), so the rewritten `rs.conf` members are unreachable
in
that environment. The structural change is otherwise fully verified and
the
issue's reproduction scenario expects a real LB-capable network.

### Release note

```release-note
fix(mongodb): expose each replica-set member through its own LoadBalancer when `external: true`, so external clients can discover the primary via the standard replica-set connection string and writes no longer fail with "not primary" errors.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* External LoadBalancer services now only render when both external and
sharding are enabled.
* Replica set external exposure is now conditional based on the external
setting.
* Service configuration now uses LoadBalancer type with Local traffic
policy for external access.
* Service selectors refined for proper mongos targeting in sharded
deployments.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 20:40:58 +02:00
Andrei Kvapil 0b70197bf1 fix(kafka): bump default resourcesPreset to medium (#2537)
## What this PR does

Changes the default `resourcesPreset` for both `kafka` and `zookeeper`
in the Kafka managed application from `small` (1 CPU / 512Mi) to
`medium` (1 CPU / 1Gi).

The `small` preset is no longer sufficient for the current Strimzi 0.45
/ Kafka 3.9 image baseline:

- Strimzi sets JVM heap to 50% of the container memory limit for Kafka
and 75% for ZooKeeper. With a 512Mi limit that leaves only 256Mi (Kafka)
or 128Mi (ZooKeeper) for the JVM non-heap.
- Actual non-heap usage in the Kafka 3.9 image (Metaspace, code cache,
direct buffers, JMX javaagent, embedded Jetty admin server for
`/v1/ready` and KRaft migration endpoints, thread stacks) systematically
exceeds those budgets at startup, triggering a cgroup OOM kill (exit
137) before the broker can connect to ZooKeeper.

Result on a fresh deployment with the previous default: every Kafka
broker and every ZooKeeper pod ends up in `CrashLoopBackOff`, with
`lastState.terminated.reason: OOMKilled`. ZooKeeper occasionally hits
`OOMKilled` once quorum traffic ramps up, breaking the quorum and
causing Kafka brokers to fail with `ZooKeeperClientTimeoutException`.

The `medium` preset (1Gi) is the smallest preset that gives both Kafka
(heap 512Mi + ~500Mi non-heap budget) and ZooKeeper (heap 768Mi + 256Mi
non-heap budget) enough headroom to start cleanly. Users who need more
can still pick `large`, `xlarge`, or `2xlarge`, or override `resources`
explicitly.

The `small` preset is intentionally kept in the enum so existing
deployments that pin `resourcesPreset: small` are not broken; only the
default changes.

### Release note

```release-note
fix(kafka): default `resourcesPreset` for both Kafka and ZooKeeper is now `medium` (1 CPU / 1Gi). The previous default (`small`, 512Mi) is no longer sufficient under the current Strimzi 0.45 / Kafka 3.9 image and caused OOMKilled CrashLoopBackOff on fresh deployments. Existing deployments that explicitly set `resourcesPreset` are unaffected.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated default resource allocation preset from "small" to "medium"
for Kafka and ZooKeeper components across CRD type definitions, Helm
chart values, and JSON schema configurations.

* **Documentation**
* Updated parameter reference tables to reflect the new default resource
preset values for both Kafka and ZooKeeper.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 20:40:13 +02:00
Andrei Kvapil 0a3a5039d4 fix(kubevirt-operator): move PrometheusRule to Helm-rendered template (#2543)
## What this PR does

Move `alerts/PrometheusRule.yaml` from raw-file loading (via
`.Files.Glob` in
`templates/alerts.yaml`) to a proper Helm-rendered template
`templates/prometheusrule.yaml`.

The raw-file approach silently passed unescaped `{{ $labels.xxx }}` to
Kubernetes unchanged, which worked by accident. As a Helm template,
those
expressions must be escaped with `{{ "{{" }}` to prevent Helm from
trying
to evaluate them as Go template variables.

Also removes `templates/alerts.yaml` which became dead code once the
`alerts/` directory is empty.

### Release note

\`\`\`release-note
fix(kubevirt-operator): move PrometheusRule alerts to Helm-rendered
template and escape Prometheus template variables
\`\`\`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Reorganized alert configuration structure for VM and VMI monitoring.
Alerts that trigger when virtual machines or machine instances remain in
a non-running state for 10 minutes are now managed through an updated
template-based approach, consolidating alert definitions while
maintaining the same monitoring behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 20:38:44 +02:00
Andrei Kvapil 87f2f64414 chore(kubevirt-operator): update to v1.8.2 (#2502)
## What this PR does

Updates KubeVirt operator from v1.6.3 to v1.8.2.

This update brings:
- New CRD fields for changed block tracking (required for
storage-agnostic backups)
- Architecture support updates (s390x support, ppc64le deprecated)
- Updated operator manifest with latest capabilities

The standalone ServiceMonitor and alerts templates were removed as they
are now integrated into the main operator manifest.

### Screenshots

N/A - infrastructure component update

### Release note

\`\`\`release-note
chore(kubevirt-operator): update to v1.8.2

Updated KubeVirt operator from v1.6.3 to v1.8.2, adding support for
changed block tracking and latest architectural improvements.
\`\`\`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added new configuration fields for changed block tracking, feature
gate management, hypervisors, and VM backups.

* **Chores**
  * Upgraded KubeVirt operator to v1.8.2.
  * Updated RBAC permissions and cluster access policies.
* Removed Prometheus monitoring integration and alert template
processing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 20:38:12 +02:00
Myasnikov Daniil 7d292eb187 chore(e2e): drop seaweedfs/monitoring flux-reconcile workaround
With Strategy=RetryOnFailure + RetryInterval=30s plus the bats
--set cozystackOperator.helmReleaseInterval=30s override, the existing
2m wait converges without the manual flux reconcile --force band-aid.
Removing the workaround turns CI itself into the test of the fix.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 22:13:45 +05:00
Myasnikov Daniil 730037c85d fix(installer): address review feedback on cozy-system bootstrap
- Bump alpine/k8s to 1.33.4 (was 1.32.0, EOL); matches in-tree pin used
  by cozystack-api, seaweedfs, vm-disk, and vm-instance hooks.
- Skip hostNetwork=true and not-ready/CNI tolerations on the hosted
  variant: managed clusters provide CNI before install, and kube-system
  may carry strict PSA there that would reject hostNetwork pods.
- Add packages/core/installer/templates/cozy-system-namespace.yaml gated
  on .Values.bareNamespace (default false). The top-level Makefile's
  manifests target now passes --set bareNamespace=true and renders
  Namespace + operator into _out/assets/cozystack-operator-*.yaml so
  the kubectl-apply install path stays self-contained. Helm
  install/upgrade callers keep the default and bootstrap via
  --create-namespace + the pre-install labeler hook.
- Add a bats assertion in hack/e2e-install-cozystack.bats checking
  cozy-system carries pod-security.kubernetes.io/enforce=privileged and
  cozystack.io/system=true after helm install.
- Inline comment near hostNetwork=true documenting why dnsPolicy left
  at Default (cluster DNS not up at install time;
  ClusterFirstWithHostNet would not help).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 22:13:39 +05:00
Myasnikov Daniil 929c623af1 test(operator): assert Install/Upgrade Remediation stays nil
helm-controller's XValidation rule rejects Strategy.Name=RetryOnFailure
with RetryInterval set alongside a Remediation entry, so a future
"for safety" reintroduction of Remediation{Retries: -1} would silently
break every HR. Pin the deliberate omission.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 22:13:38 +05:00
Myasnikov Daniil d0e4b4dbe5 refactor(operator): extract parsePositiveDuration into testable helper
The previous closure inside main() couldn't be tested because every error
path called os.Exit(1). Lifting it to a package-level function returning
(time.Duration, error) lets a table test exercise happy paths and rejections
("0s", "-5m", "5x", empty input) directly. main() retains the os.Exit
behaviour via a thin wrapper.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 22:13:29 +05:00
Myasnikov Daniil 266465b410 fix(operator): copy MaxHistory to local before taking address
Avoid pointer aliasing across all generated HelmReleases. r.HelmReleaseMaxHistory
is set once at startup today, but a future mutation through the field would
silently affect every in-flight HelmReleaseSpec.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 22:13:21 +05:00
myasnikovdaniil d9023c9b15 feat(cozyreport): collect Flux/cert-manager/host context + summary.txt (#2553)
## What this PR does

Makes `cozyreport.tgz` actually useful when triaging an E2E failure
deeper than a single pod log. Adds:

- Flux controller logs (helm-controller, source-controller,
notification-controller, kustomize-controller, last 2000 lines each)
- Flux source resources (`HelmRepository`, `OCIRepository`,
`GitRepository`, `ExternalArtifact`)
- Decoded Helm storage secrets for non-Ready HRs (`base64 -d | base64 -d
| gzip -d`)
- Cluster events (all + warning-only filtered file)
- cert-manager `Certificate` / `CertificateRequest` / `Order` /
`Challenge` resources + cert-manager logs
- `cozystack-operator` deployment logs (current + previous)
- `Application` / `ApplicationDefinition` / `Tenant` resources
- Sandbox host context per node: `df`, `free`, `ps`, `dmesg`, `talosctl
logs/dmesg/kubelet/containerd`

New executable `hack/cozyreport-summary.sh` writes a `summary.txt` at
the archive root listing what is broken right now — the first thing to
read when downloading the artifact from a CI failure.

Surfaced from #2500.

### Release note

```
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a timestamped diagnostic summary report that highlights failing
resources and key cluster issues.
* Expanded diagnostic collection: controller logs (current+previous),
Flux source status, cert-manager certificate details, Helm release
payloads, and Cozystack resource status.
* Added host-level diagnostics (disk/memory/process/dmesg) and improved
event/warning listings to aid troubleshooting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 20:39:44 +05:00
myasnikovdaniil 36dbd6b27c test(tests): pre-pull timing-sensitive platform images before install (#2556)
## What this PR does

Some workloads (kube-ovn raft, LINSTOR controller / piraeus-server) fail
when replicas start at materially different times due to image-pull
stagger across nodes. Adds a DaemonSet-based pre-pull step that runs
before `helm install`, ensuring every node has the images cached so
replicas start within milliseconds of each other.

`hack/e2e-prepull-images.sh` accepts image refs on stdin and creates one
container per image so pulls run in parallel — total time = max(any
single image) rather than sum. The bats test sources image refs directly
from the rendered platform charts via `yq`, walking only PodSpec-shaped
objects, so version bumps stay in sync without a hardcoded list.

Surfaced from #2500.

### Release note

```
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Added an end-to-end test that pre-pulls platform container images
before the installation workflow to improve install reliability.

* **Chores**
* Added an automated pre-pull utility that reads image lists,
deduplicates and sorts them, deploys a short-lived pre-puller across
nodes, waits for completion, then cleans up.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 18:52:11 +05:00
myasnikovdaniil 6ad5bf0bbf fix(objectstorage-controller): retry on Bucket conflict during BucketAccess reconcile (#2529)
## What this PR does

Carries an upstream-bound patch against COSI v0.2.2's BucketAccess
sidecar reconciler so it survives the optimistic-concurrency race with
the central Bucket reconciler running in the same process.

In `sidecar/pkg/bucketaccess/bucketaccess_controller.go`, the `Add` path
does:

```go
bucket, _ := buckets().Get(...)
// ...
if controllerutil.AddFinalizer(bucket, BABucketFinalizer) {
    buckets().Update(ctx, bucket, ...)   // <-- 409 here
}
```

Between `Get` and `Update` the Bucket reconciler in the same binary can
mutate the object and bump `resourceVersion`, producing:

```
Operation cannot be fulfilled on buckets.objectstorage.k8s.io "bucket-...":
the object has been modified; please apply your changes to the latest
version and try again
```

surfaced to users as a `FailedGrantAccess` event on the BucketAccess.

This patch wraps the Bucket finalizer add in
`retry.RetryOnConflict(retry.DefaultRetry, ...)` and re-`Get`s the
Bucket inside the closure so each retry mutates the freshest version.

Patch is held as `91-bucketaccess-conflict-retry.diff` and consumed at
image build time (re-introduces `git` to the source stage and the `COPY
patches /patches` + `git apply /patches/*.diff` steps that existed
before `c29d501b` dropped them when 89/90 were upstreamed). The
convention is identical: drop the local patch the moment the upstream PR
merges and ships in a tagged release.

### Release note

~~~release-note
[objectstorage-controller] Carry a patch that retries on Bucket update
conflict during BucketAccess reconcile, eliminating the "object has been
modified" FailedGrantAccess event seen in CI.
~~~

## Test plan

- [x] `make image` for `system/objectstorage-controller` and verify
`values.yaml` picks up the new digest.
- [x] Re-run the cozystack-e2e suite that emitted the
`FailedGrantAccess: ... object has been modified` event on
`tenant-test/BucketAccess/bucket-test-admin` and confirm it no longer
fires.
- [ ] Verify `kubectl describe bucketaccess` reports
`AccessGranted=true` on first reconcile, no requeue from the conflict
path.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced bucket access update reliability with improved conflict
handling during concurrent operations. The system now automatically
retries operations and refetches data to prevent update conflicts.

* **Chores**
* Improved Docker build process with automatic patch application support
for source customization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 18:50:38 +05:00
myasnikovdaniil 8fbbef3f02 fix(harbor): drive bucket-secret.yaml from values, gate HR on BucketInfo (#2528)
## Summary
- `cozy-harbor/templates/bucket-secret.yaml` no longer relies on
`lookup` against the BucketAccess credentials Secret. It is now driven
by `.Values.bucket.bucketInfo` (a JSON string) with `dig`-based safe
accessors, so a missing/empty/partial value renders nothing rather than
crashing with `index of untyped nil` (the failure mode that broke harbor
in CI #25049445125).
- The downstream `<release>-system` HelmRelease now sources `BucketInfo`
via `valuesFrom` (`valuesKey: BucketInfo`, `targetPath:
bucket.bucketInfo`). With the default `optional: false`, helm-controller
refuses to compose values until the COSI BucketAccess controller has
populated the Secret — both gating initial reconciliation and forcing a
config-digest change (and thus a helm upgrade) when the credentials
arrive. Flux HR `dependsOn` cannot reference COSI resources directly, so
`valuesFrom` is the correct primitive here.
- `bucket.secretName` was the only consumer of the old `lookup` and is
removed from both the system chart's `values.yaml` default and the apps
chart's `values:` block.

## Why this shape
helm-controller's upgrade trigger is digest-based over composed values +
chart artifact; a `lookup` returning new data on a later reconcile is
not enough to force an upgrade on its own. Routing `BucketInfo` through
`valuesFrom` puts the bucket data on the digest path, so the chart
re-applies the moment credentials become available.

## Test plan
- [x] `helm template` of `packages/system/harbor` with
`bucket.bucketInfo` unset, empty string, empty JSON object `{}`, and
fully populated — only the populated case renders the `*-registry-s3`
Secret; the others render nothing without erroring.
- [x] `helm template` of `packages/apps/harbor` produces the expected
`<release>-system` HR with the new `valuesFrom` entry and the cleaned-up
`values:` block.
- [ ] Re-run the harbor E2E on a fresh stand and confirm 15/15.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Harbor bucket config now comes from a centralized Secret merged into
chart values; in-values bucket entry removed.
* **Bug Fixes**
* Deployment synchronization and Helm upgrade behavior improved by
depending on the Secret’s readiness/contents.
* **Tests**
* Added unit tests covering when the bucket secret is rendered (or not)
and a make target to run those tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 17:21:17 +05:00
Aleksei Sviridkin 7eaafe0b40 fix(mongodb): reclaim PVCs and operator secrets on uninstall (#2525)
## What this PR does

Adds the upstream Percona finalizer pair to the `PerconaServerMongoDB`
custom resource so the operator performs an ordered shutdown and
reclaims persistent storage when a MongoDB release is uninstalled.
Previously, deleting a MongoDB application left PVCs and
operator-managed secrets orphaned in the tenant namespace, requiring
manual cleanup before the same release name could be reused.

The finalizers added on the CR:

- `percona.com/delete-psmdb-pods-in-order` — ensures the primary is shut
down last for graceful cluster termination.
- `percona.com/delete-psmdb-pvc` — instructs the operator to delete PVCs
and operator-managed secrets when the CR is removed.

Coverage applies to all replica sets in both replicaset and sharded
modes, including the config server in sharded deployments.

### Behavior change

Users who previously relied on orphaned PVCs as an implicit "soft
delete" for data recovery should be aware that uninstalling a MongoDB
release now removes the underlying storage and operator-managed
credentials. The README's new "Data lifecycle" section enumerates
exactly which resources are reclaimed by the operator versus by the
standard `helm uninstall`. Take a backup before deletion if data
retention is required.

This resolves cozystack/cozystack#2513.

### Screenshots

Not applicable — no UI changes.

### Release note

```release-note
fix(mongodb): reclaim PVCs and operator-managed secrets on uninstall via Percona finalizers.

WARNING: upgrading scopes the admin credentials secret per release instead of the namespace-shared default. This triggers an in-place rotation of the five operator-managed admin passwords on first reconcile. Pods are not restarted; application user passwords are unaffected. Workloads consuming `<release>-credentials` must be restarted to pick up the new admin passwords.
```

### Out of scope / follow-up

- Bats integration test would require a live cluster with the Percona
MongoDB operator and belongs in the `hack/e2e-*.bats` family — left as a
follow-up.
- Issue #2513 mentions considering similar treatment for
PostgreSQL/MariaDB/Redis/ClickHouse. Each operator has a different
cleanup mechanism; that work is intentionally out of scope here and
should be tracked per-app.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Expanded uninstall and upgrade guidance: which secrets/pvc remain,
cert-manager TLS secrets must be deleted manually, remediation for
orphaned legacy secret, and recovery steps for stuck deletions when the
operator is removed before CRs.

* **New Features**
  * Added deletion finalizers to enforce controlled teardown.
* User credentials secret now uses a deterministic, release-scoped name.

* **Tests**
* Added chart tests for finalizer behavior (sharded/non‑sharded) and
release-scoped users secret.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 13:18:02 +03:00
Myasnikov Daniil 9df5950731 test(tests): pre-pull timing-sensitive platform images before install
Some workloads (kube-ovn raft, LINSTOR controller / piraeus-server)
fail when replicas start at materially different times due to
image-pull stagger across nodes. Add a DaemonSet-based pre-pull step
that runs before `helm install`, ensuring every node has the images
cached so replicas start within milliseconds of each other.

The script accepts image refs on stdin and creates one container per
image so pulls run in parallel — total time = max(any single image)
rather than sum. The bats test sources image refs directly from the
rendered platform charts via `yq`, walking only PodSpec-shaped
objects, so version bumps stay in sync without a hardcoded list.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
2026-05-04 12:09:55 +05:00
Myasnikov Daniil 1dc33dff79 fix(operator): reject non-positive HelmRelease duration flags
time.ParseDuration accepts "0s" and negative values, but Flux
HelmRelease.Interval / Install.Timeout / Strategy.RetryInterval require
strictly positive durations to function. Validate at startup so a
misconfigured flag fails fast instead of propagating into every
generated HelmRelease.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 11:41:35 +05:00
Myasnikov DaniilandClaude Opus 4.7 b40ab290eb fix(cozyreport): portability and Ready-filter fixes from review
- Resolve hack/cozyreport-summary.sh via SCRIPT_DIR so cozyreport.sh
  can be invoked from any working directory, not only the repo root.
- Discover Talos node IPs via `kubectl get nodes` (with the existing
  192.168.123.11-13 list as fallback) instead of hardcoding the E2E
  sandbox topology.
- Replace broken `awk '$NF != "True"'` filter on the cozystack-apps
  loop with a jsonpath check against the Ready condition; the previous
  filter compared AGE ("10m" etc.) against "True" and let every
  resource through, defeating the non-Ready filter.
- Switch the Flux source CRD summary loop from column parsing to
  jsonpath (column positions differ between HelmRepository and
  OCIRepository printer columns) and add ExternalArtifact, matching
  what cozyreport.sh already collects.
- Add `kubectl get crd` guards before HR/source CRD queries so the
  summary script does not abort under `set -eu` when Flux v2 is not
  installed.
- Relabel "OOMKilled in last 30 min" to "Recent OOMKilled events
  (last 20)" — the kubectl call only limits count, not time.

Surfaced from coderabbit and gemini review on #2553.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 11:17:35 +05:00
Myasnikov Daniil 6ca5566391 fix(operator): handle MaxHistory=0 and cover HelmRelease spec build
Chart template now uses `ne (toString …) ""` for the five helmRelease*
knobs, so integer 0 (notably MaxHistory=0 = unlimited history per Helm
semantics, documented in values.yaml) is no longer silently dropped.

Extract buildHelmReleaseSpec from Reconcile and add unit tests
asserting Interval, MaxHistory, Install/Upgrade timeouts,
Strategy=RetryOnFailure, RetryInterval, and ChartRef propagate from
PackageReconciler config to the generated HelmRelease.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 11:14:38 +05:00
Myasnikov DaniilandClaude Opus 4.7 85ced0d729 fix(installer): harden cozy-system-labeler hook and add hosted variant
Address review feedback on PR #2508:

- Pin alpine/k8s by digest (matches operator image pinning style).
- Add resources requests/limits and a minimal securityContext to the
  labeler Job: capabilities drop ALL, allowPrivilegeEscalation: false,
  seccompProfile RuntimeDefault.
- Include hook-failed in helm.sh/hook-delete-policy across all four
  hook resources so a failed install doesn't leave SA / ClusterRole /
  ClusterRoleBinding / Job objects behind.
- Add explicit hosted variant case in the env block, mirroring the
  pattern in cozystack-operator.yaml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 10:59:36 +05:00
Myasnikov Daniil b6badeeb96 fix(cert-manager): bump chart to v1.20.2
cert-manager v1.20.0 introduced the
acme.cert-manager.io/http01-ingress-ingressclassname Ingress annotation
(cert-manager#8244). Cozystack templates already emit it (since
2b6e20cc) but v1.19.x silently ignored it, so cert-manager fell back
to the ClusterIssuer's solver class. HTTP-01 solver Ingresses for
non-publishing tenants ended up on the wrong IngressClass, breaking
Let's Encrypt issuance for any tenant ingress not on tenant-root.

Targeting v1.20.2 picks up cert-manager#8655 (clusterissuers/issuers
finalizer RBAC restored after v1.20.0 dropped it) and #8665 (webhook
helm-template fix).

Verified live on dev10:
- v1.19.3: Challenge solver class = "tenant-root" (BUG)
- v1.20.2: Challenge solver class = "<tenant>" (FIXED)

Notable upgrade considerations:
- Container UID/GID change 1000/0 -> 65532/65532
- New per-component NetworkPolicy templates (gated, default off)
- Image template signature refactored (cozystack values.yaml empty,
  renders identically)
- CRDs gain selectableFields (k8s >=1.30) and Azure DNS01 zoneType
- RBAC adds clusterissuers/issuers finalizers + listenersets

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-03 20:49:47 +05:00
Myasnikov Daniil 4672d88f72 fix(ci): fetch full history in E2E checkout for TIA selector
The "Select E2E tests" step runs `git diff --name-only
origin/${BASE_REF}...HEAD` inside the sandbox copy of the workspace.
Without fetch-depth: 0 the e2e job's checkout only resolves a single
commit, so origin/main is missing and the diff aborts with
"ambiguous argument 'origin/main...HEAD': unknown revision".

Mirror the Build job's setup and request full history.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-03 20:47:18 +05:00
Myasnikov Daniil 77f80b597f fix(ci): make select-e2e_test.bats run under cozytest.sh
CI invokes hack/cozytest.sh, a project-local pure-shell harness, not
bats. cozytest does not honor setup()/teardown() and does not provide
bats' run / $status / $output, so the original tests failed with
"TMPDIR: parameter not set" under set -u.

Inline tmpdir creation and an EXIT trap for cleanup in each test.
Replace `run` + `$output` with direct command substitution, and
replace `! grep` (which set -e ignores) with an explicit if-then-exit.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-03 20:00:52 +05:00