The chart placed `faultDomain` nested inside `automationOptions`, but the
FoundationDBCluster CRD schema defines `faultDomain` as a direct child of
`spec`. Flux v2.7 (helm-controller v1.4.3) used permissive client-side
patch and silently dropped the unknown field; v2.8 (helm-controller
v1.5.0) does strict Server-Side Apply and rejects:
failed to create typed patch object: .spec.automationOptions.faultDomain:
field not declared in schema
This was a latent bug from day one — `automationOptions.faultDomain`
never actually configured anything. Moving the field one level up applies
the value to where the operator looks for it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The bats step had two waits for tenant-root child HRs:
kubectl wait hr/etcd hr/ingress hr/tenant-root hr/seaweedfs --timeout=4m
kubectl wait hr/monitoring hr/seaweedfs-system --timeout=2m
The second line was the residue of a flux-reconcile-force workaround
removed in 7d292eb1; the mechanical cleanup kept the same HRs on a
separate 2m wait without revisiting whether the line still made sense.
After PR #2509 fixed the underlying timing bug:
- monitoring is a top-level tenant-root child, identical shape to etcd
and ingress; its Ready is already implied by hr/tenant-root Ready on
the first line.
- seaweedfs-system is an internal HR of the seaweedfs umbrella chart;
its Ready is already implied by hr/seaweedfs Ready on the first line.
So the second wait was always tautologically true by the time it ran,
and the naming inconsistency (top-level monitoring next to internal
seaweedfs-system) was an accident of the historical workaround. Promote
monitoring to the first line where it belongs and drop the redundant
second wait.
Also bump the wait from 4m to 10m. seaweedfs now installs as a serial
chain seaweedfs-db (CNPG bootstrap) -> seaweedfs-system (raft quorum)
-> seaweedfs wrapper, so the parent tenant-root HR's Ready flip lags
~5-6 min. The previous 4m budget was sized for the old single-HR
seaweedfs and is the immediate cause of the e2e failure on PR #2619.
The 10m budget stays inside tenant-root HR.spec.timeout (15m).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Mirror the diagnostics group already on `Run E2E tests` to the
`Install Cozystack into sandbox` step. Without this, install-step
failures (e.g. an HR not reaching Ready inside the bats `kubectl wait`
window) surface only as "error: timed out waiting for the condition
on helmreleases/<name>" with no cluster snapshot — re-running is the
only way to triage, which is the exact pattern the parent commit
removed.
On failure, dump three groups:
- `kubectl get hr -A -o wide` (last 100 lines): which HRs exist and
their Ready/Status columns.
- `kubectl describe hr` for each not-Ready HR (tail -50): the
Conditions block surfaces helm install/upgrade errors, missing
CRDs, and chart-render failures — the actual cause.
- `kubectl get events -A --sort-by=.lastTimestamp` (last 50): cluster
events that often pinpoint the failing pod/PVC/job.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on
.github/workflows/pull-requests.yaml:235: $attempt is incremented only on
failure, so first-try success printed "completed after 0 attempts". Use
$((attempt + 1)) so the message reflects the total number of runs.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Audit of 30 successful PR runs found that across 5 sampled failure
attempts, **25/25 retries** on `Run E2E tests` failed — the retry
loop never recovered a flake, only stretched deterministic failures
and tripled diagnostic wall-time. Same data shape on `Install
Cozystack`. Drop both.
`Prepare environment` keeps its 3x retry — that step is pure
infrastructure (Talos image download, sandbox VM boot, network)
where transient runner hiccups warrant a retry.
On failure, the test step now captures `kubectl get hr -A -o wide`
and `kubectl get events -A` under a collapsible group so triage
starts with the actual broken-state snapshot.
Depends on:
- #2508 — installer namespace bootstrap (Helm namespace-ownership conflict)
- #2509 — operator HelmRelease config knobs (`seaweedfs-system` 2-min wait race)
- #2528 — harbor bucket-secret + BucketInfo gating (harbor ValuesError)
- #2529 — objectstorage-controller BucketAccess conflict retry
- the daniil/split-vminstance PR (vminstance disk race + VM IP/ready timeouts)
- the daniil/split-event-driven PR (existence backstops surfacing real errors)
Until those land, dropping the retry will fail CI for unrelated PRs
that hit the seaweedfs / harbor / installer / vminstance races.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
PR #2619 split seaweedfs into a serialized chain
(seaweedfs-db -> seaweedfs-system -> seaweedfs) that needs ~5 min of
wall-clock to install: CNPG bootstrap, master quorum, then the wrapper.
With the prior 5m wait budget on the parent tenant HR, helm-controller
would time out before the children reach Ready, rollback would delete
the child HRs, and the upgrade would retry in a loop. The e2e bats step
that patches the Tenant CR and waits 60s for child HRs to exist landed
in the rollback gap and timed out.
Two complementary changes:
- tenant-root is created statically by cozystack-basics, so its
HelmRelease spec.timeout is bumped 5m -> 15m directly.
- Sub-tenants (Tenant CRs) get their HR built by cozystack-api from
the tenant ApplicationDefinition. Add the
release.cozystack.io/helm-install-timeout=15m annotation that
kubernetes-rd already uses to opt the kind into the longer budget.
Keeps both values in lockstep and documents the coupling inline.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The previous healthCheckExprs flagged the seaweedfs-db HR as failed within
seconds of creating the Cluster CR. CNPG sets Ready=False with
reason=ClusterIsNotReady as the initial bootstrap condition before postgres
starts, so the generic `failed: ... Ready=False` predicate matches the
normal startup state and aborts the install early. This cascades: tenant-root
times out waiting for seaweedfs, rolls back (deleting child HRs), retries,
and the e2e "Configure Tenant" step races the rollback gap.
Use Flux's three-predicate form: route ClusterIsNotReady to `inProgress`
so the HR keeps polling, keep `failed` for any other future Ready=False
reasons, and rely on the HR `timeout: 10m` as the real backstop for a
genuinely stuck cluster.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Each of the 19 child HR templates in packages/apps/kubernetes/templates/
helmreleases/ added the parent kubernetes-application HR as a dependsOn
entry, guarded by a `lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease"`
on the parent so it only stamped after the parent already existed.
Under Flux 2.7 (helm-controller v1.4.x), the parent HR's Ready=True flips
after helm install returns, regardless of any child HRs it just applied,
so the guard was a no-op for first install and a soft ordering hint for
subsequent reconciles.
Under Flux 2.8 (helm-controller v1.5.0), the default waitStrategy is
`poller` -- the parent HR now waits via kstatus for every applied
resource, including the 19 child HRs, to be Ready=True before flipping
its own Ready. The child HRs in turn cannot become Ready until they have
a working `super-admin.svc` kubeConfig and the lookup-stamped dependsOn
back-edge resolves -- which only happens after the parent HR is Ready.
Deadlock: parent waits on child, child waits on parent.
Removing the parent dependsOn from each child breaks the cycle. The
children start reconciling immediately; helm-controller's existing retry
loop covers the brief window before the parent's helm install lands the
admin-kubeconfig secret. The cross-child ordering (cilium -> cert-manager,
prometheus-operator-crds -> metrics-server, etc.) stays intact via the
unconditional sibling entries.
Five files (the *-crds and vsnap-crd HRs) had only the parent reference
under dependsOn; remove \`dependsOn:\` entirely in those. Cilium had one
conditional gateway-api-crds entry under it; pull the \`dependsOn:\` key
inside that conditional so it disappears when gateway-API is off rather
than rendering as a null array.
This is option B from issue #2642 task 1.
Verified:
- \`helm template\` clean with default values-ci.yaml and with every addon
enabled (cert-manager, ingress-nginx, monitoring-agents, velero, fluxcd,
gateway-api, gpu-operator, hami, ouroboros).
- No remaining \`- name: {{ .Release.Name }}\` dependsOn entries in
packages/apps/kubernetes/templates/helmreleases/.
- \`helm unittest\` passes (123/123).
Refs: #2642
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The packages/system/vertical-pod-autoscaler chart's vpa-for-vpa.yaml template
emits an HR in cozy-vpa-for-vpa that the parent helm install applies and waits
on. That nested HR had `dependsOn: monitoring-agents`, which never resolves:
parent VPA helm install
waits for nested vpa-for-vpa HR Ready
waits for monitoring-agents HR Ready
waits for cozystack.vertical-pod-autoscaler PackageSource Ready
waits for parent VPA HR Ready <-- back to start
Symptom observed on PR #2619 e2e (cozyreport):
Helm upgrade failed for release cozy-vertical-pod-autoscaler/
vertical-pod-autoscaler with chart cozy-vertical-pod-autoscaler@...:
timeout waiting for: [HelmRelease/cozy-vpa-for-vpa/vpa-for-vpa
status: 'InProgress']
The recommender's /health-check readiness probe is served by the binary
itself and doesn't require Prometheus. Removing the dependsOn lets the
nested HR become Ready quickly; the recommender will log connection
errors against vmselect until monitoring-agents installs later in the
boot sequence, then start producing recommendations normally.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Main has a conditional dependsOn:[ingress-nginx-system] in the application
HR (added after #2601 branched). PR #2601 added an unconditional
dependsOn:[seaweedfs-db]. The merge produced two `dependsOn:` keys in the
same HelmRelease spec, which the Helm post-render rejected:
Helm install failed for release tenant-root/seaweedfs ...:
error while running post render on files: map[string]interface {}(nil):
yaml: unmarshal errors
Collapse into one list: the db dependency is unconditional; the ingress
dependency stays inside the existing `if eq $ingress .Release.Namespace`
guard so sub-tenants that inherit ingress from a parent namespace don't
deadlock on a non-existent local HR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The pre-pull test captured each rendered chart into a shell variable
(`kubeovn_manifests=$(helm template ...)`) so a render failure would
trip `set -e` (bats has no `pipefail`, so piping helm into yq would mask
a helm failure with yq's exit code).
cozytest.sh runs every bats test under `set -x`, and bash expands
captured command output into the trace as `+ var=<entire stdout>`.
Two full helm renders (kubeovn + linstor) plus a `printf` re-expansion
were ballooning the install step to ~40k log lines per CI run.
Stage each render to a tmp file instead. The same exit-code semantics
apply (a failed `helm template ... > file` still trips `set -e`), but
the trace now only shows the commands themselves, not the rendered YAML.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
`make generate` in packages/apps/vm-instance/ rebuilds the instanceProfile
enum from packages/system/kubevirt-instancetypes/templates/preferences.yaml.
The pre-commit job picked up the drift after the upstream refresh in the
previous commit. No source changes — only regenerated outputs.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
`make update` stripped `persistent: true` from upstream's
`preferredTPM: { persistent: true }`, leaving the parent key with a null value.
KubeVirt v1.6's CRD tolerated it; v1.8 (the operator version we bumped to in
6af0bf71) enforces structural schema and rejects null:
VirtualMachineClusterPreference "windows.11" is invalid:
spec.devices.preferredTPM: Invalid value: "null": spec.devices.preferredTPM
in body must be of type object: "null"
Stop stripping the line and regenerate the templates from upstream
common-instancetypes. Persistent TPM state falls back to the cluster default
StorageClass with RWO access mode when `vmStateStorageClass` is unset on the
KubeVirt CR, per pkg/storage/backend-storage/backend-storage.go.
Drive-by from the regen:
- Removed (upstream EOL): centos.7*, centos.stream8*
- Added: debian, oraclelinux, legacy, linux*, fedora.s390x, rhel.{9,10}.s390x,
windows.{xp,7,7.virtio,2k3,2k8,2k8.virtio,2k12,2k12.virtio}, plus new
cx1.*1gi and d1 instancetype variants.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Splits the seaweedfs-system HelmRelease in two so the CNPG
Cluster/seaweedfs-db lives in its own HR (seaweedfs-db), and the
application HR (seaweedfs-system) dependsOn it. The new HR uses Flux v2
HelmRelease.spec.healthCheckExprs with a CEL expression on
Cluster.status.conditions[type=Ready] plus waitStrategy.name: poller, so
its Ready=True only flips after the postgres primary is actually serving
connections — not just after helm install applied the Cluster CR.
This eliminates the seaweedfs-filer CrashLoopBackOff race on a fresh
tenant install. With Cilium kubeProxyReplacement: true, socket-LB
returns EPERM from connect(2) to ClusterIPs with no Ready endpoints.
Pre-split, the filer StatefulSet scheduled concurrently with the CNPG
bootstrap (~55–70s of unavailable postgres), each connect() failed
EPERM, kubelet exponential restart backoff pushed past the e2e bats
'kubectl wait hr/seaweedfs-system --timeout=2m' window, and the
'Configure Tenant and wait for applications' test failed.
Migration 39 adopts existing Cluster/seaweedfs-db resources into the new
release on upgrade by rewriting meta.helm.sh/release-name and stamping
helm.sh/resource-policy: keep.
Requires Flux 2.8.x (helm-controller v1.5.0+). Cozystack currently pins
Flux to 2.7.x; a separate PR bumps the pin.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Re-render internal/fluxinstall/manifests/fluxcd.yaml via
`make update` in packages/core/flux-aio (timoni bundle build against
oci://ghcr.io/stefanprodan/modules/flux-aio at the latest tag) so
the cozystack-operator binary embeds Flux 2.8.0 instead of 2.7.3.
Auto-syncs the helm-controller image into fluxcd-tenants.yaml.
Embedded versions now match the chart bump in the prior commit:
flux: v2.7.3 -> v2.8.0
helm-controller: v1.4.3 -> v1.5.0 (enables healthCheckExprs)
source-controller: v1.7.3 -> v1.8.0
kustomize-controller: v1.7.2 -> v1.8.0
notification-controller:v1.7.4 -> v1.8.0
source-watcher: v2.0.2 -> v2.1.0
Verified by rebuilding cmd/cozystack-operator and running `strings`
on the binary -- the expected v1.5.0 / v1.8.0 / v2.1.0 image strings
are now embedded in place of the v1.4.x / v1.7.x ones.
Structure preserved:
- single AIO Deployment named "flux" in namespace cozy-fluxcd
- all 5 controllers + source-watcher as containers in one pod
- custom tolerations from flux-aio.cue intact
- Namespace pod-security label still "privileged" via the
Makefile yq post-processing step
Net diff is -3664 lines (mostly CRD schema simplification in 2.8).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Bump flux-operator and flux-instance vendored charts from v0.33.0 to
v0.48.0, moving the Flux distribution from 2.7.x to 2.8.x. This brings
helm-controller v1.5.x, which exposes spec.install.healthCheckExprs on
HelmRelease — a prerequisite for adding real readiness gating to
dependency-target packages (postgres, mongodb, kafka, etc.).
The new flux-operator chart defaults web.enabled=true; override it to
false in packages/system/fluxcd-operator/values.yaml so the Flux Status
Web UI stays opt-in. Operators wanting it can set web.enabled=true and
configure web.config / web.configSecretName / web.ingress per the
upstream chart values.
Patches reapplied:
- kubernetesEnvs.diff: matched cleanly (offset shift only)
- networkPolicy.diff: extended to also restrict port 9080 (web UI port);
hunk count updated 20 -> 22
Pre-flight: no references to the v0.39-removed --disable-wait-interruption
flag anywhere in packages/ or internal/.
Verified locally:
- helm template renders both packages without errors
- web.enabled=false (default): no web Deployment/Service/NetworkPolicy
- web.enabled=true: renders web NetworkPolicy on port 9080
- CiliumClusterwideNetworkPolicy lists 8080/8081/9080
- FluxInstance distribution.version: "2.8.x", all 7 components present
- Cozystack kustomize patches (concurrent, requeue, storage-adv-addr,
events-addr) intact
Notable upstream changes between v0.33 and v0.48:
- Flux 2.8.x with HelmRelease inventory format change (auto-migrated)
- Reliability fixes for stuck FluxInstance/ResourceSet on health-check
cancellation (v0.39, v0.46)
- New CLI: flux-operator migrate, diff, patch instance, distro mirror
- ResourceSetInputProvider: GitLab Environments, GitHub App, Gitea/Forgejo
- CVE-2026-23990 (Web UI OIDC impersonation bypass) — N/A while web is off
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This PR adds the changelog for release `v1.3.4`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.3.4.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* OpenSearch is now integrated into the 1.3 PaaS bundle so operator and
dashboard components deploy when PaaS is enabled.
* Storage class input now renders as a dropdown including OpenSearch as
an option.
* **Bug Fixes**
* Fixed Harbor reconciliation regression related to bucket secret
handling and render-path coverage.
* **Documentation**
* v1.3.4 release notes published, expanded docs, summary of related
project changes, contributor list and compare link.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2676?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Adds the missing entry for #2649 (OpenSearch wired into PaaS bundle, plus
StorageClass dropdown override for the legacy 1.3 openapi-ui dashboard),
corrects six website-PR author attributions against gh pr view, picks up
two website docs PRs merged just before the v1.3.4 cut (cozystack/website#536
and #538), updates the contributors list, and refreshes the date to today.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This PR adds the changelog for release `v1.4.0`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.4.0.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Released v1.4.0 changelog describing a redesigned schema-driven
dashboard, persistent tenant storage, unified instance-type presets,
declarative multi-database backups, optional fractional GPU sharing, and
PROXY-protocol support
* Includes upgrade guidance, operator timing knobs, kubelet/autoscaler
behavior changes, security fixes, component version bumps, and full
changelog link
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2675?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
OpenSearch has shipped as a complete package set in the repo for several
releases — `packages/apps/opensearch/`,
`packages/system/opensearch-operator/`,
`packages/system/opensearch-rd/`, and `PackageSource` files under
`packages/core/platform/sources/` — but the PaaS bundle template was
never
updated to include the two `cozystack.opensearch-*` PackageSources.
Result on any cluster with `bundles.paas.enabled=true`:
- `opensearch-operator` is not deployed → no `OpenSearchCluster` CRD on
cluster
- `opensearch-rd` release is not deployed → no
`ApplicationDefinition/opensearch` → the dashboard catalog has no
OpenSearch entry and tenants cannot create
`opensearches.apps.cozystack.io`
This PR adds the missing includes alongside every other DB
operator/application in
`packages/core/platform/templates/bundles/paas.yaml`.
### Release note
```release-note
Fix: OpenSearch is now installed by the PaaS bundle (`bundles.paas.enabled=true`); previously the operator and ApplicationDefinition were defined but never referenced from the bundle, so OpenSearch did not appear in the dashboard catalog.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* OpenSearch operator and application packages are now included in the
PaaS bundle when enabled, providing additional search and analytics
capabilities for supported deployments.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2648)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Reverts the `publishing.exposure=loadBalancer` mode for ingress-nginx
added in #2468.
The mode rendered the ingress-nginx Service as `type: LoadBalancer` and
a `CiliumLoadBalancerIPPool` over `publishing.externalIPs`, as a
migration path off `Service.spec.externalIPs` (deprecated upstream in
Kubernetes v1.36 per KEP-5707). In default Cozystack the implementation
produces two failure modes:
- `publishing.externalIPs` is the operator-supplied list of node public
IPs (addresses the cluster nodes already hold and listen on, see [docs
section "4b. Node Public IP
Setup"](https://cozystack.io/docs/v1.3/install/cozystack/platform/#4b-node-public-ip-setup)),
not a virtual-VIP pool. Treating it as one and DNATing those IPs via
Cilium LB-IPAM collides with the existing ingress-nginx DNAT on the same
IPs (`Service.spec.externalIPs` path), so traffic is forwarded
non-deterministically.
- Cilium in default Cozystack runs with `ipam.mode: kubernetes` and no
L2/BGP announcer, so a pool over addresses outside the node range is
allocated but never advertised — the Service stays unreachable.
The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side `IPPool` baked into the tenant chart. That follow-up belongs
in its own PR.
### Removed
- `publishing.exposure` value (and the inline comment) from
`core/platform/values.yaml`.
- `expose-mode` plumbing line in `core/platform/templates/apps.yaml`.
- `CiliumLoadBalancerIPPool` template at
`apps/tenant/templates/cilium-lb-pool.yaml`.
- `type: LoadBalancer + externalTrafficPolicy: Local` branch in
`extra/ingress/templates/nginx-ingress.yaml`. The Service now renders as
`ClusterIP + spec.externalIPs` when externalIPs are set, falling through
to `type: LoadBalancer` when they are not — pre-#2468 shape.
- Exposure mode section from `extra/ingress/README.md`.
- `exposure_test.yaml` fixtures in both charts.
### Compatibility
Operators who explicitly set `publishing.exposure=loadBalancer` will see
their ingress Service revert to `ClusterIP + spec.externalIPs` on
upgrade. Anyone on the default value (`externalIPs`) is unaffected.
### Release note
```release-note
revert(ingress): remove the publishing.exposure=loadBalancer mode added in #2468. The default externalIPs path is unchanged; operators who explicitly opted into loadBalancer will have their ingress Service revert to ClusterIP + spec.externalIPs on upgrade. Migration off Service.spec.externalIPs (deprecated in KEP-5707) will be reintroduced via a mechanism-agnostic API knob in a follow-up.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Refactor**
* Removed conditional generation of the Cilium load‑balancer IP pool and
simplified service exposure to rely on explicit external IP values.
* Streamlined configuration for service exposure keys.
* **Documentation**
* Removed the exposure-mode guidance from ingress documentation.
* **Tests**
* Deleted the exposure-mode-related test suites covering various
external IP and loadBalancer scenarios.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2603)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Marks the `generate-changelog` job in `.github/workflows/tags.yaml` with
`continue-on-error: true`, and changes the `update-website-docs` gate to
`always() && needs.prepare-release.result == 'success' &&
needs.prepare-release.outputs.skip != 'true'`.
Result: a failure inside `generate-changelog` (AI step timing out, PR
creation step erroring, etc.) no longer blocks `update-website-docs`.
`prepare-release` remains the hard gate — if it fails or signals
`skip=true`, downstream jobs still don't run.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
This pull request contains internal infrastructure changes only and has
no impact on end-users or the product.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2670?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review feedback from gemini-code-assist on
docs/changelogs/v1.4.0.md:326: the in-prose mention of dislogical was
rendered as inline code rather than as a contributor link. Switch to
the [**@user**](https://github.com/user) format used elsewhere in the
changelog.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from gemini-code-assist on
docs/changelogs/v1.4.0.md:102 and coderabbitai on :103: the AMD
SEV-SNP feature gate in upstream KubeVirt source is WorkloadEncryptionSEV
(see pkg/virt-config/featuregate/active.go), not WorkloadEncruptionSEV
as quoted from the v1.7.0 release notes.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Mark the generate-changelog job with continue-on-error and switch the
update-website-docs gate to always() so a failed or skipped changelog
generation no longer blocks the website docs update. prepare-release
remains the hard gate.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix regression of broken auth by token to the cozy-dashboard
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed a regression in JWK cache initialization where custom HTTP
client configurations for cluster authentication were not being properly
applied, ensuring secure cluster communication and service-account token
handling work as intended.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2661)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Adds `docs/changelogs/v1.2.4.md` so the `Update Release Notes` workflow
can publish the existing `v1.2.4` draft release.
`v1.2.4` is a hotfix patch backporting only the duplicate-`template:`
key removal from #2279 — the breaking service-naming changes in that PR
were intentionally omitted as not appropriate for a patch (see #2652 for
the minimal-backport details).
The `Generate Changelog` step on the tag-push workflow failed twice in a
row (yesterday's cron run and today's manual dispatch) due to `402 You
have no quota` on the Copilot API, so this changelog is hand-written
following `docs/changelogs/patch-template.md` and matches the v1.2.3
style.
## Test plan
- [x] file follows `patch-template.md` structure
- [x] author attribution verified against `gh pr view` for #2279 and
#2652
- [ ] CI passes
- [ ] after merge, `Update Release Notes` picks up the file and
publishes draft release `v1.2.4`
Hand-written patch changelog for v1.2.4 (Copilot quota exhausted on the
automatic Generate Changelog step). Single fix: minimal backport of the
MariaDB backup CronJob duplicate-key removal from #2279 via #2652.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The opensearch chart, operator chart, ApplicationDefinition and
PackageSources have existed since the original OpenSearch series, but the
PaaS bundle template never references the opensearch-operator or
opensearch-application PackageSources. On isp-* variants with
bundles.paas.enabled=true, neither the operator nor the ApplicationDefinition
is ever installed, so the dashboard catalog has no OpenSearch entry and
opensearches.apps.cozystack.io resources cannot be created.
Add both includes to packages/core/platform/templates/bundles/paas.yaml,
following the same pattern used for every other DB application.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## Summary
Replaces the old openapi-ui with the new cozystack-ui build from the
cozystack-ui repository.
## Key Changes
### New UI
- **Image**: `999669/cozystack-ui:latest@sha256:ea4e832c2...`
- **Architecture**: `linux/amd64`
- **Repository**: https://github.com/cozystack/cozystack-ui
(feat/ui-overhaul branch)
### Removed Components
- Removed BFF (Backend-For-Frontend) container - new UI communicates
directly with Kubernetes API
- Removed old openapi-ui and openapi-ui-k8s-bff build targets from
Makefile
- Removed old UI Dockerfiles and patch files
### RBAC Updates
- Added read permissions for `cozystack.io/applicationdefinitions` to
fix 403 errors
- New UI requires access to applicationdefinitions for marketplace and
catalog functionality
### Bug Fixes
- Fixed VNC WebSocket connections to use dynamic URLs instead of
hardcoded localhost:8001
- VNC now works properly in-cluster deployments
## Testing
Tested on dev3.infra.aenix.org cluster:
- ✅ Dashboard loads correctly
- ✅ Application definitions accessible
- ✅ VNC connections working
- ✅ Backup pages functional
- ✅ All TypeScript errors resolved
## Migration Notes
The new UI is a complete rewrite with:
- Modern React 19 + TypeScript stack
- Improved backup management pages
- Better type safety and error handling
- Direct Kubernetes API communication (no BFF needed)
… with others
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
align Altinity backup strategy field name for similarity with others
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
implement backup controller for foundationdb backups
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added FoundationDB backup & restore support via a new FoundationDB
backup strategy and controller; controller now handles FoundationDB
backup/restore jobs.
* **Examples**
* Added a complete FoundationDB backup/restore demo with helper scripts,
step-by-step step scripts, run-all, and cleanup tooling.
* **Documentation**
* New admin and tenant guides plus a README explaining the example
workflow and verification steps.
* **Deprecations**
* In-chart FoundationDB backup configuration marked DEPRECATED—migrate
to BackupClass + FoundationDB strategy.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2606)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
implement backup strategy for MariaDB
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added native MariaDB backup/restore strategy using the operator-native
BackupClass/Plan/RestoreJob flow with example manifests for end-to-end
workflows.
* **Documentation**
* Added MariaDB how‑to and examples; marked legacy restic/chart-side
backup parameters as DEPRECATED and documented migration guidance.
* **Tests**
* Added unit tests covering MariaDB backup/restore reconciliation,
templating, storage handling, idempotency, URIs, and deadlines.
* **Chores**
* Extended controller RBAC to allow observing MariaDB operator
resources.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2605)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds infrastructure for building a patched kilo image from upstream
`main` branch source, following the same pattern as the kamaji package.
The patch introduces a new `cross` mesh granularity mode — WireGuard
tunnels are created only between nodes in different locations, while
intra-location traffic stays on the CNI overlay.
Changes:
- `images/kilo/Dockerfile` — multi-stage build: fetches upstream
tarball, applies patches, builds `kg`/`kgctl` binaries with CNI plugins
and iptables-wrapper
- `images/kilo/patches/feat-add-cross-mesh-granularity.diff` — adds
`CrossGranularity` to the kilo binary
- `Makefile` — new `image:` target for building the patched image,
updated `update:` target to pin upstream commit SHA
### Screenshots
N/A — no UI changes.
### Release note
```release-note
feat(kilo): add patched image build with cross mesh granularity support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added "cross" mesh granularity mode for finer topology grouping,
adjusted routing/encryption behavior
* Added e2e tests covering cross-mode scenarios for improved reliability
* **Chores**
* Build automation now tracks latest main commit and derives image
digests from build metadata
* Introduced multi-stage container build and runtime packaging for kilo,
including required networking tooling
* **Configuration**
* Updated default image reference to a new repository and digest
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2604)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Two related quality-of-life additions to the Keycloak system component:
1. **`keycloak` chart now accepts `extraEnv`.** Operators can append
extra
`EnvVar` entries to the Keycloak StatefulSet container without forking
the
chart. Useful when a custom theme needs runtime configuration via
`kcContext.properties`, or when an extension expects environment
variables.
2. **`keycloak-configure` chart now exposes a top-level `userProfile`
value**
that is mirrored verbatim into
`ClusterKeycloakRealm.spec.userProfileConfig`.
This lets operators declare custom user-profile attributes (required
fields, validators, permissions, ...) declaratively, alongside the
existing
`login` and `smtp` blocks.
Both values default to empty, so existing deployments are unaffected.
### Release note
```release-note
feat(keycloak): support extraEnv values and userProfile customization
```
## What this PR does
Closes#1959.
Continuation of @matthieu-robin's instance-type resource presets work,
addressing the migration gaps and resource regressions surfaced during
the review of #1959.
Resource presets now follow a cloud-style `<series>.<size>` naming
convention. Five series cover the full CPU-to-memory ratio range, each
with eight sizes (`nano` through `4xlarge`):
| Series | Ratio | Use case |
| ------ | ----- | -------- |
| `t1` | 1:0.5 | Tiny / burstable, low memory |
| `c1` | 1:1 | Compute-balanced |
| `s1` | 1:2 | Standard — proxies, caches |
| `u1` | 1:4 | Universal — databases, messaging |
| `m1` | 1:8 | Memory — search, analytics |
40 instance-type presets in total. The seven legacy flat names (`nano`,
`micro`, `small`, `medium`, `large`, `xlarge`, `2xlarge`) remain
accepted as deprecated aliases.
### Upgrade path
The combination of three mechanisms makes the rename a no-op for
existing deployments:
1. **`cozy-lib` legacy aliases** — preserve the exact pre-rename CPU and
memory for every legacy name. `helm template ... --set
resourcesPreset=small` continues to render `1 CPU / 512Mi`, identical to
before. No HelmRelease that already lives on the cluster breaks.
2. **Migration 39** — a pre-upgrade hook walks every
`HelmRelease.spec.values` and every app CR under
`apps.cozystack.io/v1alpha1`, rewriting legacy values to their 1:1
instance-type equivalents (`nano → t1.nano`, `medium → c1.small`, etc).
Idempotent, best-effort, dry-runnable via `MIGRATION_DRY_RUN=1`,
distinguishes "no items" from "kubectl errored" so a broken cluster
cannot silently stamp the version.
3. **`cozystack-api` deprecation warning** — every `Create` and `Update`
on an app CR with a legacy value emits a `klog` warning naming the JSON
path and the 1:1 replacement, so operators discover any remaining legacy
data after the platform upgrade.
The legacy-to-instance-type mapping is 1:1 by CPU and memory:
| Legacy | CPU | Memory | Instance-type |
| -------- | ---- | ------ | ------------- |
| `nano` | 250m | 128Mi | `t1.nano` |
| `micro` | 500m | 256Mi | `t1.micro` |
| `small` | 1 | 512Mi | `t1.small` |
| `medium` | 1 | 1Gi | `c1.small` |
| `large` | 2 | 2Gi | `c1.medium` |
| `xlarge` | 4 | 4Gi | `c1.large` |
| `2xlarge`| 8 | 8Gi | `c1.xlarge` |
Mirrored in four places (cozy-lib template, Go package, migration jq,
docs) with cross-reference comments so drift is visible.
### Why not just #1959
The implementation in #1959 had three blockers that motivated this
rewrite:
- **`cozy-lib` lost the legacy names entirely**, so any existing
HelmRelease with `resourcesPreset: small` failed render with `Preset key
'small' invalid`. Aliases close this hole.
- **Migration only touched HelmReleases**, leaving app CRs unmigrated;
the next `kubectl edit` would fail enum validation. The new migration
walks both.
- **Defaults silently inflated memory**, e.g. Redis `nano → m1.nano`
jumped from 128Mi to 2Gi (×16). With two replicas plus sentinel the e2e
VM ran out of memory, producing the Redis 90s timeout that blocked the
original PR. New defaults preserve resources 1:1.
### Deprecation timeline
Legacy aliases are scheduled for removal in a future cozystack release.
The combination of migration + operator warnings is expected to converge
production clusters before then. Removal will land as a separate PR with
its own migration cleanup pass.
### Test plan
- `make unit-tests` — Go tests for the preset walker + warning helper,
`helm unittest` legacy↔instance-type parity suites for
`postgres`/`clickhouse`/`opensearch`, RD schema invariance check via
`hack/check-rd-presets.sh`.
- `make generate` is idempotent.
- `helm template ./packages/apps/redis` with the new default renders
250m / 128Mi (matches pre-rename Redis exactly — closes the e2e
blocker).
- `helm template ./packages/apps/postgres --set resourcesPreset=small`
renders 1 / 512Mi (legacy alias works).
- Migration dry-run is exercised via `MIGRATION_DRY_RUN=1`.
### Documentation
See
[`docs/operations/resource-presets.md`](docs/operations/resource-presets.md)
for the full size matrix and migration table.
### Release note
```release-note
feat(cozy-lib): introduce `<series>.<size>` instance-type resource presets (t1/c1/s1/u1/m1 × 8 sizes). Legacy flat names (nano…2xlarge) remain accepted as deprecated 1:1 aliases; migration 39 rewrites stored values to the new form. CPU and memory are preserved exactly across the rename.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded resource preset options to instance-style names (e.g., t1.*,
c1.*, m1.*); legacy flat names remain supported as deprecated aliases.
* Deprecation warnings now surface when legacy presets are detected.
* **Documentation**
* Added a comprehensive resource-presets guide and updated chart/app
docs to the new naming convention.
* **Tests**
* Added compatibility tests ensuring legacy ↔ instance-type presets
render identical resources.
* **Chores**
* Migration and CI checks to detect/upgrade legacy preset values.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2607)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace deployment-specific examples in the new extraEnv and
userProfile docs with generic placeholders so the chart comments do
not lean on any particular downstream use case.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Myasnikov Daniil
