This PR adds the changelog for release `v1.3.0`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.3.0.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Published Cozystack v1.3.0 release notes featuring LINSTOR scheduler
extender for storage-aware pod placement, managed LINSTOR GUI web
console, VM Default Images catalog, expanded observability with Events
dashboard and S3 metering, cross-namespace VMInstance backup/restore,
and various platform enhancements and bug fixes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
HAMi-core relies on _dl_sym (private glibc symbol removed in 2.34)
for CUDA interception. This breaks compute isolation on modern
container images using Ubuntu 22.04+ and makes Alpine/musl completely
incompatible.
Include upstream issue references and a compatibility matrix so users
can make informed decisions about base image selection.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add HAMi HelmRelease to the kubernetes app as a toggleable addon.
When enabled, GPU Operator's built-in device plugin is automatically
disabled via a values merge to avoid conflicts with HAMi's own
device plugin.
The HelmRelease fails fast if GPU Operator is not enabled, since HAMi
depends on it for driver management and container toolkit.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add HAMi v2.8.1 (CNCF Sandbox) as a new system package for fractional
GPU sharing in tenant Kubernetes clusters. The chart enables workloads
to request specific amounts of GPU memory and compute cores instead of
claiming entire GPUs.
Vendored upstream chart includes device plugin, scheduler extender,
mutating webhook, and DRA subchart. Wrapper values configure nvidia
RuntimeClass and clear hardcoded node config from upstream defaults.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Harbor is a PaaS service (`category: PaaS`), not a tenant module. It is
not deployed automatically into tenant namespaces — there is no
corresponding manifest in `packages/apps/tenant/templates/`, unlike
actual tenant modules (monitoring, ingress, etcd, info, seaweedfs).
Two flags were incorrectly set on its `ApplicationDefinition`:
- `spec.dashboard.module: true` — caused Harbor to appear in the sidebar
"Modules" section and be excluded from its proper PaaS category.
- `spec.release.labels."internal.cozystack.io/tenantmodule": "true"` —
caused controllers (`pkg/registry/core/tenantmodule`, dashboard) to
treat the Harbor HelmRelease as a tenant module.
Both flags are removed so Harbor is listed correctly under PaaS and
handled as an ordinary managed application.
### Release note
```release-note
fix(harbor): remove incorrect tenant module flags from Harbor ApplicationDefinition so it appears under the PaaS category in the dashboard and is no longer treated as a tenant module
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Simplified internal application configuration settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Bumps `packages/system/kubeovn` to `cozystack/kubeovn-chart` tag
`v1.15.10-cozy.1`, which:
1. Updates upstream kube-ovn from v1.15.3 to v1.15.10 (latest patch in
the v1.15 series).
2. Carries a patch over `pkg/controller/pod.go` that preserves a VM
LSP's port-group memberships when kubernetes GCs a completed
virt-launcher pod while another virt-launcher pod of the same VM is
still running.
Without the patch, the destination pod of a successful live migration
loses its security groups, network policies and node-scoped routing
after kubernetes cleans up the source pod, and only recovers after a
`kube-ovn-controller` restart. The buggy code path is identical between
v1.15.3, v1.15.10, release-1.15 HEAD and master HEAD — confirmed by diff
and by reproduction on a cozystack cluster.
- Upstream issue: kubeovn/kube-ovn#6665
- Upstream fix PR: kubeovn/kube-ovn#6666
- Chart carry: cozystack/kubeovn-chart#4
### Release note
```release-note
fix(kube-ovn): bump kube-ovn to v1.15.10 and carry upstream fix that keeps VM LSP port-group memberships when kubernetes GCs a completed virt-launcher pod (post-migration regression)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VpcEgressGateway CRD extended with optional resources
(claims/limits/requests) and bandwidth (ingress/egress) fields.
* Added a pre-upgrade hook to run compatibility steps before upgrades.
* CNI DaemonSet now optionally respects MTU when configured.
* **Bug Fixes**
* Updated Kube-OVN and NAT gateway container images to the latest
maintenance release.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes an RBAC failure in `kube-ovn-plunger` that prevented it from
reconciling the `ovn-central` Deployment:
```
deployments.apps is forbidden: User "system:serviceaccount:cozy-kubeovn:kube-ovn-plunger"
cannot list resource "deployments" in API group "apps" at the cluster scope
```
Two causes addressed:
- The controller-runtime cache issued cluster-wide list/watch for
Deployments and Pods by default. The manager cache is now scoped to the
kube-ovn namespace, so requests go to the namespaced endpoints covered
by the Role.
- The Role's deployments rule used `resourceNames`, which does not apply
to list/watch in Kubernetes RBAC. The restriction is removed; the Role
is still namespace-scoped via RoleBinding to the `cozy-kubeovn`
namespace.
`--kube-ovn-namespace` and `--ovn-central-name` are now passed to the
binary from chart values so the previously unused value is wired up.
### Release note
```release-note
fix(kube-ovn): fix kubeovn-plunger RBAC forbidden error on ovn-central Deployment reconcile
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable parameters for KubeOVN namespace and OVN central
name in deployment configuration.
* **Improvements**
* Enhanced cache configuration with namespace-scoped resource management
capabilities.
* Updated RBAC permissions to remove resource-specific constraints,
enabling broader deployment access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Options field was added to RestoreJobSpec without re-running
'make generate', leaving zz_generated.deepcopy.go out of sync.
Regenerated to include the missing DeepCopyInto handling for the
runtime.RawExtension pointer.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Run root 'make generate' as a pre-commit hook and as a dedicated
CI workflow so missed codegen updates (CRDs, deepcopy, clients, RBAC)
are caught instead of merging stale generated files.
Pre-commit hook is scoped to files that actually affect codegen
(api/, pkg/apis/, hack/update-codegen.sh, hack/boilerplate.go.txt)
so unrelated commits are not slowed down. CI job sets up Go from
go.mod, runs make generate, and fails on drift with a pointer to
the local fix.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Harbor is a PaaS service, not a tenant module. It is not deployed
automatically into tenant namespaces (no manifest in
packages/apps/tenant/templates/). Remove the misplaced
`dashboard.module: true` flag and
`internal.cozystack.io/tenantmodule: "true"` release label so Harbor
appears under the PaaS category in the dashboard and is not treated as
a tenant module by the controllers.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Pulls cozystack/kubeovn-chart v1.15.10-cozy.1, which bumps upstream
kube-ovn from v1.15.3 to v1.15.10 and carries a patch over
pkg/controller/pod.go that preserves a VM LSP's port-group memberships
when kubernetes GCs a completed virt-launcher pod while another
virt-launcher pod of the same VM is still running.
Without the patch, the destination pod of a successful live migration
loses its security groups, network policies and node-scoped routing
after kubernetes cleans up the migration source pod, and only recovers
after a kube-ovn-controller restart.
Upstream issue: kubeovn/kube-ovn#6665
Upstream fix PR: kubeovn/kube-ovn#6666
Chart carry: cozystack/kubeovn-chart#4
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
- Introduce list, details, create, and sidebar views for `RestoreJob` in
the dashboard controller.
- Auto-generate stable component IDs for sidebar entries and detail
sub-blocks so upserts remain consistent across reconciles.
- Render a single "Same as backup" fallback for an omitted
`spec.targetApplicationRef` on the details view instead of concatenating
three missing-path fallbacks.
- Mark non-CRD-backed `stock-project-factory-*-details` sidebars
(`kube-*`, `plan`, `backupjob`, `backup`, `restorejob`) as static so
they pick up consistent managed-by labels.
### Release note
```release-note
[dashboard] Add RestoreJob views (list, details, create, sidebar link) to the dashboard controller.
```
## Test plan
- [x] Deploy controller to dev stand and open the restore pages in the
dashboard
- [x] Create a RestoreJob through the UI against an existing Backup;
reconcile succeeds and the details view renders
- [ ] CI green
## Previous PR
https://github.com/cozystack/cozystack/pull/2387
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced comprehensive RestoreJobs dashboard interface enabling
users to view and manage backup restoration operations.
* New RestoreJobs section added to the Backups menu with dedicated
details page and navigation.
* Dashboard displays restore job status, phase, backup references,
target applications, and operation timestamps for easy monitoring.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The kubeovn-plunger controller-runtime cache attempted cluster-wide
list/watch on Deployments and Pods, which the namespace-scoped Role
cannot satisfy. Additionally, the deployments rule relied on
resourceNames, which does not restrict list/watch verbs and left the
permission effectively unusable.
Scope the manager cache to the kube-ovn namespace so list/watch hit
the namespaced API, and drop resourceNames from the deployments rule.
Wire --kube-ovn-namespace and --ovn-central-name through the
Deployment so values are actually consumed.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
HTTP-01 ACME issuance in Cozystack has been relying on deprecated
cert-manager fields that happen to still work, but point at a
non-existent ingress class on default installs.
**The bug:**
- `ClusterIssuer`s (`letsencrypt-prod`, `letsencrypt-stage`) hardcode
`http01.ingress.class: nginx`, but Cozystack's nginx ingress classes are
named after tenants (`tenant-root`, `tenant-*`) — there is no class
called `nginx`.
- Every ingress that needs a cert overrides this with the deprecated
annotation `acme.cert-manager.io/http01-ingress-class: <class>`.
cert-manager still honors it, which is why issuance works at all.
- Mixing the modern `ingressClassName` field with the legacy
`http01-ingress-class` annotation produces `fields ingressClassName and
class cannot be set at the same time` (cert-manager#6651), so the
migration has to be atomic.
**The fix:**
- Parameterize the `ClusterIssuer` HTTP-01 solver from
`_cluster.expose-ingress` (default `tenant-root`) using the modern
`ingressClassName` field.
- Rename the override annotation on all ingresses to the modern
`acme.cert-manager.io/http01-ingress-ingressclassname`.
- No new features, no behavior change for operators who already set
`_cluster.expose-ingress` correctly — just removes the dependency on the
deprecated API and stops pointing at a phantom class.
Touched ingresses (9 templates): keycloak, dashboard, linstor-gui,
bucket, grafana, alerta, bootbox/matchbox, seaweedfs, harbor. Plus
`seaweedfs` values.yaml default annotation and the
`cert-manager-issuers` ClusterIssuer template.
Verified renders for `http01` + `tenant-root`, `http01` + custom class,
and `dns01` (correctly omits the http01 branch).
### Release note
```release-note
fix(platform): ACME HTTP-01 challenges now use the modern `ingressClassName` API and target the tenant ingress class from `_cluster.expose-ingress` (default `tenant-root`) instead of a hardcoded `nginx` class that did not exist on most installs. Ingress templates switched from the deprecated `acme.cert-manager.io/http01-ingress-class` annotation to `acme.cert-manager.io/http01-ingress-ingressclassname`.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated cert-manager ACME HTTP-01 solver annotations across ingress
configurations to use the newer `ingressclassname` format for improved
compatibility with current cert-manager versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates the `update-website-docs` job in `.github/workflows/tags.yaml`
to match the new docs-versioning contract introduced in
cozystack/website#495.
The website repo replaces the old "pre-create `vX.Y/` draft directory"
scheme with a permanent `content/en/docs/next/` trunk. Released version
directories are no longer implicitly pre-created — the upstream release
workflow owns the explicit decision of when to promote `next/` →
`vX.Y/`.
### Changes
- **New step `Determine if this release promotes next/`** — parses the
tag and only sets `promote=true` for final `vX.Y.Z` tags where
`content/en/docs/vX.Y/` does not already exist. Prereleases
(`vX.Y.Z-rc.N`, etc.) and patch releases skip promotion.
- **New step `Promote next/ to released version`** — gated on
`promote=true`; runs `make release-next RELEASE_TAG=<tag>`.
- **Step order**: `release-next` runs **before** `update-all`. This way
`update-all` routes into the freshly-created `vX.Y/` directory (per the
website Makefile's routing logic), and `next/` stays untouched as the
trunk for the following release.
- **`git add content hugo.yaml`** — `release-next` registers the new
version in `hugo.yaml` via `register_version.sh`, so the commit step
must stage it.
- Converted the existing `update-all` step to use `env:` vars for
consistency with the new step and workflow-injection hardening.
### Behaviour by tag
| Tag | `release-next`? | `update-all` targets |
|-----|-----------------|----------------------|
| `v1.3.0` (v1.3 does not exist) | yes — promotes next/ → v1.3/ | v1.3/
|
| `v1.3.1` (v1.3 exists) | no — skipped | v1.3/ |
| `v1.3.0-rc.1` | no — skipped (prerelease) | next/ |
| `v2.0.0` (v2.0 does not exist) | yes — promotes next/ → v2.0/ | v2.0/
|
### Release note
```release-note
Release tagging now promotes the website's `next/` docs trunk to `vX.Y/` when cutting a new minor or major release (requires cozystack/website#495 to be merged first).
```
### Dependency
Must not merge until cozystack/website#495 is merged on `main` of the
website repo, otherwise `make release-next` won't exist there and
new-minor release tags will fail.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated release automation to apply conditional logic for version
promotion based on tag patterns and documentation availability. Modified
documentation staging behavior in release workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Replace rename-only wording on spec.targetApplicationRef form labels so
users discover that the field also supports restoring into a different
application, not just renaming.
- Drop VMInstance-specific keys from the spec.options form label; the
restore options are driver-specific, so hardcoding one driver's keys is
misleading once other drivers land.
- Render a single "Same as backup" fallback for an omitted
spec.targetApplicationRef on the details page instead of the noisy
"-.-/-" produced by concatenating three missing path fallbacks.
- De-duplicate the time-block / time-icon / time-value component IDs
within the restorejob-details factory by prefixing them with
created-, started-at-, and completed-at-.
- Mark non-CRD-backed stock-project-factory-*-details sidebars (kube-*,
plan, backupjob, backup, restorejob) as static in
upsertMultipleSidebars so they pick up consistent managed-by labels
instead of being left unmanaged.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
ClusterIssuer solver referenced IngressClass "nginx" which does not
exist on cozystack clusters — real classes are named after tenant
namespaces (e.g. tenant-root). Cert issuance only worked because every
requesting Ingress overrode the ClusterIssuer via the legacy
acme.cert-manager.io/http01-ingress-class annotation.
Switch both sides to the modern cert-manager API (available since
cert-manager 1.12; cozystack ships 1.19.3):
- ClusterIssuer: http01.ingress.ingressClassName, value parameterized
from _cluster.expose-ingress (default "tenant-root")
- Ingress annotation: http01-ingress-ingressclassname
These must migrate together — mixing ingressClassName (ClusterIssuer)
with the old http01-ingress-class annotation triggers cert-manager's
"fields ingressClassName and class cannot be set at the same time"
validation and breaks issuance.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Update oldName and serverName field descriptions based on code review
feedback to avoid confusion about their actual roles:
- oldName: Remove misleading "(matches serverName in backup.info)"
text. This field represents the Kubernetes cluster resource name,
not the Barman server name.
- serverName: Provide clearer explanation that it's the S3 path prefix
(barmanObjectStore.serverName) used by the original cluster, and should
only be set when it differs from the Kubernetes resource name.
Updated in:
- values.yaml (source of truth for field documentation)
- types.go (Go API type comments)
- values.schema.json (JSON schema for validation)
- postgres.yaml (CRD with embedded OpenAPI schema)
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add serverName field to bootstrap configuration to explicitly specify
Barman server name from backup.info. This fixes "no target backup found"
errors when server_name in backup.info differs from Kubernetes cluster name.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Upgrades the vendored `victoria-metrics-operator` Helm chart from
`0.59.1` to `0.61.0` (operator appVersion `v0.68.1` → `v0.68.4`) by
re-running `make update` in
`packages/system/victoria-metrics-operator/`.
Picks up upstream bugfixes in the `v0.68.x` operator line:
- Correct `VMPodScrape` port for `VMAgent` / `VLAgent` (previously
misrouted, causing scrape failures)
- `StatefulSet` pod *deletion* (not eviction) when `maxUnavailable=100%`
— matches the minimum-downtime upgrade strategy
- `VMDistributed` PVC no longer owned by both the `StatefulSet` and the
top-level object
- Finalizer cleanup on core Kubernetes resources and correct finalizer
targeting for `VLAgent`/`VLogs`/`VMAgent`/`VMSingle`
- Ingest-only `VMSingle`/`VMAgent` no longer mount scrape config secrets
(avoids RBAC errors)
- `StatefulSet` recreation when immutable fields change
- Operator waits for PVC resize completion before continuing reconcile
Chart `0.60.0` is skipped deliberately: it switched the operator
Deployment's selector to `app.kubernetes.io/component`, which required
Deployment recreation on upgrade. `0.61.0` reverts that change, so
jumping straight from `0.59.1` to `0.61.0` avoids the disruption.
Local patch `patches/disable-ca-key-rotation.patch` still applies
cleanly against the new `templates/webhook.yaml`.
### Release note
```release-note
[monitoring] Upgrade victoria-metrics-operator chart to 0.61.0 (operator v0.68.4) for upstream STS/PVC bugfixes, correct VMPodScrape port, and finalizer cleanup.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for configuring Deployment rolling update strategy
* Added Deployment unhealthyPodEvictionPolicy for Pod Disruption Budgets
* Added webhook annotations configuration
* Extended CRD with statefulRollingUpdateStrategyBehavior and
vmauth.enabled settings
* **Bug Fixes**
* Reverted Deployment matchLabels change from release 0.60.0
* **Documentation**
* Added changelog links to documentation
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds @myasnikovdaniil to the default owners in `.github/CODEOWNERS` so
PRs automatically request review from me alongside the existing
maintainers.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated internal repository code ownership configuration to extend
responsibility coverage.
---
**Note:** This release contains only internal administrative updates
with no end-user-facing changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
cozytest.sh does not implement the bats 'run <cmd>' helper and '$status'
variable, so the 'Create Etcd with empty backup block (disabled by
default)' test died with 'run: not found' on every attempt. Replace the
bats idiom with plain shell: use '!' prefix and redirect stderr to
assert the resources absent-when-backup-disabled invariant directly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Increases the LINSTOR satellite startup probe `failureThreshold` from
the Kubernetes default of 3 (30 seconds) to 30 (300 seconds / 5
minutes).
The default startup probe is too aggressive for nodes with slow storage
initialization — satellites get killed and restarted before they have a
chance to come up. This adds an explicit `startupProbe` override in the
`LinstorSatelliteConfiguration` pod template, keeping the same
`tcpSocket` check on the `linstor` port but giving satellites
significantly more time to start.
### Release note
```release-note
fix(linstor): increase satellite startup probe failure threshold from 3 to 30 (5 minutes to start)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a startup health check for the LINSTOR satellite container to
improve pod reliability and ensure the service fully initializes before
accepting traffic.
* This change only introduces container startup probing behavior and
does not alter other runtime behavior or public interfaces.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The website repo is switching to a permanent `content/en/docs/next/`
trunk (cozystack/website#495). Released version directories are no
longer pre-created — the upstream release workflow is now responsible
for explicitly promoting `next/` → `vX.Y/` on new minor/major releases
via `make release-next`.
Update the `update-website-docs` job to match that contract:
- New `Determine if this release promotes next/` step parses the tag
and only enables promotion for final `vX.Y.Z` tags where
`content/en/docs/vX.Y/` does not already exist. Prereleases and
patch releases keep the old behaviour (target routing is handled
by the website Makefile on its own).
- New `Promote next/ to released version` step runs
`make release-next RELEASE_TAG=...` before `make update-all`, so
the subsequent `update-all` routes into the freshly-created
`vX.Y/` directory and `next/` stays untouched as the trunk for
the following release.
- `git add content hugo.yaml` to pick up the version registration
that `release-next` writes via `register_version.sh`.
- Convert the existing `update-all` step to use `env:` vars,
matching the new step and addressing the workflow-injection
hardening guidance.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Run 'make generate' under packages/extra/etcd to refresh the JSON schema
and the corresponding ResourceDefinition YAML after adding backup and
resources fields. Matches cozyvalues-gen v1.3.0 output used by the
pre-commit CI check.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On fast ephemeral runners a freshly booted Talos node has not yet
converged NTP when the previous 10s outer timeout fired, so the
embedded 'until talosctl bootstrap; do sleep 1; done' retry loop only
got one 5-6s attempt before the loop was killed. Every RPC returned
'FailedPrecondition: time is not in sync yet' and E2E failed in the
Prepare environment step.
Bump the outer timeout to 60s and the inter-attempt sleep to 2s. This
aligns with the surrounding bootstrap gates (etcd members check at
180s, drain RPC errors at 60s, node registration at 60s) and gives
NTP plenty of time to converge on ephemeral runners.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
v0.4.3 introduced new CRDs (EtcdBackup, EtcdBackupSchedule). Flux defaults to Skip on upgrade, so on existing clusters those CRDs never land without a manual kubectl apply.
Setting upgradeCRDs: CreateReplace on the etcd-operator component tells PackageReconciler to wire hr.Spec.Upgrade.Crds accordingly. etcd-operator's CRDs evolve additively between versions, so CreateReplace is safe here.
Depends on the PackageSource.Spec.Variants[].Components[].Install.UpgradeCRDs field — must ship with a cozystack-operator release that recognizes it. Without that, this field is ignored and the HelmRelease still defaults to Skip.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Fourth review round fixes.
Simplify backup validation by dropping placeholder-regex guards and switching default values of s3AccessKey, s3SecretKey, destinationPath, and endpointURL from angle-bracketed hints to empty strings. A single 'required' check on each field is now the whole validation for presence; the regex guard was brittle (silently stopped matching if the hint ever changed) and dropping it makes the values.yaml defaults match the behavior contract the README describes. s3:// prefix and '//' rejection still live in etcd-backup-schedule.yaml as template-time fail branches.
Add helm-unittest suites in packages/extra/etcd/tests/ covering both templates end-to-end: rendered output assertions for the happy path (bucket/key split, region passthrough, conditional rendering of optional fields) and failedTemplate assertions for every fail branch. A 'test' target is wired into the package Makefile so hack/helm-unit-tests.sh picks it up on 'make unit-tests'.
Add type: Opaque to backup-secret.yaml for consistency with the mongodb template.
E2E: wait for HelmRelease, Secret, and EtcdBackupSchedule deletion in setup() so the empty-backup case does not flake against lingering state from the previous test. Replace the fictitious minio-e2e hostname with no-such-endpoint.invalid and explicitly document that the e2e verifies schedule/CronJob materialization, not backup execution.
Restore via EtcdCluster.spec.bootstrap and the one-shot EtcdBackup CRD are still not exposed through this chart; both are explicit follow-ups called out in the README.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Use name='etcd' in all three e2e cases. The chart pins the Helm
release name to 'etcd' via check-release-name.yaml, so any other
name causes template rendering to fail before the HelmRelease
can reconcile. This is why the tenant chart hardcodes HR name
'etcd' in packages/apps/tenant/templates/etcd.yaml.
- Consolidate all backup validation guards into
etcd-backup-schedule.yaml so Helm's alphabetical render order no
longer determines which failure a user sees first. Previously, a
bad destinationPath would be masked by the earlier placeholder
check in backup-secret.yaml.
- Use regex match '^<.*>$' for placeholder detection instead of
exact string equality. The previous check silently stopped
firing if anyone updated the default placeholder string in
values.yaml; the regex form catches any angle-bracketed default.
- Reject destinationPath strings containing '//' after the s3://
prefix. Previously 's3://bucket//subdir' silently produced a key
beginning with '/', confusing S3 clients and retention policies.
- Document the backup contract in README.md: what enabling it
produces, which fields are required, why placeholders are
rejected, and that restore is a known follow-up gap.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Mark backup.enabled as optional in the values typedef so
generated JSON schema no longer lists it in 'required'.
Previously, 'backup: {}' (relying on the enabled:false default)
failed schema validation at CRD admission.
- Validate backup.destinationPath has s3:// prefix. Without this
check, passing a path without the scheme silently used the
first path segment as the bucket name.
- Add tail newline to values.schema.json to match repo convention
(cozyvalues-gen output lacks it — worth fixing in that project
separately).
- Add bats setup() that clears tenant-test etcd resources before
each test so a failure mid-test does not leak into subsequent
tests. Hardcoded chart-level resource names ('etcd',
'etcd-s3-creds') make two Etcd apps in the same namespace
collide; setup() enforces a single owner per test.
- Add dump_diagnostics() helper that dumps operator logs + the
relevant CRD state on wait-for-ready timeouts, so CI failures
surface actionable context instead of bare exit code 124.
- Add a third e2e case 'Create Etcd with empty backup block' that
asserts backup:{} admits and produces no backup resources — a
regression test for the required:['enabled'] bug fixed here.
Known scope: the upstream EtcdCluster.spec.bootstrap restore field
(also new in v0.4.3) is not exposed through this values.yaml yet.
Restoring a cluster from an existing snapshot is a follow-up PR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Review found three issues:
1. The guard in backup-secret.yaml only triggered on empty strings,
not on the placeholder defaults ('<your-access-key>' etc). A user
enabling backup.enabled=true without overriding the values would
produce a Secret containing the literal placeholder, and the
backup CronJob would fail at runtime with an opaque 403 far from
the source. Add explicit template guards that fail rendering when
any of the four required fields still hold their placeholder
default.
2. The backup e2e test waited on .spec.schedule of the newly-created
EtcdBackupSchedule, which is set at apply time — the wait was
tautological and did not prove the etcd-operator reconciled
anything. Replace it with a wait on the EtcdCluster's Ready
condition plus a poll for the CronJob the operator generates
from the schedule (selected via the
etcd.aenix.io/etcdbackupschedule-name label).
3. The backup e2e test also did not wait for EtcdCluster readiness
before tearing down. Add the same Ready wait as the baseline
test, and verify the Secret contains the real access key value
rather than just existing.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds two bats cases: a baseline Etcd managed-app deployment and a
deployment with backup.enabled=true that verifies the
EtcdBackupSchedule resource and the S3 credentials Secret are
materialized. The backup case uses fake S3 credentials — the
operator will not successfully push backups, but the schedule
resource itself is reconciled, which is what this test validates.
Also hardcodes the EtcdBackupSchedule and backup Secret names to
'etcd' / 'etcd-s3-creds' to match the existing hardcoded
EtcdCluster name in the chart (one etcd per tenant namespace).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Expose the EtcdBackupSchedule CRD (etcd.aenix.io/v1alpha1) through
optional values so operators can declaratively configure periodic
S3 backups of the platform etcd cluster. Kamaji stores tenant
control-plane etcd data here, so losing this cluster means losing
every tenant cluster — a backup schedule is critical.
The backup is disabled by default (backup.enabled: false), keeping
existing installations unchanged. When enabled, a Secret is created
with AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY keys referenced by
the new EtcdBackupSchedule resource. Field naming mirrors the
existing postgres backup pattern (destinationPath, endpointURL,
s3AccessKey, s3SecretKey, schedule) for consistency.
Requires etcd-operator v0.4.3+ (shipped in the previous commit).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Upstream release notes: https://github.com/aenix-io/etcd-operator/releases/tag/v0.4.3
Notable changes:
- Add EtcdBackup CRD for declarative etcd snapshots (#307)
- Add EtcdBackupSchedule CRD for scheduled backups (#307)
- Expose metrics port on client service (#299)
- Replace deprecated gcr.io/kubebuilder/kube-rbac-proxy with
quay.io/brancz/kube-rbac-proxy:v0.18.1 (#308)
- Update Kamaji example manifest
Note: Helm 3 does not install CRDs on upgrade. Existing clusters
require manual application of the new EtcdBackup and
EtcdBackupSchedule CRDs:
kubectl apply --filename packages/system/etcd-operator/charts/etcd-operator/crds/
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bumps the vendored victoria-metrics-operator chart from 0.59.1 to 0.61.0
(operator appVersion v0.68.1 to v0.68.4) via `make update`.
Picks up upstream bugfixes in the v0.68.x line: correct VMPodScrape port
for VMAgent/VLAgent, StatefulSet pod deletion when maxUnavailable=100%,
VMDistributed PVC ownership fix, finalizer cleanup, ingest-only mode not
mounting scrape config secrets, STS recreation on immutable field changes,
and PVC resize completion wait.
Chart 0.60.0 is skipped because it introduced a matchLabels change
requiring Deployment recreation that was reverted in 0.61.0.
Local patch disable-ca-key-rotation.patch reapplies cleanly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* WorkloadMonitor labels with the workloads.cozystack.io/ prefix are now
propagated onto created Workloads; created Workloads always include the
reserved workloads.cozystack.io/monitor label and source-object labels
take precedence on conflicts.
* Helm app charts now add workloads.cozystack.io/resource-preset
metadata to WorkloadMonitor manifests.
* **Tests**
* Added tests covering label extraction, propagation, conflict
resolution, and backward compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Make the 5-minute timeout self-documenting by setting periodSeconds: 10
explicitly rather than relying on the Kubernetes default.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The default Kubernetes startup probe allows only 30 seconds (3 retries
× 10s) for LINSTOR satellites to become ready. This is insufficient on
nodes with slow storage initialization, causing unnecessary pod restarts.
Raise failureThreshold to 30, giving satellites up to 300 seconds (5
minutes) to complete startup.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- Increase memory limit from 500Mi to 512Mi
- Increase memory request from 100Mi to 256Mi
- Add startup probe with 60s timeout (12 attempts × 5s)
- Increase readiness/liveness initialDelaySeconds from 5/15 to 30s
This fixes OOMKilled crashes observed in production where kamaji
controller was being killed due to insufficient memory during startup.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
IvanHunters