Replace deprecated single-node VLogs CR with VLCluster (cluster mode)
for reliability and horizontal scalability.
Changes:
- Replace VLogs (v1beta1) with VLCluster (v1) using vlinsert/vlselect/vlstorage
- Update fluent-bit outputs to vlinsert-generic:9481
- Update Grafana datasource to vlselect:9471
- Update ExternalName service from vlogs-generic to vlinsert-generic
- Add VPA for all VLCluster components
- Update WorkloadMonitors for three-component architecture
- Add migration 35 to preserve old VLogs resources with keep annotation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Fixed off-by-one error where the first required migration was always skipped.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected migration range handling so upgrade steps run for the
intended version window, preventing skipped or duplicated migrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Prevent cozystack-version configmap from deletion
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated deployment resource configuration to improve system
reliability by ensuring critical components are properly retained and
protected during system operations and maintenance activities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds OpenBAO (open-source Vault fork) as a new managed PaaS application
in Cozystack.
**Structure follows existing app patterns (qdrant, nats):**
- System chart with vendored upstream `openbao/openbao` (chart v0.25.3,
appVersion v2.5.0)
- App chart with standalone/HA mode switching based on replicas count
- TLS via cert-manager self-signed certificates per instance
- ApplicationDefinition, PackageSource, PaaS bundle entry
- E2E test with init/unseal workflow
**Key design decisions:**
- `replicas: 1` → standalone mode with file storage; `replicas > 1` → HA
with Raft integrated storage and retry_join with TLS peer verification
- TLS enabled by default — each instance gets a self-signed Certificate
with DNS SANs covering services and pod addresses
- `disable_mlock = true` in HCL config since default security context
drops IPC_LOCK capability
- Injector and CSI provider disabled (cluster-scoped components, not
safe per-tenant)
- No auto-init/unseal — OpenBAO requires manual initialization by design
- E2E test performs full lifecycle: deploy, wait for certificate + API,
init, unseal, verify readiness, cleanup
### Release note
```release-note
[apps] Add OpenBAO as a managed secrets management service with standalone and HA Raft modes, TLS enabled by default
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added OpenBAO managed secrets management service with
high-availability and standalone deployment options
* Integrated monitoring and dashboards for operational visibility
* Enabled configurable external access and web UI
* Added automated snapshot backup capability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add a safety-net migration for users who already passed migrations 28/29
and still have orphaned -rd HelmReleases in cozy-system:
- ferretdb-rd (replaced by mongodb-rd, never had a dedicated migration)
- mysql-rd (migration 28 only handled user HRs)
- virtual-machine-rd (migration 29 only handled user HRs)
These HRs reference ExternalArtifacts that no longer exist after the
application renames (ferretdb→mongodb, mysql→mariadb,
virtual-machine→vm-disk+vm-instance), causing persistent errors.
Bump targetVersion from 33 to 34.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migrations 28 (mysql→mariadb) and 29 (virtual-machine→vm-disk+vm-instance)
only handled user-facing HelmReleases but left the system-level -rd
HelmReleases (mysql-rd, virtual-machine-rd) orphaned in cozy-system.
These HRs reference ExternalArtifacts that no longer exist, causing
persistent reconciliation failures.
Add cleanup steps to delete the orphaned -rd HRs and their Helm secrets.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The protection-webhook is not part of the cozystack platform and should
not be managed by the migration script. Old services are now deleted
directly instead of being batched through the webhook disable/enable cycle.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migration 26 was using apps.cozystack.io/application.kind=Monitoring label
which is added by migration 22 and may not be present on v0.41.1 clusters.
Switch to cozystack.io/ui=true (guaranteed on old HRs) with field-selector
for exact name match.
Also remove redundant bundle enabled flags from migrate-to-version-1.0.sh
since the variant already determines them via its values file.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migration 27 failed with set -e when Piraeus CRDs did not exist on
clusters without linstor. Add existence check before annotating CRDs.
Also add name-pattern fallback for helm secret deletion, consistent
with migration 26.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
grep returns exit code 1 when no lines match. With set -euo pipefail,
this kills the script when all secrets are helm-release secrets or when
no matching resources exist. Wrap grep calls with { ... || true; }.
Also fix reconcile annotation in migration 29 to use RFC3339 timestamp
format instead of Unix epoch, which Flux v2 expects.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migration 26 silently skipped namespace processing when kubectl
queries failed, leaving helm release secrets intact. This caused
helm to diff old vs new chart manifests during upgrade, deleting
VLogs/CNPG resources and their PVCs.
- Remove silent error suppression (2>/dev/null || true) from
namespace discovery and HR suspend commands
- Add fallback secret deletion by name pattern when label selector
does not match
- Add verification that all helm release secrets are deleted
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds Harbor v2.14.2 as a managed tenant-level container registry service
in the PaaS bundle.
**Architecture:**
- Wrapper chart (`apps/harbor`) — HelmRelease, Ingress,
WorkloadMonitors, BucketClaim, dashboard RBAC
- Vendored upstream chart (`system/harbor`) from helm.goharbor.io
v1.18.2
- System chart (`system/harbor`) provisions PostgreSQL via CloudNativePG
and Redis via redis-operator
- ApplicationDefinition (`system/harbor-rd`) for dynamic `Harbor` CRD
registration
- PackageSource and paas.yaml bundle entry for platform integration
**Key design decisions:**
- Database and Redis provisioned via CPNG and redis-operator (not
internal Helm-based instances) for reliable day-2 operations
- Registry image storage uses S3 via COSI BucketClaim/BucketAccess from
namespace SeaweedFS
- Trivy vulnerability scanner cache uses PVC (S3 not supported by
vendored chart)
- Token CA key/cert persisted across upgrades via Secret lookup
- Per-component resource configuration (core, registry, jobservice,
trivy)
- Ingress with TLS via cert-manager, cloudflare issuer type handling,
proxy timeouts for large image pushes
- Auto-generated admin credentials persisted across upgrades
**E2E test:** Creates Harbor instance, verifies HelmRelease readiness,
deployment availability, credentials secret, service port, then cleans
up.
### Release note
```release-note
[harbor] Add managed Harbor container registry as a tenant-level service
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Harbor container registry deployment with integrated Kubernetes
support, including database and cache layers.
* Enabled metrics monitoring via Prometheus integration.
* Configured dashboard management interface for Harbor administration.
* **Tests**
* Added end-to-end testing for Harbor deployment and verification.
* **Chores**
* Integrated Harbor into the platform's application package bundle.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace PVC-based registry storage with S3 via COSI BucketClaim/BucketAccess.
The system chart parses BucketInfo secret and creates a registry-s3 Secret
with REGISTRY_STORAGE_S3_* env vars that override Harbor's ConfigMap values.
- Add bucket-secret.yaml to system chart (BucketInfo parser)
- Remove storageType/size from registry config (S3 is now the only option)
- Use Harbor's existingSecret support for S3 credentials injection
- Add objectstorage-controller to PackageSource dependencies
- Update E2E test with COSI bucket provisioning waits and diagnostics
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
In v0.41.x the tenant-root Namespace and HelmRelease were applied via
kubectl apply with no Helm release tracking. In v1.0 these resources
are managed by the cozystack-basics Helm release. Without proper Helm
ownership annotations the install of cozystack-basics fails because
the resources already exist.
Add migration 31 that annotates and labels both the Namespace and
HelmRelease so Helm can adopt them, matching the pattern established
in migrations 22 and 27.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add postgres-operator and redis-operator to PackageSource dependsOn
to ensure CRDs are available before Harbor system chart deploys.
Make persistentVolumeClaim conditional to avoid empty YAML mapping
when using S3 storage without Trivy.
Increase E2E system HelmRelease timeout from 300s to 600s to account
for CPNG + Redis + Harbor bootstrap time on QEMU.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Move Harbor from packages/extra/ to packages/apps/ as it is a
self-sufficient end-user application, not a singleton tenant module.
Update bundle entry from system to paas accordingly.
Replace registry PVC storage with S3 via COSI BucketClaim/BucketAccess,
provisioned from the namespace's SeaweedFS instance. S3 credentials are
injected into the HelmRelease via valuesFrom with targetPath.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add Harbor v2.14.2 as a tenant-level managed service with per-component
resource configuration, ingress with TLS termination, and internal
PostgreSQL/Redis.
Includes:
- extra/harbor wrapper chart with HelmRelease, WorkloadMonitors, Ingress
- system/harbor with vendored upstream chart (helm.goharbor.io v1.18.2)
- harbor-rd ApplicationDefinition for dynamic CRD registration
- PackageSource and system.yaml bundle entry
- E2E test with Secret and Service verification
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## Summary
Add a new `cilium-kilo` networking variant that combines Cilium as the
CNI with Kilo as the WireGuard mesh overlay. This replaces the
standalone kilo PackageSource with a unified variant under the
networking source.
## Changes
- Add `cilium-kilo` variant to `networking.yaml` PackageSource with
proper component ordering and dependencies
- Add `values-kilo.yaml` for Cilium to disable host firewall when used
with Kilo
- Remove standalone `kilo.yaml` PackageSource (now integrated into
networking source)
- Switch Kilo image to official
`ghcr.io/cozystack/cozystack/kilo:v0.8.2`
- Remove unused `podCIDR`/`serviceCIDR` options and `--service-cidr`
flag from Kilo chart
Add a new networking variant that integrates Kilo with Cilium
pre-configured. Cilium is deployed with host firewall disabled and
enable-ipip-termination enabled, which are required for correct IPIP
encapsulation through Cilium's overlay.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Kilo is now integrated into the cilium-kilo networking variant instead
of being a separate package that users install manually.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add a new "cilium" variant to the kilo PackageSource that deploys kilo
with --compatibility=cilium flag. This enables Cilium-aware IPIP
encapsulation, routing outer packets through Cilium's VxLAN overlay
instead of the host network.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add migration script that converts VPC HelmRelease values from map
format to array format. The script discovers all VirtualPrivateCloud
HelmReleases, reads their values Secrets, and converts subnets using
yq. Idempotent: skips if subnets are already an array or null.
Bumps migration targetVersion from 30 to 31.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Replace `|| echo ""` with `|| true` to avoid newline bugs in variable
assignments. Switch `for x in $(cmd)` loops to `while read` for safer
iteration over kubectl output.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The virtual-machine application is replaced by separate vm-disk and
vm-instance applications. Migration 28 handles the conversion of
existing VirtualMachine HelmReleases.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Disable both cdi-operator and cdi-apiserver during DataVolume creation
to prevent CDI webhook from rejecting PVC adoption. Also delete mutating
webhook configurations alongside validating ones. Fix cloud-init value
serialization to avoid spurious newline. Add volumeMode detection from PV.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Splits virtual-machine HelmReleases into separate vm-disk and vm-instance
components. Handles PV data preservation, kube-ovn IP migration, and
LoadBalancer IP pinning via metallb annotation for external VMs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fixes Helm upgrade failure for `monitoring-agents` caused by unrendered
`{{ .Release.Namespace }}` template expressions in `values.yaml`.
Introduces `global.target` parameter to control the target namespace for
monitoring services (vminsert, vlogs). Default is `cozy-system`,
platform bundle passes `tenant-root`. Uses `tpl` in vmagent template to
render URLs containing template expressions.
### Release note
```release-note
[monitoring] Fix YAML parse error in monitoring-agents vmagent template
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced monitoring agents configuration to use centralized
target-based values, replacing namespace-based identifiers for improved
consistency across multiple environments.
* Updated monitoring component labeling and URL handling to leverage
parametrized configuration approach for greater flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Renames the `mysql` application to `mariadb` across the codebase. The
chart has always deployed MariaDB (via mariadb-operator), but was
incorrectly named "mysql", causing confusion.
- Rename `packages/apps/mysql` → `packages/apps/mariadb` with all
internal references updated
- Rename `packages/system/mysql-rd` → `packages/system/mariadb-rd` with
CRD kind `MySQL` → `MariaDB`
- Update platform source, bundle references, RBAC, e2e tests, and backup
controller tests
- Add platform migration 27 to auto-discover and rename all deployed
MySQL resources to mariadb
- Preserves PVC data via PV claimRef rebinding
- Handles protection-webhook and mariadb-operator webhook during
migration
- Idempotent: safe to re-run
Real MySQL CLI/config tool names (`mysqldump`, `[mysqld]`, `MYSQL_*` env
vars) are intentionally left unchanged.
### Release note
```release-note
[apps] Rename mysql application to mariadb. A platform migration automatically renames all deployed MySQL resources. No user action required.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automated migration capability from MySQL to MariaDB with data
persistence.
* **Tests**
* Added end-to-end tests for MariaDB deployment and validation.
* **Chores**
* Updated platform components and configuration to use MariaDB as the
default database solution.
* Migrated all internal references, documentation, and build
configurations from MySQL to MariaDB.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Migration 23 already removes the `cozystack-resource-definition-crd`
HelmRelease, but existing Helm release secrets
for `*-rd` applications still reference the deleted
`CozystackResourceDefinition` CRD, causing upgrade failures.
This adds a step to delete those stale Helm secrets so that Flux can
recreate the releases cleanly.
### Release note
```release-note
[platform] Clean up stale Helm release secrets for *-rd applications during migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced system cleanup routine to remove deprecated Kubernetes
secrets during version updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add platform migration that auto-discovers all deployed MySQL
HelmReleases and renames their resources to use the mariadb prefix.
The migration handles PVC data preservation via PV claimRef rebinding,
temporarily disables protection-webhook for protected resource deletion,
and scales mariadb-operator once for all instances.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The mysql chart actually deploys MariaDB via mariadb-operator, but was
incorrectly named "mysql". Rename all references to use the correct
"mariadb" name across the codebase.
Changes:
- Rename packages/apps/mysql -> packages/apps/mariadb
- Rename packages/system/mysql-rd -> packages/system/mariadb-rd
- Rename platform source and bundle references
- Update CRD kind from MySQL to MariaDB
- Update RBAC, e2e tests, backup controller tests
- Keep real MySQL CLI/config tool names unchanged (mysqldump, [mysqld], etc.)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Separates Piraeus CRD management from the `piraeus-operator` chart into
the dedicated `piraeus-operator-crds` chart. This prevents
`piraeus-operator` from deleting CRDs during upgrades.
Changes:
- Add `installCRDs` toggle to `piraeus-operator-crds` chart
- Update `piraeus-operator` Makefile to move CRDs template on chart
update
- Add migration 27 to reassign Helm ownership on existing Piraeus CRDs
and clean up stale Helm secrets
### Release note
```release-note
[linstor] Move CRDs installation logic to piraeus-operator-crds chart
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* CRD installation can now be controlled via configuration settings
(enabled by default).
* System version upgraded to 28.
* **Chores**
* Added migration script to handle system upgrade from version 27 to 28,
including metadata updates and configuration management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Remove the FerretDB managed application from Cozystack. This includes
the application Helm chart, resource definition, platform source, PaaS
bundle entry, RBAC clusterrole entry, and e2e test. Historical migration
scripts are left intact for upgrade compatibility.
### Release note
```release-note
[ferretdb] Removed FerretDB managed application
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Removed FerretDB managed database service and associated Helm chart,
documentation, and test components from the platform.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace unrendered Helm template expressions in values.yaml with
global.target parameter. Pass tenant-root from platform bundle,
default to cozy-monitoring. Use tpl in vmagent template to render URLs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
