Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR updates Talos Linux which includes DRBD fixes:
- https://github.com/LINBIT/drbd/releases/tag/drbd-9.2.15
- https://github.com/LINBIT/drbd/releases/tag/drbd-9.2.16
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update Talos Linux v1.11.
```
Increase startupProbe.failureThreshold from 3 to 30 for kube-apiserver
container. The default 30 seconds (3 failures * 10s period) is often
insufficient for apiserver to complete RBAC bootstrap, especially under
load or with slower etcd connections.
This change gives kube-apiserver up to 300 seconds to initialize,
preventing unnecessary CrashLoopBackOff cycles.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
(cherry picked from commit 8261ea4fcf)
Change the default resourcesPreset for kube-apiserver from "medium" to
"large" to prevent OOM kills during normal operation. The "medium"
preset provides insufficient memory for clusters with moderate workloads.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
(cherry picked from commit 6256893040)
Increase startup probe failureThreshold to 300 (25 minutes) to allow
etcd members more time to sync with the cluster after restart or
recovery. This prevents pods from being killed during initial
synchronization when VPA assigns minimal resources.
Also increase liveness probe failureThreshold to 10 to reduce
unnecessary restarts during temporary network issues.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
(cherry picked from commit c58c959df6)
Fixes aggregated API server Watch implementation to properly work with
controller-runtime informers. External controllers watching Tenant
resources via the aggregated API were experiencing issues with cache
sync timeouts and missing reconciliations on startup.
**ResourceVersion handling in List:**
- Compute ResourceVersion from items when the cached client doesn't set
it on the list itself
**Bookmark handling:**
- Pass through bookmark events with converted types for proper informer
sync
**ADDED event filtering (main fix):**
- Simplified the filtering logic that was incorrectly skipping initial
events
- Only skip ADDED events when `startingRV > 0` AND `objRV <= startingRV`
(client already has from List)
- When `startingRV == 0`, always send ADDED events (client wants full
state)
- Removed the complex `initialSyncComplete` tracking that had inverted
logic
When a controller starts watching resources, controller-runtime may call
Watch with `resourceVersion=""`. The server should send all existing
objects as ADDED events. The previous `initialSyncComplete` logic was
inverted and could skip these events, causing objects (like Tenants with
lock annotations) to not be reconciled on controller startup.
```release-note
[apiserver] Fix Watch resourceVersion and bookmark handling for controller-runtime compatibility
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
* **Improvements**
* Enhanced resource watching and synchronization with Kubernetes 1.27+
compatibility, including proper handling of initial events and
bookmarks.
* Optimized event filtering and resource version tracking for
applications, tenant modules, tenant namespaces, and tenant secrets to
reduce unnecessary event noise.
* Improved list metadata consistency by deriving accurate resource
versions when unavailable in responses.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes node-level RWX block validation from linstor-csi as
controller-level check is sufficient. The controller already validates
that all pods attached to RWX block volume belong to the same VM by
extracting vmName from pod owner references (VirtualMachineInstance).
This simplifies the validation logic and fixes VM live migration issues.
### Release note
```release-note
[linstor] Remove node-level RWX block validation to fix VM live migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Enhanced RWX (read-write-many) block validation with VM-aware checks
across node and controller flows, including support for hotplug-disk
pods and stricter prevention of cross-VM block sharing.
* Improved propagation and resolution of VM identity for attachments to
ensure consistent validation.
* **Tests**
* Added comprehensive unit tests covering single/multiple pod scenarios,
VM ownership, hotplug disks, upgrade paths, and legacy volumes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add custom linstor-csi image build to packages/system/linstor:
- Add Dockerfile based on upstream linstor-csi
- Import patch from upstream PR #403 for RWX block volume validation
(prevents misuse of allow-two-primaries in KubeVirt live migration)
- Update Makefile to build both piraeus-server and linstor-csi images
- Configure LinstorCluster CR to use custom linstor-csi image in
CSI controller and node pods
The RWX validation patch ensures that RWX block volumes with
allow-two-primaries are only used by pods belonging to the same
KubeVirt VM during live migration.
Upstream PR: https://github.com/piraeusdatastore/linstor-csi/pull/403
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update piraeus-server patches to address critical production issues:
- Add fix-duplicate-tcp-ports.diff to prevent duplicate TCP ports
after toggle-disk operations (upstream PR #476)
- Update skip-adjust-when-device-inaccessible.diff with comprehensive
fixes for resources stuck in StandAlone after reboot, Unknown state
race condition, and encrypted LUKS resource deletion (upstream PR #477)
```release-note
[linstor] Fix DRBD resources stuck in StandAlone state after reboot and encrypted resource deletion issues
```
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
(cherry picked from commit dc2773ba26)
## What this PR does
Fixes field selector filtering for registry resources (Applications,
TenantModules, TenantSecrets) when using kubectl with field selectors
like `--field-selector=metadata.namespace=tenant-kvaps` or
`metadata.name=test`.
Controller-runtime cache doesn't support field selectors natively, which
caused incorrect filtering behavior. This PR implements manual filtering
for `metadata.name` and `metadata.namespace` field selectors in List()
and Watch() methods.
Changes:
- Created `pkg/registry/fields` package with `ParseFieldSelector`
utility for common field selector parsing
- Refactored field selector logic in application, tenantmodule, and
tenantsecret registries to use the common implementation
- Implemented manual post-processing filtering after label-based queries
- Removed `Raw` field usage and field selectors from
`client.ListOptions`
### Release note
```release-note
[registry] Fix field selector filtering for kubectl queries with metadata.name and metadata.namespace selectors
```
Adds multus as a dependency as other CNIs
```release-note
Adds multus as a dependency as other CNIs
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
* **Chores**
* Updated component initialization ordering to ensure more consistent
and reliable platform startup sequencing.
* Streamlined dependency requirements for several core system components
to improve deployment reliability and reduce startup complexity.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Update kube-ovn from v1.14.11 to v1.14.25.
Changes synced from upstream include:
- Updated chart templates
- New configuration options in values
### Release note
```release-note
[kube-ovn] Update to v1.14.25
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added tolerations support for VpcNatGateway resources
* Added automatic VLAN subinterface creation capability for provider
networks
* Enhanced installation documentation with OCI Registry and Talos Linux
deployment examples
* **Security Improvements**
* Applied runtime-default seccomp profiles across deployments
* Configured service account token mounting behavior for improved pod
security
* **Chores**
* Updated Kube-OVN to v1.14.25
* Updated Helm chart metadata and dependencies
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
After satellite restart with patched linstor-server, some DRBD resources
ended up in **Unknown** state. Investigation revealed a race condition
between `drbdadm adjust` completion and `updateDiscGran()` lsblk check:
1. `drbdadm adjust` completes successfully (21:53:49.791) - brings
devices up
2. `updateDiscGran()` immediately tries to check discard granularity
(21:53:49.799)
3. `/dev/drbd*` device node doesn't exist yet - kernel hasn't created it
4. `lsblk` fails with exit code 32: "not a block device"
5. `StorageException` interrupts DeviceManager cycle (21:53:50.811)
6. DRBD device remains in incomplete state → **Unknown**
### Timeline from logs
```
21:53:49.791 - [DeviceManager] Resource 'pvc-aafbd92a' [DRBD] adjusted ✓
21:53:49.799 - [lsblk_parser] ERROR: /dev/drbd1169 not a block device
21:53:49.804-50.807 - [lsblk_parser] 10+ retry errors (every ~100ms)
21:53:50.811 - [DeviceManager] ERROR: Error executing lsblk
21:53:50.878 - [DeviceManager] End cycle 9 (WITH ERROR!)
```
## Solution
Update `skip-adjust-when-device-inaccessible.diff` patch to add physical
device path existence check before calling `lsblk`:
- Check `Files.exists(devicePath)` in `updateDiscGran()` before lsblk
- If device doesn't exist yet → skip check, set `discGran =
UNINITIALIZED_SIZE`
- Next DeviceManager cycle (few seconds later) → device node available →
lsblk succeeds
This complements the existing patch which checks **child layer** devices
(LUKS/Storage) for deletion scenarios, while this fix addresses the
**DRBD device itself** during adjust operations.
## Testing
Manual `drbdadm up` on affected devices confirmed they were in down
state and brought them back to UpToDate, proving the issue was
incomplete device initialization.
## Related
- Upstream linstor-server PR:
https://github.com/LINBIT/linstor-server/pull/471#issuecomment-3723392917
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced robustness for storage operations by adding device
accessibility validation. Operations now gracefully skip when device
paths are unavailable or invalid, preventing unnecessary failures and
improving system resilience during device access issues.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
By its nature, the kubernetes client provisioned in keycloak is public:
anyone can read the client secret. Therefore the secret is not necessary
and the client should be explicitly provisioned as a public client, not
as a confidential client.
### Release note
```release-note
[keycloak] Change the kubernetes client to be public, without a client
secret.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Add OIDC enablement flag and optional host derivation from tenant
release; allow explicit management kubeconfig endpoint override.
* **Refactor**
* Convert Keycloak client to public mode and remove reliance on a
Kubernetes client secret, simplifying authentication configuration.
* Remove secret-based credential plumbing from deployment templates,
streamlining kubeconfig generation.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Enable DRBD auto-diskful options to automatically convert diskless nodes
to diskful when they remain in Primary state for an extended period.
### How it works
When LINSTOR is integrated with Kubernetes, the platform may schedule
workloads on nodes that don't have local storage replicas. In such
cases, DRBD operates in "diskless" mode, accessing data over the network
from nodes that have actual disk replicas.
The `auto-diskful` feature addresses this by:
1. **Monitoring Primary state duration**: When a diskless node holds a
DRBD resource in Primary state (actively using the volume) for more than
the configured time (30 minutes), LINSTOR automatically creates a local
disk replica on that node.
2. **Automatic cleanup**: With `auto-diskful-allow-cleanup` enabled,
when the resource is no longer in Primary state on that node, LINSTOR
can automatically remove the disk replica that was created, freeing up
storage space.
### Configuration
- `DrbdOptions/auto-diskful: 30` — Convert diskless to diskful after 30
minutes in Primary state
- `DrbdOptions/auto-diskful-allow-cleanup: true` — Allow automatic
removal of auto-created replicas when no longer needed
### Benefits
- Improves I/O performance for long-running workloads by creating local
replicas
- Reduces network traffic for frequently accessed data
- Automatic cleanup prevents storage waste from temporary replicas
### Release note
```release-note
[linstor] Enable auto-diskful to automatically create local replicas on diskless nodes that hold volumes in Primary state for more than 30 minutes, improving I/O performance for long-running workloads.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced an "auto diskful" option to automatically manage diskful
resources with configurable interval and optional cleanup.
* **Chores**
* Added default configuration values to enable and time the auto-diskful
behavior and to control automatic cleanup.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Add linstor-scheduler-extender for optimal pod placement on nodes with
LINSTOR storage.
### How it works
The linstor-scheduler consists of two components:
#### 1. Scheduler Extender
A custom Kubernetes scheduler that works alongside the default
kube-scheduler. When a pod requests LINSTOR-backed storage, the
scheduler extender communicates with the LINSTOR controller to find
nodes that have local replicas of the requested volumes, prioritizing
placement on nodes with existing data to minimize network traffic.
#### 2. Admission Webhook
A MutatingWebhookConfiguration that automatically sets `schedulerName:
linstor-scheduler` on pods that use LINSTOR CSI driver volumes. This
ensures that only pods with LINSTOR PVCs are routed through the custom
scheduler, while all other pods continue using the default scheduler.
The webhook inspects pod creation requests and checks if any of the
pod's PVCs use a StorageClass with the LINSTOR CSI provisioner
(`linstor.csi.linbit.com`). If so, it mutates the pod spec to use the
linstor-scheduler.
### Components
- `linstor-scheduler` package in `packages/system/linstor-scheduler/`
- PackageSource definition in
`packages/core/platform/sources/linstor-scheduler.yaml`
- Integration into `paas-full` and `distro-full` bundles
### Upstream PRs
- https://github.com/piraeusdatastore/helm-charts/pull/67 — fix:
KubeSchedulerConfiguration v1 support for Kubernetes 1.25+
- https://github.com/piraeusdatastore/helm-charts/pull/68 — feat:
admission webhook support with cert-manager
- https://github.com/piraeusdatastore/helm-charts/pull/69 — fix chart
template issues
### Release note
```release-note
[linstor] Add linstor-scheduler for optimal pod placement on nodes with local LINSTOR replicas. Includes admission webhook that automatically routes pods using LINSTOR CSI volumes to the custom scheduler.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added LINSTOR scheduler component to the platform, packaged as a Helm
chart and installable package.
* Includes admission webhook, scheduler extender, autoscaling support,
pod disruption budget, RBAC and service account integration, and Helm
test hooks.
* **Documentation**
* Added chart README and default configuration values for easy
deployment and customization.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Addes traffic locality capabilities to Seaweedfs.
### Release note
```release-note
Added traffic locality capabilities to Seaweedfs
## What this PR does
Cozystack-api and lineage-webhook tolerate all taints, so they are able
to run on masters no matter what.
Needed for unschedulable control-plane setup, quorum nodes, etc.
### Release note
```release-note
Cozystack-api and lineage-webhook tolerate all taints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated pod scheduling tolerations to improve deployment flexibility
and compatibility across diverse cluster configurations.
* Enhanced workload distribution by making pods more resilient to
cluster taints, enabling better resource utilization in various
environments.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Fix bug in toggle-disk retry logic that left orphaned DRBD devices in
kernel.
## Problem
When toggle-disk retry was triggered (e.g., user retries after a failed
operation), the code called `removeLayerData()` to clean up and recreate
the layer stack. However, `removeLayerData()` only removes data from the
controller's database — it does NOT call `drbdadm down` on the
satellite.
This caused DRBD devices to remain in the kernel (visible in `drbdsetup`
but not managed by LINSTOR), occupying ports and blocking subsequent
operations.
## Solution
Changed retry logic to simply repeat the operation with existing layer
data intact. The satellite handles this idempotently without creating
orphaned resources.
## Upstream
- https://github.com/LINBIT/linstor-server/pull/475 (updated)
```release-note
[linstor] Fix orphaned DRBD devices during toggle-disk retry
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added cancel and retry capabilities for disk addition operations
* Added cancel and retry capabilities for disk removal operations
* Improved cleanup handling for diskless resources with orphaned storage
layers
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Build piraeus-server (linstor-server) from source with custom patches:
- **adjust-on-resfile-change.diff** — Use actual device path in res file
during toggle-disk; fix LUKS data offset
- Upstream: [#473](https://github.com/LINBIT/linstor-server/pull/473),
[#472](https://github.com/LINBIT/linstor-server/pull/472)
- **allow-toggle-disk-retry.diff** — Allow retry and cancellation of
failed toggle-disk operations
- Upstream: [#475](https://github.com/LINBIT/linstor-server/pull/475)
- **force-metadata-check-on-disk-add.diff** — Create metadata during
toggle-disk from diskless to diskful
- Upstream: [#474](https://github.com/LINBIT/linstor-server/pull/474)
- **skip-adjust-when-device-inaccessible.diff** — Skip DRBD adjust/res
file regeneration when child layer device is inaccessible
- Upstream: [#471](https://github.com/LINBIT/linstor-server/pull/471)
Also updates plunger-satellite script and values.yaml for the new build.
### Release note
```release-note
[linstor] Build linstor-server with custom patches for improved disk handling
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automatic DRBD stall detection and recovery, improving storage
resync resilience without manual intervention.
* Introduced configurable container image references via Helm values for
streamlined deployment.
* **Bug Fixes**
* Enhanced disk toggle operations with retry and cancellation support
for better error handling.
* Improved metadata creation during disk state transitions.
* Added device accessibility checks to prevent errors when underlying
storage devices are unavailable.
* Fixed LUKS encryption header sizing for consistent deployment across
nodes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds a `k=kubectl` alias and bash completion for kubectl to the
e2e-testing sandbox container to maintainers have an easier time
exec'ing into the CI container when something needs to be debugged.
### Release note
```release-note
[testing] Add k=kubectl alias and enable kubectl completion in the CI
container.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced the e2e sandbox image to enable shell bash-completion and
kubectl command completion.
* Added an alias (k) and completion wiring for kubectl to improve
interactive command use.
* These changes augment the test environment shell during image build to
provide a smoother developer/testing experience.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Integrate fix
- https://github.com/kubevirt/cloud-provider-kubevirt/pull/379
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Fix endpoints for cilium-gateway
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of incomplete endpoint data by introducing fallback
detection mechanisms.
* Enhanced service discovery to gather endpoints from all available
resources when standard detection fails.
* Updated logging to provide better visibility into fallback operations
and current resource status.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Replaces Helm lookup functions with FluxCD valuesFrom mechanism for
passing configuration to HelmReleases. This provides cleaner config
propagation and eliminates the need for force reconcile controllers.
### Changes:
**Platform/Tenant charts:**
- Add Secret `cozystack-values` creation in platform chart (for
tenant-root and system namespaces)
- Add Secret `cozystack-values` creation in tenant chart (for child
namespaces)
**cozystack-api:**
- Add `valuesFrom` references to HelmRelease when creating applications
- Filter keys starting with `_` when returning Application specs
- Validate that user values don't contain `_` prefixed keys
**cozystack-controller:**
- Add validation that HelmRelease contains correct valuesFrom
configuration
- Remove `CozystackConfigReconciler` (no longer needed)
- Remove `TenantHelmReconciler` (no longer needed)
**Helm charts (40+ files):**
- Add helper templates in cozy-lib for `_cluster`/`_namespace` access
- Replace ConfigMap lookups with `.Values._cluster.*`
- Replace Namespace annotation lookups with `.Values._namespace.*`
### Architecture:
```
Secret cozystack-values (in each namespace)
├── _cluster: YAML with data from ConfigMaps (cozystack, cozystack-branding, cozystack-scheduling)
└── _namespace: YAML with namespace service references (etcd, host, ingress, monitoring, seaweedfs)
HelmRelease
└── spec.valuesFrom:
├── Secret/cozystack-values → _namespace → .Values._namespace
└── Secret/cozystack-values → _cluster → .Values._cluster
```
### Release note
```release-note
[platform] Replace Helm lookup functions with FluxCD valuesFrom mechanism for configuration propagation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Helm releases and namespaces now source centralized cluster/namespace
configuration via a new secret (cozystack-values), and many templates
read values from chart-provided _cluster/_namespace entries.
* **Bug Fixes**
* API now rejects application specs containing reserved keys prefixed
with "_" to prevent invalid configurations.
* **Refactor**
* Two background reconciler controllers were removed from startup,
simplifying controller initialization.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes an issue where backport releases incorrectly became marked as
"Latest" despite passing `make_latest: 'false'` to the GitHub API.
**Root cause:** The `getLatestRelease()` API returns the release with
the "Latest" flag, not the highest semver version. Combined with race
conditions during parallel release publishing and GitHub API potentially
ignoring `make_latest: 'false'`, backport releases were incorrectly
marked as latest.
**Solution:**
- Replace `getLatestRelease()` with semver-based max version detection
across all published releases
- After publishing a backport release, explicitly restore the latest
flag on the highest semver release
- Remove unused dead code from `tags.yaml` workflow
### Release note
```release-note
[ci] Fix latest release detection to use semver comparison instead of GitHub's "Latest" flag
```
## What this PR does
Reverts PR #1770 which incorrectly changed status/phase checks from
lowercase to uppercase.
The actual KubeVirt metrics use **lowercase**:
- `kubevirt_vm_info` uses `status="running"` (not `"Running"`)
- `kubevirt_vmi_info` uses `phase="running"` (not `"Running"`)
### Verification
Queried virt-controller metrics directly in the instories cluster:
```
kubevirt_vm_info{...,status="running",status_group="running",...} 1
kubevirt_vmi_info{...,phase="running",...} 1
```
Note: `kubectl get vm` shows `STATUS: Running` with capital R, but this
is display formatting — the actual metric labels use lowercase.
### Release note
```release-note
[kubevirt-operator] Fix VM alert rules to use correct lowercase status values
```
## What this PR does
This PR splits the monolithic `cozystack-resource-definitions` package
into 25 individual resource definition packages (`*-rd`) for better
modularity and independent versioning.
**Changes:**
- Created 25 separate `*-rd` packages (bootbox-rd, bucket-rd,
clickhouse-rd, etcd-rd, ferretdb-rd, foundationdb-rd, http-cache-rd,
info-rd, ingress-rd, kafka-rd, kubernetes-rd, monitoring-rd, mysql-rd,
nats-rd, postgres-rd, rabbitmq-rd, redis-rd, seaweedfs-rd,
tcp-balancer-rd, tenant-rd, virtual-machine-rd, virtualprivatecloud-rd,
vm-disk-rd, vm-instance-rd, vpn-rd)
- Removed `packages/system/cozystack-resource-definitions`
- Updated platform bundles (paas-hosted, paas-full, distro-full) to
reference individual -rd packages
- Updated `hack/update-crd.sh` to use package-specific directories
Each `*-rd` package contains:
- `Chart.yaml` - package metadata
- `values.yaml` - default values
- `Makefile` - build instructions
- `cozyrds/<name>.yaml` - CRD definition
- `templates/cozyrd.yaml` - Helm template
**Benefits:**
- **Modularity**: Each resource definition is now a standalone package
- **Independent versioning**: Resources can be versioned independently
- **Maintainability**: Easier to update individual resources
- **Build efficiency**: Parallel building of resource packages
### Release note
```release-note
[platform] Split cozystack-resource-definitions into 25 separate *-rd packages for better modularity and independent versioning. Each resource definition is now a standalone package.
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **Refactor**
* Split the monolithic resource-definitions into many independent resource-definition packages (e.g., bootbox-rd, bucket-rd, clickhouse-rd, etc.) for modular deployment and finer-grained management
* Updated deployment bundles to reference the new per-resource releases with uniform cozy-system namespace and CRD dependency
* **Chores**
* Added packaging/Helm stubs (Chart.yaml, Makefile, values, templates) for each new resource-definition
* **Bug Fixes**
* Made CRD path resolution dynamic (NAME validated before assignment)
<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Add lb tests for tenant k8s
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Increased readiness and port-forward timeouts to improve stability.
* Added full end-to-end provisioning and validation: automated namespace
and backend deployment, load balancer provisioning, health checks with
retries, reachability validation, and cleanup.
* Provisioning sequence now runs earlier and is duplicated within the
test flow, altering execution order and adding extra validation/cleanup
steps.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Allows maintainers to trigger test rerun by commenting /retest on a PR.
The workflow finds the latest run for the PR and reruns starting from
the "Prepare environment" job, skipping the build step.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Andrei Kvapil