Commit Graph
6870 Commits
Author SHA1 Message Date
Aleksei Sviridkin ff0752d8e9 fix(kubernetes): probe Keycloak CRDs by output in oidc-bootstrap cleanup (#3520)
## What this PR does

Fixes #3516.

- `packages/apps/kubernetes/templates/oidc-rbac-job.yaml`: the
`mode=None` cleanup path of the `oidc-bootstrap` Job guarded its
Keycloak deletes with `if kubectl api-resources
--api-group=v1.edp.epam.com >/dev/null 2>&1`. That command **exits 0 for
an absent API group** (it simply prints an empty list), so the guard
always fell through and the delete failed with `error: the server
doesn't have a resource type "keycloakclient"` — `--ignore-not-found`
covers a missing *object*, not an unknown *type*. Since the Job is a
`post-install,post-upgrade` Helm hook with `backoffLimit: 6`, this
blocked the Helm upgrade of every `kubernetes-<cluster>` release on any
cluster without the EDP Keycloak operator CRDs, i.e. at the chart
default `oidc.mode: None`.
The probe now captures the discovery output once and gates each delete
on its own resource type being present, matching the idiom already used
in `packages/core/platform/images/migrations/migrations/{34,38,39}` and
`packages/system/ouroboros/.../cleanup-hook.yaml`. `api-resources` is
kept (rather than `get crd`) on purpose: it hits discovery, which
`system:discovery` grants to every authenticated principal, so the
namespaced Job Role needs no cluster-scoped CRD read.
- `packages/apps/kubernetes/tests/oidc_test.yaml`: new unit test
asserting the rendered script probes by output and that the exit-code
form does not come back. All four of its assertions fail against the
pre-fix template.

`cmd/check-readiness` and the platform migrations were checked too: they
already test the output, not the exit code.

## Validation

- `helm unittest packages/apps/kubernetes` on the branch rebased onto
current `main`: 317/317 pass. Putting `main`'s template back fails only
the new case, all 4 assertions, so it is a real regression guard.
- `make generate` in `packages/apps/kubernetes`: no diff.
- Extracted the rendered `mode=None` script and ran `sh -n` /
`shellcheck -s sh` — clean (only pre-existing `SC3040` for `pipefail`
and one pre-existing `SC2086`).
- Executed the extracted script against a stub `kubectl` reproducing the
two real behaviors (`api-resources` exits 0 with empty output for an
absent group; `delete <unknown type>` exits 1 despite
`--ignore-not-found`): the pre-fix script exits 1 with `the server
doesn't have a resource type "keycloakclient"`, the fixed script exits 0
with the CRDs absent and still issues both deletes with the CRDs
present.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

The diff touches no `values.schema.json` and no behaviour the docs
describe: the template change restores the documented `oidc.mode: None`
path rather than altering it.

## Release note

```release-note
fix(kubernetes): the oidc-bootstrap hook no longer fails on clusters without the Keycloak operator CRDs, which blocked Helm upgrades of kubernetes-* releases at oidc.mode=None
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* OIDC cleanup now skips deleting Keycloak resources when their resource
types are unavailable, preventing cleanup failures.
* **Tests**
* Added coverage for resource discovery and conditional cleanup
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-29 14:26:00 +03:00
Aleksei Sviridkin abc07a3927 fix(backups): hold a BackupJob while its application is being restored (#4544)
## What this PR does

Nothing on the backup path looks at restores. A Plan creates its
BackupJobs on schedule (`plan_controller.go`), and `BackupJobReconciler`
dispatches them to their driver whatever else is happening to the
application. A backup taken during a restore captures a half-restored
state at best, and for CNPG it does worse.

We hit it on a v1.6.3 cluster. helm-controller applied the restored
Cluster with `bootstrap.newServerName`, and four seconds later a
BackupJob of the same application re-applied the barman plugin with the
source's `serverName`. The restored cluster archived onto the source's
WAL prefix, and its recovery job died with `WAL archive check failed for
server <cluster>: Expected empty archive`. Correcting `serverName` by
hand let the same restore finish, which is what identified the writer.
`currentBarmanServerName` preserves the live value on most paths, but
not in the window where the restored Cluster is being re-rendered. An
on-demand run can be refused by whoever creates it, but a scheduled one
cannot, and there is no `suspend` on a Plan. With an hourly plan and a
CNPG restore allowed thirty minutes, the overlap happens about half the
time.

A BackupJob that has not started yet (`status.startedAt` unset) now
waits while a non-terminal RestoreJob in its namespace writes into the
same application. The target is resolved as the drivers resolve it
(`resolveCNPGRestoreTarget` and the Job-based paths): the referenced
Backup's `applicationRef`, with the target's `kind` and `name` when a
`targetApplicationRef` is set, and the target's API group only when it
names one, so a target without one stays in the Backup's group. A Velero
restore always stays in the Backup's group, since the Velero driver
drops the target's API group and restores the Backup's resources. A
RestoreJob whose Backup is gone is skipped, since the restore controller
fails it on its next pass. `options.targetNamespace` is honoured only
when the Backup was taken by the Velero strategy, the one driver that
reads it: the CNPG and Job-based drivers ignore the option and restore
in place, so their restores hold whatever it says. The held job stays
`Pending` with a `Ready=False` condition, reason `RestoreInProgress`,
naming the restore, and is requeued every 30s. It is not failed: a
scheduled run should still happen once the restore ends. A failed write
of that status is returned, so the manager retries it, and once the
restore ends the condition is removed before the job starts, since it
would otherwise read "waiting" until the driver next writes `Ready`,
which for a healthy run is when the backup completes.

The gate sits after BackupClass resolution and strategy validation, so
third-party strategies are unaffected, and before credentials
projection, so a held job projects nothing.

What happens to a delayed scheduled run: the Plan controller only
decides when to *create* a BackupJob (a slot within `startingDeadline`,
300s, of now). Once created, the job is not subject to that deadline, so
a held run keeps its `<plan>-<unix-minutes>` name and runs once the
restore ends, later than its slot. If the restore outlasts the next
slot, that slot's job is created and held too, and both run after the
restore. Nothing is skipped, and nothing is deduplicated.

Not covered, deliberately:
- A backup already running when a restore starts is not interrupted. Its
driver owns state the gate would strand. The reverse guard, a RestoreJob
waiting for a running BackupJob, would belong in the restore path,
together with a per-application claim that closes the race between the
two checks, and is tracked in #4559.
- A Velero RestoreJob in another namespace that targets this one through
`options.targetNamespace` does not hold this namespace's backups. The
lookup is namespaced.
- The Velero restore path has no deadline, so a stuck VM restore holds
that application's backups until the restore is ended or deleted. That
matches the restore already blocking the application.

Tests: `TestActiveRestoreTargeting` covers the resolution matrix (in
place, to-copy into and out of the application, terminal phases, another
kind or another API group under the same name, a to-copy target
inheriting the Backup's API group, a Velero target naming another API
group, a Velero restore sent to another namespace or naming its own, a
Postgres restore carrying `targetNamespace`, a missing Backup with or
without a target). `TestReconcile_RetriesAHoldItCouldNotRecord` and
`TestReleaseRestoreHold` cover the status writes, and the held-job test
reconciles again after the restore ends to check the condition is gone.
`TestReconcile_HoldsABackupJobWhileItsApplicationIsRestored` is the
behavioural regression: it fails with the gate removed (the job reaches
projection and dispatch). `TestReconcile_DoesNotHoldARunningBackupJob`
checks that a started job is not held.

Backport: a candidate for `release-1.6`, where the reproduction was
seen. `backupjob_controller.go` differs from `main` only in the dispatch
switch and the Go 1.27 modernizers, so the cherry-pick should be clean
or close to it.

### Screenshots

N/A, no UI change.

### Downstream repositories

The diff touches only `internal/backupcontroller/` Go code. It changes
no CRD, API type, chart, values file, hack script or naming, so the
trigger map comes out empty.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(backups): a BackupJob that has not started yet, scheduled or on demand, now waits while a RestoreJob is restoring the same application, instead of running against it. It stays Pending with a Ready=False RestoreInProgress condition and runs once the restore ends. This stops a CNPG backup that has not started yet from re-pointing an in-place restored cluster at the source's WAL archive; a backup already running when a restore starts is not held.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Unstarted backup jobs now wait when an active restore targets the same
application. They remain pending and are retried after a short delay,
then can proceed once the restore finishes.
* Backup jobs that have already started continue without being held by
an active restore. Completed or failed restores do not hold pending
backup jobs.
* Restore matching accounts for the specified target and, for Velero
restores, the target namespace. A backup job’s restore hold is cleared
when no matching active restore remains.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-29 14:24:41 +03:00
Kirill Ilin b314514950 chore(hetzner-robotlb): update robotlb to 0.1.0 (#4557)
## What this PR does

Updates the bundled robotlb to 0.1.0. The upstream project moved from
the `intreecom` GitHub organization to `treetscom`, so the chart and the
image are now pulled from the new location.

Upgrade note from upstream: balancer targets now follow the Service's
`externalTrafficPolicy`. A Service that leaves it unset gets the
Kubernetes default `Cluster`, and its balancer targets every eligible
node instead of only the nodes running its pods. Set
`externalTrafficPolicy: Local` to keep the previous targeting.

### Screenshots

Not applicable, no UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: cozystack/website#715
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The website still links to the old `Intreecom/robotlb` repository in the
Hetzner install guide, the platform-stack guide and the licenses page.
GitHub redirects those links, so nothing is broken;
cozystack/website#715 updates them.

### Release note

```release-note
chore(hetzner-robotlb): update robotlb to 0.1.0 and pull it from the new treetscom organization. Load balancer targets now follow the Service's externalTrafficPolicy; set it to Local to keep targeting only the nodes that run the Service's pods.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Updates**
* Updated the RobotLB installation to use the `treetscom` chart and
container image sources.
* Updated the installation notes to link to the RobotLB project’s
current documentation location.
  * Updated the RobotLB chart’s application version to 0.1.0.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-29 13:51:32 +05:00
Mattia Eleuteri 28e42fbe4d fix(backups): match a Velero restore in its Backup's API group
The Velero driver resolves a restore's target from the Backup's
applicationRef, overriding only the kind and name, and restores the
Backup's resources. A targetApplicationRef.apiGroup never reaches it.
The restore gate honoured that group all the same, so a Velero
RestoreJob naming another group let a BackupJob of the application
actually being restored start against it, and held one of an
application the restore never touches.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-09-29 09:38:36 +02:00
Mattia Eleuteri 06400ac033 fix(kubernetes): probe Keycloak CRDs by output in oidc-bootstrap cleanup
`kubectl api-resources --api-group=<absent group>` exits 0 and prints an
empty list, so guarding on the exit code always fell through. The delete
then failed with `the server doesn't have a resource type "keycloakclient"`
— `--ignore-not-found` covers a missing object, not an unknown type.

Because the oidc-bootstrap Job is a post-install/post-upgrade Helm hook,
that failure blocked the Helm upgrade of every `kubernetes-<cluster>`
release on any cluster without the EDP Keycloak operator CRDs, which is
the default configuration (`oidc.mode: None`).

Capture the discovery output once and gate each delete on its own
resource type, matching the probe idiom already used by the platform
migrations and the ouroboros cleanup hook.

Fixes #3516

Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
Assisted-by: LLM
2026-09-29 09:36:55 +02:00
Kirill Ilin 5d02c94e71 chore(hetzner-robotlb): update robotlb to 0.1.0
The upstream project moved from the intreecom GitHub organization to
treetscom, so the chart and image are now pulled from the new location.

Assisted-By: LLM
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
2026-09-29 10:50:55 +05:00
Aleksei Sviridkin 5e32ef35b0 feat(build): build Talos, Harbor, the Velero KubeVirt plugin and the unlisted images for arm64 (#4552)
## What this PR does

This brings together four approved PRs that give the remaining
first-party images an arm64 build. Each of them lost its approval when
#4498 merged and GitHub retargeted it to main, so I merged them into one
branch to review and approve once. Each original PR has the full
description and review history, and I'll close them in favour of this
one.

- #4507 builds the Talos installer and matchbox for arm64. Releases also
publish the arm64 ISO, disk images, kernel and initramfs.
- #4515 builds the KubeVirt Velero plugin in-tree for amd64 and arm64.
- #4525 adds flux-plunger, keycloak-operator, kilo and
migration-controller to the root `build:` list, so CI rebuilds them like
every other image. A test keeps any new image target from being left
out.
- #4528 rebuilds the Harbor v2.15.1 images for amd64 and arm64, because
upstream publishes v2.15 for amd64 only.

Release builds still run with `PLATFORM=linux/amd64`, except the Talos
installer, so these images ship arm64 only once #4505 adds the arm64
leg. This PR makes that leg buildable.

A few commits are new on top of the originals. The velero test no longer
pins the plugin's image string, because the release build stamps its own
rebuild over it. It now checks the plugin init container by repository
name. A promote-retag test comment now says what it guards, and the
image-refs doc no longer calls its fixed gaps open. The merge with main
keeps the Harbor portal in the list of stages whose output has no
architecture.

### Screenshots

Not a UI change.

### Downstream repositories

I left every box empty and opened nothing in other repositories. Two
repositories are touched, and neither follow-up is needed before this
merges:

- cozystack/website names the amd64 assets in
`content/en/docs/next/install/talos/iso.md` and
`install/providers/oracle-cloud.md`. The install pages could offer the
arm64 downloads once a release carries them. No asset is renamed, so the
current links keep working.
- cozystack/talm points at `ghcr.io/cozystack/cozystack/talos:<version>`
in `charts/cozystack/values.yaml`. Once a release publishes that tag as
a two-arch index, the same reference works on arm64 nodes, so nothing
there has to change.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(talos): the Talos installer image is built for amd64 and arm64, with arm64 profiles that carry drbd and zfs, and the bootbox matchbox image network-boots arm64 machines as well as amd64 ones. Releases publish the arm64 Talos ISO, disk images, kernel and initramfs.
feat(velero): the KubeVirt Velero plugin is now built in-tree for amd64 and arm64, so the velero server starts on arm64 clusters.
build: flux-plunger, keycloak-operator, kilo and migration-controller are now rebuilt by CI like every other first-party image, so their pinned images pick up source patches and base-image updates and are published for amd64 and arm64.
feat(harbor): the Harbor application now runs on arm64 nodes. Its component images are rebuilt from the pinned upstream v2.15.1 release for amd64 and arm64, because upstream publishes v2.15 images for amd64 only.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Talos images and boot assets now support both AMD64 and ARM64,
including architecture-specific installation and cloud images.
* Added build support for Harbor components and the KubeVirt Velero
plugin.
  * Nightly builds now include ARM64 release assets.
* **Improvements**
* Harbor uses pinned chart and image versions, with built image
references recorded automatically.
* Added checks to catch image packages missing from the main build and
to verify multi-architecture installer output.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-29 01:37:52 +03:00
IvanHunters 689b7eefb9 feat(system): update ingress-nginx, cozy-proxy, cozystack-scheduler and keycloak-kms-proxy to multi-arch releases (#4549)
## What this PR does

Four components move to their first releases built for both linux/amd64
and linux/arm64. The current pins are amd64 only, so these pods cannot
start on an arm64 node.

- ingress-nginx: the cozystack controller and protobuf-exporter rebuild
goes to v1.11.6, and the vendored chart moves to the matching 4.11.6.
Upstream 1.11.6 is a maintenance release (Go, dependencies, NGINX base).
The template and Lua code the rebuild patches are unchanged since
1.11.5.
- cozy-proxy v0.4.0
- cozystack-scheduler v0.4.0
- keycloak-kms-proxy 0.2.3

The charts were re-vendored with `make update`. Every digest resolves to
an index that has both platforms.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(system): ingress-nginx v1.11.6, cozy-proxy v0.4.0, cozystack-scheduler v0.4.0 and keycloak-kms-proxy 0.2.3 now ship linux/arm64 images.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Updates**
  - Updated Cozy Proxy and Cozystack Scheduler to version 0.4.0.
- Updated Ingress-NGINX to 1.11.6, including its certificate-generation
component.
  - Updated Keycloak’s KMS proxy to version 0.2.3.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-29 00:32:18 +03:00
Aleksei Sviridkin 88b61c96de docs(kubevirt): state the preconditions of the eviction comments (#4441)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

This fixes two comments in the KubeVirt CR that #4403 got wrong. It
follows up on the [review of
#4403](https://github.com/cozystack/cozystack/pull/4403#pullrequestreview-5301013108),
and I checked both findings against the source first. Only comments
change, the rendered CR is the same.

The comments didn't say that the new eviction behaviour applies only to
VMIs created after the switch. The VMI mutator copies the cluster
`evictionStrategy` into the VMI spec at creation (KubeVirt v1.9.0
`pkg/defaults/defaults.go:208-211`), and the eviction admitter reads the
spec first. So a non-migratable VMI created before the switch still has
`LiveMigrate` and keeps blocking the drain. The workload updater can't
evict it either, and it stays on the old launcher until its VM restarts.

The worker comment described the wrong mechanism. When a VMI can't
migrate and has `evacuationNodeName` set, CAPK's `IsTerminal()` returns
true and the controller sets `FailureReason` on the KubevirtMachine.
This is upstream CAPK code at the pinned commit, the cozystack patch
doesn't touch it. Every kubernetes-nodes group has a MachineHealthCheck,
and CAPI v1.10.1 treats `FailureReason` as unhealthy, so the Machine
gets replaced. If `maxUnhealthy` holds remediation back (at the default
`50%`, a group of one), the VM starts a new VMI in place. CAPK never
clears `FailureReason` though, so the Machine stays failed until
remediation is allowed.

The comment stays on the `evictionStrategy:` line, because `make update`
inserts blocks right before it. `helm unittest` passes 33/33 and
`update_idempotency_test.sh` passes. `make update` leaves the file
byte-identical.

The third finding (tests pin values, not behaviour) is tracked in #4422.

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

No UI change.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
docs(kubevirt): the KubeVirt CR comments now say that a VMI created before the evictionStrategy change keeps LiveMigrate until its VM restarts, and that the MachineHealthCheck replaces a non-migratable kubernetes-nodes worker when the workload updater evicts it.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Clarified that existing VM instances retain the eviction strategy they
were created with, and may need to be restarted before they can migrate
during a drain.
* Expanded guidance on how eviction of non-migratable workers affects
machine health and replacement, including when remediation is delayed by
the `maxUnhealthy` limit.
  * No rendered configuration values changed.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 23:09:22 +03:00
Aleksei Sviridkin 8d547be7fe fix(tests): reject unreadable RD preset schemas (#4191)
## What this PR does

`rd-presets-check` discards `jq` errors, so malformed schema can pass
the check or appear as missing presets. It now fails with the schema
path and `jq` diagnostic. An absent schema or a readable schema without
`resourcesPreset` is still skipped.

Regression tests use real `jq` for malformed input and partial output.
They also cover the canonical preset set, exact matching and the
long-list case fixed by
[#4204](https://github.com/cozystack/cozystack/pull/4204).

### Testing

- `make unit-tests -j4 --output-sync=target` passed.
- Nine cases in `hack/check-rd-presets_test.bats`, run with `bats` and
`dash hack/cozytest.sh`.
- Both read-failure cases fail against `main`. Eleven deliberate code
mutations also fail the expected test.
- `shellcheck hack/check-rd-presets.sh hack/check-rd-presets_test.bats`.

### Screenshots

Not applicable.

### Downstream repositories

- [x] No downstream repository is affected by this change

No files or make targets were moved or renamed. The change is limited to
the RD checker and its tests.

### Release note

```release-note
fix(tests): Fail the RD preset check when a schema cannot be read, with the schema path and jq diagnostic.
```
2026-09-28 22:23:09 +03:00
Aleksei Sviridkin cb035f8eac docs(agents): stop calling the image-ref known gaps unfixed
Two of the gaps listed there are handled in the tree now, and each entry
already says whether it is. The heading sentence still claimed all of
them were open.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 22:17:07 +03:00
Aleksei Sviridkin 029707f0b2 test(hack): say what the promote-retag ownership check guards
The comment above the ownership loop described how both images used to
be dropped. Describe the reason the check matters instead: neither image
has a release tag to fall back on, so a miss would go unnoticed.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 22:16:58 +03:00
Aleksei Sviridkin d190dcec74 test(velero): check the KubeVirt plugin init container by repository
The release build stamps its own rebuild of the plugin over the
upstream ref, so the test matches the repository name alone. It still
fails when the init container is dropped or points at another image.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 21:56:58 +03:00
Aleksei Sviridkin 02461acdde Merge remote-tracking branch 'origin/main' into feat/arm64-bundle
Signed-off-by: Aleksei Sviridkin <f@lex.la>

# Conflicts:
#	hack/image-builders-cross-compile.bats
2026-09-28 21:42:18 +03:00
Aleksei Sviridkin 0040d6aa3d feat(keycloak): update keycloak-kms-proxy to 0.2.3 with arm64 images
0.2.3 publishes the proxy as a linux/amd64 and linux/arm64 index;
earlier releases were amd64 only.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 21:41:32 +03:00
Aleksei Sviridkin a0d3dba07f feat(cozystack-scheduler): update to v0.4.0 with arm64 images
v0.4.0 publishes the scheduler as a linux/amd64 and linux/arm64
index; earlier releases were amd64 only.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 21:41:32 +03:00
Aleksei Sviridkin c22cc2851e feat(cozy-proxy): update to v0.4.0 with arm64 images
v0.4.0 publishes cozy-proxy as a linux/amd64 and linux/arm64 index;
earlier releases were amd64 only.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 21:41:31 +03:00
Aleksei Sviridkin 75d7573942 feat(ingress-nginx): update controller to v1.11.6 with arm64 images
The cozystack controller and protobuf-exporter rebuilds are now
published for linux/amd64 and linux/arm64, so the ingress controller
can start on arm64 nodes. The rebuild is tagged by its upstream
version, and v1.11.6 is the first multi-arch one, so the chart moves
to the matching 4.11.6.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 21:41:31 +03:00
Aleksei Sviridkin b2ae6b773a fix(tenant): fail fast on an empty tenant apex and correct gateway docs (#4538)
Closes #4536

## What this PR does

Two fixes to how tenant charts handle the tenant apex host
(`_namespace.host`), from an audit of what happens when a tenant's
`host` is changed or left empty.

**1. Fail fast on an empty tenant apex (was: silent malformed
hostname).**
Charts that publish a service compose their hostname as
`<sub>.<_namespace.host>` (for example `harbor.<apex>`, `s3.<apex>`,
`grafana.<apex>`). When the injected apex was empty and no per-app
`host` override was set, they emitted a malformed trailing-dot hostname
(`harbor.`) into Ingresses, HTTPRoutes and Certificate dnsNames with no
error. Only the gateway path guarded against this.

A new library helper `cozy-lib.ns-host-required` now sources the apex at
each hostname-emitting site and fails the render with a clear message
when the apex is empty and no override is set. It is scoped to the
injected state, so standalone renders (`helm template` / `helm lint`
with no `_namespace`, or an empty `_namespace` map), no-exposure
tenants, and conditionally emitted resources (external=false, the
kubernetes no-etcd beacon path) are unaffected. The two post-delete
cleanup hooks are left unguarded so an empty apex cannot block
uninstall.

Charts covered: harbor, postgres (external), qdrant (external),
kubernetes (control-plane endpoint), vpn, seaweedfs, bootbox, monitoring
(grafana, alerta, grafana OIDC client), and system/bucket (which also
gains the `cozy-lib` library it was missing, both the local subchart
symlink and its package-source registration).

**2. Correct the tenant `gateway` parameter documentation.**
The `gateway` description (values.yaml, values.schema.json, README, the
Go type, the tenant-rd CRD, and an inline comment in namespace.yaml)
claimed the chart auto-enables a Gateway for tenants whose apex is
derived from the parent (empty `host`) and leaves it off for custom-apex
tenants. The resolver `tenant.gatewayEffective` never inspects `host`:
an unset `gateway` key always means the tenant owns no Gateway and
inherits from the nearest ancestor that owns one. The unit tests already
pin this behavior. The description is rewritten to match and
regenerated.

**Behavior change.** A tenant with an empty injected apex and no `host`
override that enables one of the covered components now fails its render
(the HelmRelease reports NotReady) instead of deploying a broken
hostname. In practice the apex is always injected (the root tenant has
`root-host`, children inherit it), so this only trips a genuinely
misconfigured tenant.

Tests: a new `cozy-lib.ns-host-required` unit test, plus per-chart guard
tests for harbor, qdrant, kubernetes, postgres, vpn, bootbox, seaweedfs,
monitoring and system/bucket. All affected chart suites pass.

### Screenshots

Not applicable (no UI change).

### Downstream repositories

I walked the trigger map against the diff. The only touchpoint is the
tenant `README.md` gateway description, which the website release bot
regenerates automatically from `README.md` on a stable tag (no manual
follow-up). The terraform provider is not affected: this is a
description-only change to the existing `gateway` field, with no field,
default or enum change, and the tenant-rd CRD schema reaches the
provider as opaque JSON. Boxes left for a maintainer to confirm.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(tenant): tenant charts now fail rendering with a clear error when a tenant's apex host is empty and no per-app host override is set, instead of silently emitting a malformed hostname; the tenant `gateway` parameter description is corrected to match actual behavior (an unset `gateway` never auto-enables a Gateway).
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Host-dependent services now show a clear rendering error when neither
a tenant host nor an app host is configured, instead of producing an
invalid hostname.
* Tenants without `gateway: true` use the nearest ancestor’s Gateway,
whether their host is inherited or custom. Enable `gateway: true` for a
tenant’s own Gateway, IP, and certificate; this is required to make a
custom apex publicly reachable.
* **Documentation**
* Clarified Gateway inheritance behavior and when a tenant needs its own
Gateway.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 21:32:36 +03:00
Aleksei Sviridkin 3d2dfb1ed4 fix(build): let a builder stage inherit its base stage's platform (#4545)
## What this PR does

The check that builder stages compile natively is red on main, and this
is a false positive. The shadowbox Dockerfile builds its webpack stage
as `FROM source AS app`, where `source` is a `--platform=$BUILDPLATFORM`
stage. The check reads only the literal FROM line, so it reports `app`
as emulated. BuildKit gives a stage the platform of the stage it starts
from, so `app` already runs natively.

The check now tracks which named stages are native and carries that to
any stage built from one of them. A stage built from an emulated stage
stays emulated. The arch-neutral list now also takes a `<file>:<stage>`
entry. With it only the shadowbox JS stage is excused, and its Go stage
still has to use TARGETARCH.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```
2026-09-28 21:13:04 +03:00
Aleksei Sviridkin f301cb551b feat(api): add configurable resources and replicas for cozystack-engine components (#2956)
## What this PR does

Exposes standard Helm `resources` values for the three core workloads in
the `cozystack-engine` package, and makes `cozystack-controller`
replicas configurable (was hardcoded to `1`).

**Changes by component:**

| Component | `resources` | `replicas` |
|---|---|---|
| `cozystack-api` | ✅ NEW (`cozystackAPI.resources`) | already
configurable |
| `cozystack-controller` | ✅ NEW (`cozystackController.resources`) | ✅
NEW (`cozystackController.replicas`, default `1`) |
| `lineage-controller-webhook` | ✅ NEW
(`lineageControllerWebhook.resources`) | already configurable |

Each template uses `{{- with .Values.xxx.resources }}` so the
`resources:` block is only rendered when values are provided. Default
`resources: {}` produces no output, so resource requests/limits stay
opt-in.

**Leader election:** `cozystack-controller` now always runs with leader
election — the chart passes `--leader-elect` unconditionally and ships a
namespaced `Role`/`RoleBinding` for the Lease in `cozy-system` (where
controller-runtime creates it). Raising `cozystackController.replicas`
therefore adds standbys instead of a second active writer over the
reconcilers that patch HelmReleases, Gateways, Certificates,
ResourceQuotas and Secrets; this holds whatever `replicas` is. A restart
can pause reconciliation for up to the Lease duration (15s by default),
because `LeaderElectionReleaseOnCancel` is off. With `replicas: 1` the
change for an existing install is the flag plus the two RBAC objects.

**Why:** Without these values, operators cannot set QoS guarantees and
must resort to `kubectl patch`, Flux Kustomize SSA patches, or Kyverno
mutations — all fragile under Helm upgrades.

Closes #2955

### Release note

```release-note
feat(api): add configurable `resources` (requests/limits) for cozystack-api, cozystack-controller, and lineage-controller-webhook.
`cozystack-controller` replicas is now configurable via `cozystackController.replicas` (default: 1).
The `cozystack-controller` Deployment now always runs with leader election and creates a Lease in `cozy-system`, so raising `replicas` adds standbys rather than a second active controller; a restart can pause reconciliation for up to the Lease duration (15s by default).
An empty `resources: {}` renders no resources block, so requests and limits remain opt-in.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Container resource requests and limits can now be configured for the
API, controller, and webhook services; defaults remain unchanged.
* The controller replica count is configurable, and leader election is
enabled for multi-replica operation.
* Controller leader-election access is limited to the `cozy-system`
namespace.

* **Documentation**
  * Documented the webhook resource configuration option.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 20:56:43 +03:00
Mattia Eleuteri af8cf743cd fix(backups): hold a BackupJob while its application is being restored
Nothing on the backup path looks at restores. A Plan creates its
BackupJobs on schedule, and the BackupJob reconciler dispatches them to
their driver whatever else is happening to the application. For CNPG that
is not merely a backup of a half-restored state: every BackupJob reconcile
re-applies the Cluster's barman plugin, and on a cluster under an in-place
restore it can land after the chart has re-rendered the Cluster with its
new serverName and put it back on the source's. The restored cluster then
archives onto the source's WAL prefix and its recovery fails with "WAL
archive check failed ... Expected empty archive". This was reproduced
with a backup triggered a few seconds after an in-place restore, and an
hourly plan overlaps a restore of up to thirty minutes about half the
time.

A BackupJob that has not started yet now waits while a non-terminal
RestoreJob in its namespace writes into the same application, resolved
as the drivers resolve it: the Backup's applicationRef, with the target's
kind and name when a targetApplicationRef is set, and the target's API
group only when it names one. A RestoreJob whose Backup is gone holds
nothing, since the restore controller fails it on its next pass.
options.targetNamespace is honoured only for a Backup taken by the Velero
strategy, the one driver that reads it; the CNPG and Job-based drivers
ignore the option and restore in place, so their restores hold whatever
it says.

The held job stays Pending with a Ready=False RestoreInProgress condition
naming the restore and is requeued, rather than failed, so a scheduled run
still happens once the restore ends. A failed write of that status is
returned, so the manager retries it. When the restore ends the condition is
removed before the job starts, since it would otherwise read "waiting"
until the driver next writes Ready, which for a healthy run is when the
backup completes. A run that has already started is left alone: its driver
owns state the gate would strand.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-09-28 19:29:22 +02:00
Aleksei Sviridkin 4c945dbc40 fix(build): let a builder stage inherit its base stage's platform
The cross-compile check read only the literal FROM line, so a stage
built FROM an earlier --platform=$BUILDPLATFORM stage counted as
emulated even though it runs natively. The shadowbox webpack stage is
such a stage, and it failed the check on main.

The check now follows FROM <stage> to its base and inherits its
platform. ARCH_NEUTRAL also accepts "<file>:<stage>", so the webpack
stage can be marked as producing no architecture without excusing the
Go stage in the same Dockerfile. A lookup failure now fails the stage
instead of passing it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 20:19:52 +03:00
Aleksei Sviridkin b75507bbf1 test(velero): stop asserting the KubeVirt plugin image string
The plugin image is now built in-tree and stamped by digest, so any
assertion on its string only restates the pin and breaks on every
build or bump without testing what the chart does.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 20:04:11 +03:00
Aleksei Sviridkin 5a619c5fdc Merge remote-tracking branch 'origin/feat/velero-kubevirt-plugin-multiarch' into feat/arm64-image-builds
Signed-off-by: Aleksei Sviridkin <f@lex.la>

# Conflicts:
#	Makefile
2026-09-28 19:57:27 +03:00
Aleksei Sviridkin 8c1266649d Merge remote-tracking branch 'origin/feat/harbor-multiarch' into feat/arm64-image-builds
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 19:49:55 +03:00
Aleksei Sviridkin 79854b16fb Merge remote-tracking branch 'origin/fix/build-list-orphan-images' into feat/arm64-image-builds
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 19:49:54 +03:00
Aleksei Sviridkin 2f08832392 Merge remote-tracking branch 'origin/feat/talos-arm64' into feat/arm64-image-builds
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-28 19:49:46 +03:00
Aleksei Sviridkin 6d759cc73e feat(build): build package images for amd64 and arm64 (#4498)
## What this PR does

Package images can now be built for amd64 and arm64 in one `make image`,
and a published build is multi-arch unless `PLATFORM` says otherwise.
This is the build half of arm64 support from #1961 and #903. CI still
publishes amd64 only until it gets a native arm64 leg, which comes in a
separate PR.

Builder stages now run on the build platform and compile for the target
one, so Go never runs under emulation. Under emulation it is slow and it
crashes: the migration-controller Dockerfile already records a "bad
pointer in Go heap" from exactly that. On top of this, `PLATFORM`
defaults to `linux/amd64,linux/arm64`. A build on an arm64 workstation
can't publish an arm64-only digest any more. That happened once already,
the cilium pin was arm64-only until 3f36a1b45 rebuilt it by hand.

Two images were only right because they were emulated. On a native
builder they would ship the wrong architecture. token-proxy ran `go
build` with no GOARCH, and kube-ovn calls upstream `make build-go`,
which hardcodes `GOARCH=amd64`. Both are fixed now. The new bats test
wants every compiling stage on `$BUILDPLATFORM` and actually using
`TARGETARCH`. A bare `ARG TARGETARCH` does not count, this is how
token-proxy got through.

Some things stay amd64. talos and testing pin it with `override`,
because matchbox serves amd64 Talos assets and the sandbox runs
`qemu-system-x86_64`. `LOAD=1` builds for the host only, since the
classic docker image store can't load an index. Every CI build job pins
`PLATFORM: linux/amd64`, so CI output does not change.

This changes local builds. A `make image` on the default docker driver
with the classic image store now fails with "Multi-platform build is not
supported for the docker driver", where before it built for the host.
Use a `docker-container` builder (`BUILDER=<name>`), or pass
`PLATFORM=linux/amd64` (or `LOAD=1`) to build one architecture.

I checked it on an arm64 host with a docker-container builder.
cozystack-api, dashboard, kubeovn, platform-migrations, cilium and
capi-providers-cpprovider build and push as amd64+arm64 indexes.
Binaries pulled from each platform are x86-64 and aarch64:
cozystack-api, token-proxy, the kamaji provider manager, cilium-agent
and all four kube-ovn binaries. piraeus-server builds for both
platforms, but the push to my test registry failed on registry 500s. The
capi-providers-cpprovider inline cache works with a multi-platform push.
Not covered is running the arm64 images on an arm64 cluster, CI has
none.

This is stacked on #4485 and targets its branch, so the diff here is
only the two commits on top.

### Screenshots

No UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: cozystack/website#711
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(build): package images build for linux/amd64 and linux/arm64 by default, with Go builders cross-compiling on the build platform; a local `make image` now needs a docker-container builder, or PLATFORM=linux/amd64 / LOAD=1 for a single architecture. CI still publishes amd64.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Build Improvements**
* Container image builds now target both amd64 and arm64 by default when
building without loading images locally.
* CI and release builds are pinned to amd64; Talos and testing images
remain amd64-only.
* Cross-platform builds now compile on the builder’s native platform
while targeting the requested architecture.
* **Documentation**
* Clarified platform defaults and architecture requirements for local
and CI builds.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 19:43:47 +03:00
Aleksei Sviridkin 597d966d7d fix(redis): run the multi-arch redis_exporter so the sidecar starts on arm64 (#4522)
## What this PR does

Redis and Valkey instances, and the Redis inside Harbor, never become
fully ready on arm64 nodes. Their failovers pin
`oliver006/redis_exporter:v1.55.0-alpine`, which upstream publishes for
amd64 only. On arm64 the exporter sidecar exits with `exec
/redis_exporter: exec format error`, and the pod stays not ready.

The same release without the suffix is a multi-arch index with amd64,
arm and arm64 builds. Its entrypoint, user and port match the `-alpine`
image, and none of these charts run a shell in the exporter container.
So this swaps the tag and pins it by digest, with no version bump.

I checked both tags with `crane`. On an arm64 test cluster a Redis and a
Valkey instance hit the exec format error before the change and came up
2/2 with it. The Harbor Redis pods went to 2/2 the same way.

On upgrade the exporter image in the pod template changes, so the
operator rolls every Redis and Valkey instance and Harbor's Redis, one
pod at a time as it does for any failover update.

Fixes #4519

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff changes one image reference in three templates. No entry in the
trigger map covers a sidecar image.

### Release note

```release-note
fix(redis): Redis, Valkey and Harbor's Redis now use the multi-arch redis_exporter image, so the exporter sidecar starts on arm64 nodes.
```
2026-09-28 19:36:36 +03:00
Aleksei Sviridkin a2b89236a0 feat(vpn): build shadowbox for amd64 and arm64 (#4521)
## What this PR does

The VPN app can't run on arm64 nodes. Upstream publishes
`quay.io/outline/shadowbox` for amd64 only, so on arm64 the pod
crash-loops with `exec /usr/local/bin/docker-entrypoint.sh: exec format
error`. The last upstream release is v1.12.3 from February 2025. The ARM
request there, OutlineFoundation/outline-server#1009, is still open.

Upstream's own build already knows how to target arm64, it just never
publishes that build. It pins an arm64 node base, cross-compiles
outline-ss-server and has prometheus checksums for both architectures.
This PR rebuilds v1.12.3 from source as a multi-arch image, following
the layout of upstream's `docker:build` task. The source is pinned to
the release commit. The webpack bundle is built once on the build
platform, the Go server and prometheus per target. The runtime base is
the node 18.18.0 index whose amd64 and arm64 manifests are exactly the
ones upstream pins, so amd64 keeps the same base layers.

The chart now reads the image the same way as the other first-party
images, from a stamped `.tag` file. It is pinned by digest instead of
the floating `:stable` tag. Until the first CI build stamps the file, it
holds upstream v1.12.3 by digest, which is what `:stable` resolves to
today. The package is in the root `build:` list, so every build path
refreshes it.

I built both platforms and checked each one. The binaries are the right
ELF architecture, the management API answers and access keys get
created. On an arm64 test cluster the VPN app with this image went
Ready. An HTTPS request through a real shadowsocks client on another
node came back 200. The helm-unittest no longer asserts the image
string, since that would only restate the pin.

This gives an arm64 variant only when the build gets a platform list.
Current CI builds amd64 only, so until the multi-arch build lands
(#4498) this is a same-behaviour rebuild. Also, `cozystack/shadowbox` is
a new image. Its e2e registry repo may come up private on first push,
which shows up as a 403 on pull, not as a code problem.

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff touches one app's chart and Makefile and the root `build:`
list. No entry in the trigger map covers an app's image source.

### Release note

```release-note
feat(vpn): the VPN app now uses a shadowbox image that Cozystack builds from the upstream v1.12.3 source, pinned by digest, instead of the floating amd64-only `:stable` upstream tag. The image builds for amd64 and arm64; releases publish the arm64 variant once the multi-arch release build is in place.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Updates**
* The VPN application now uses a pinned Shadowbox image instead of the
stable image tag.
  * The VPN image build supports multiple architectures.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 19:36:30 +03:00
Aleksei Sviridkin 17838d46dc fix(ingress): run the default backend on arm64 (#4517)
## What this PR does

The ingress default backend can't start on arm64 nodes: it crash-loops
with `exec /server: exec format error`. `packages/system/ingress-nginx`
turns the default backend on without setting an image, so it gets the
chart default `registry.k8s.io/defaultbackend-amd64:1.5`. That image is
amd64 only, and upstream never published a multi-arch defaultbackend.

This pins `registry.k8s.io/ingress-nginx/custom-error-pages:v1.2.9` by
its multi-arch index digest. Upstream ships it as the replacement
default backend, with amd64, arm and arm64 builds. It answers 404 on
:8080 and 200 on `/healthz`, so the chart's port and probes still fit.
The host ingress, the tenant ingresses and the tenant Kubernetes
ingress-nginx addon all render this package without their own image, so
this one change covers them.

I ran the arm64 image locally under the chart's securityContext (uid
65534, read-only root filesystem, all capabilities dropped). `/healthz`
returned 200, `/` returned 404, and memory stayed around 6Mi of the 20Mi
limit. The same override runs on an arm64 test cluster. A new
helm-unittest case checks the default backend image by digest; it fails
on main and passes here.

There are two side effects. custom-error-pages logs a line for every
request, because it looks for error page templates under `/www` that we
don't ship and falls back to a plain `404 page not found`. The old
backend didn't log, and its body was `default backend - 404`. The status
code is still 404.

Also, Renovate won't update this pin. The `helm-values` manager is
disabled in `.github/renovate.json`, and no regex manager matches the
registry/image/tag/digest layout. The controller image in the same file
has the same problem, so I kept Renovate config out of this PR.

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff touches only `packages/system/ingress-nginx/values.yaml` and a
new test in that package. No entry in the trigger map covers a system
package's image values.

### Release note

```release-note
fix(ingress): the ingress default backend now uses the multi-arch custom-error-pages image instead of the amd64-only defaultbackend image, so it starts on arm64 nodes.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Pinned the default error-page image to a specific version and digest,
improving consistency across deployments.
  * Added a chart test to verify the configured image.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 19:36:25 +03:00
Aleksei Sviridkin aa59c2f1d5 fix(foundationdb-operator)!: drop the amd64-only 7.1 init container (#4512)
## What this PR does

The FoundationDB operator never starts on arm64. Its
`foundationdb-kubernetes-init-7-1` init container runs
`foundationdb/fdb-kubernetes-monitor:7.1.67` and fails with `exec format
error`. All 67 published 7.1.x tags of that image are single amd64
manifests, so there is no multi-arch 7.1 tag to move to. The 7.3.63 and
7.4.1 tags, and the operator image itself, publish linux/arm64. Upstream
hits the same gap in its own build: the operator Dockerfile at v2.30.0
swaps its built-in 7.1 client for 7.3 on arm64.

The fix removes the 7.1 entry when the chart is vendored, instead of
overriding it in the package values. The `update` recipe deletes it with
`yq`, and the committed chart equals what the recipe produces. A `null`
override would not last. Helm 4.2 and later ignore a `null` that a
parent chart sets on a subchart key when no user-supplied values reach
that subchart (helm/helm#32132). The platform passes this package no
values, so that is exactly the case here. helm-unittest renders with
Helm 3, so the override would silently stop working after a Flux bump.
`helm template` now renders only the 7.3 and 7.4 init containers under
both Helm 4.1.1 (what helm-controller v1.5.0 uses) and Helm 4.3.0. The
test pins exactly two init containers, so it fails if a later `make
update` brings 7.1 back.

This is a breaking change. The operator loses the 7.1 `fdbcli`,
`fdbbackup`, `fdbrestore` and client library, so it can no longer manage
or upgrade a FoundationDB cluster pinned to 7.1.x. The app defaults to
7.3.63 and nothing in this repository sets 7.1, but `cluster.version` is
a free string. A user who set 7.1.x has to upgrade that cluster to 7.3
before installing this release.

Related: #4511, the app still accepts any string as the FoundationDB
version, 7.1 included.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The diff touches one system package: its update recipe, its vendored
chart values, and its test. No app schema, platform value, CRD or shared
tooling changes, and a search of the cozystack org finds no page or code
that offers FoundationDB 7.1.

### Release note

```release-note
fix(foundationdb-operator)!: the FoundationDB operator now starts on arm64. It no longer ships the 7.1 client, because no arm64 build of it exists, so it cannot manage or upgrade a FoundationDB cluster on 7.1.x. Upgrade such clusters to 7.3 before installing this release.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Updates**
* The FoundationDB operator chart no longer includes the 7.1
init-container configuration. Its remaining configured init-container
versions are 7.3 and 7.4.
* The chart update process applies the same version change to copied
chart values.
* Image validation now checks for exactly two init containers, using
versions 7.3.63 and 7.4.1.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 19:36:19 +03:00
Aleksei Sviridkin 2e8219fb92 fix(build): pass the shared buildx flags from every image target (#4485)
## What this PR does

Five image builds skipped the shared buildx flags in
`hack/common-envs.mk` and wrote their own: flux-plunger,
kubeovn-plunger, both dashboard images, and metallb. The plungers
ignored `PLATFORM`, so on an arm64 workstation they came out as arm64
images that fail on amd64 nodes with `exec format error`. CI builds on
amd64, so it never showed. Dashboard and metallb hardcoded
`--platform=linux/amd64`; for metallb that sat next to `$(BUILDX_ARGS)`,
so setting `PLATFORM` gave buildx two platforms. All of them now build
through `$(BUILDX_ARGS)` only.

How it split: kubeovn-plunger landed two days before 9f2b98d36 moved the
other Makefiles onto `BUILDX_ARGS`, flux-plunger was copied from it, and
dashboard kept its own flags when that change touched it.

CI builds do not change. CI sets only `BUILDER`, which these targets
already honoured, and every build runner is x86-64.

One behaviour change is intentional: a local `make image` for dashboard
or metallb now follows the host architecture, like every other package.
On an arm64 machine, build for an amd64 cluster with
`PLATFORM=linux/amd64`.

A new bats test dry-runs every package image target with a probe
platform and a marker in `BUILDX_EXTRA_ARGS`. Each buildx call must
carry the marker and exactly that platform. On main it flags the five
targets above.

### Screenshots

No UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(build): every package image build honours PLATFORM, SBOM and BUILDX_EXTRA_ARGS; local dashboard and metallb builds now follow the host architecture unless PLATFORM is set.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* System image builds now use shared Buildx settings for platform
selection and publishing options, while retaining their image tags,
cache settings, and metadata.
* MetalLB image builds no longer explicitly target `linux/amd64`; the
selected platform follows Buildx defaults and configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 19:36:07 +03:00
Aleksei Sviridkin 9c94337f17 fix(mariadb): correct webhook cert-manager override key path (#4537)
## What this PR does

Fixes cozystack/cozystack#3974: the values override at
`packages/system/mariadb-operator/values.yaml` targeted
`webhook.certificate.certManager`, a field that does not exist in the
vendored mariadb-operator chart. The chart's actual key is
`webhook.cert.certManager.enabled` (see
`charts/mariadb-operator/values.yaml:139-142`), so the override was
silently ignored and the webhook always fell back to mariadb-operator's
own cert-controller instead of cert-manager. This corrects the key path
so the override takes effect.

### Screenshots

Not applicable, no UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change

### Release note

```release-note
fix(mariadb): correct webhook cert-manager override key path so cert-manager is actually used for the mariadb-operator webhook certificate
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated the MariaDB operator’s certificate-management configuration to
use its current setting structure. The certificate manager remains
enabled, now under the updated configuration path. Installations that
use the operator will apply the enabled setting from the revised values
configuration.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 18:53:51 +03:00
Aleksei Sviridkin 26e4083d52 fix(linstor): adopt existing CR on 409 AlreadyExists in k8s backend (#4532)
## What this PR does

This adds a LINSTOR server patch for a controller that could refuse
every write of one property until it was restarted, which left a PV
stuck in `Released` with its ZFS volume still allocated. It is a
backport of LINBIT/linstor-server#531.

linstor-csi starts every volume delete by setting
`BalanceResourcesEnabled` on the resource definition. With the
Kubernetes CRD backend that write can come back as 409 AlreadyExists
when the fabric8 client retries a POST the apiserver already persisted.
A single-object transaction has no rollback entry, so the controller
drops the key from memory while the CR stays, and the caching client
never re-reads it. Every retry of the delete then fails the same way.

The patch makes create treat a 409 as success when the object exists: it
takes the live resourceVersion and replaces the object with the intended
content. The unit test that comes with the patch was red before the fix
and green after, on v1.33.3 and on upstream master, and the whole patch
series still applies to v1.33.3.

The retry being the trigger is my best explanation, not proven: the run
had no apiserver logs. Until this ships, restarting linstor-controller
clears the state.

Closes #4531

### Screenshots

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(linstor): the LINSTOR controller no longer gets stuck refusing writes to a property after a retried Kubernetes API request, which could leave a deleted volume in Released with its storage still allocated.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved recovery when Kubernetes reports that an object already
exists during creation. The system now adopts the existing object and
continues updating it, helping prevent volume operations from becoming
stuck and requiring a controller restart.
* Other conflict and error cases continue to report their original
errors.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 18:39:43 +03:00
IvanHunters 2fa51bf587 test(charts): isolate guard coverage for every guarded template
Add per-template guard suites so each guarded hostname site is exercised
on its own: harbor harbor.yaml and httproute.yaml, bootbox machines.yaml,
seaweedfs cosi-deployment.yaml, bucket deployment.yaml and httproute.yaml,
monitoring alerta.yaml, grafana httproute.yaml and the grafanaHost helper
via oidc-keycloak.yaml. Each suite renders only its target template, so
removing that file's guard reddens it. Also add a postgres positive case
that renders the Certificate with an apex set (external=true).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 18:32:49 +03:00
IvanHunters de359f858c docs(tenant): correct gateway=false line in the test rules table
The rules-under-test header still described gateway=false as "parked,
no inherit". It behaves like unset and still inherits. Align it with the
fixed comment below it.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 18:23:03 +03:00
IvanHunters 30782d003c fix(platform): attach cozy-lib to the bucket-system package source
system/bucket now calls cozy-lib.ns-host-required, but in a cluster the
chart is built from the ExternalArtifact per its PackageSource, not from
the charts/cozy-lib symlink used by helm and helm-unittest. The
bucket-system component did not list cozy-lib, so the built chart lacked
the helper and every cozy-bucket install (including the platform backups
bucket) failed to render. Attach the library to the component, as the
apps/bucket component already does.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 18:23:03 +03:00
Aleksei Sviridkin fa6e3ce850 docs(kubernetes): clarify node pool Helm timeout semantics (#4496)
## What this PR does

Corrects the explanation for the Kubernetes node-pool Helm timeout.

With readiness waiting disabled, nothing in this chart's install or
upgrade is bounded by the 15-minute timeout: the ordinary
`talos-reconcile` Job continues asynchronously, and the chart has no
install or upgrade hooks. The timeout takes effect only if readiness
waiting is re-enabled or such a hook is added.

The annotations and their values are unchanged.

Fixes #4406

### Screenshots

N/A — no UI changes.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Clarified that with readiness waiting disabled, the 15-minute Helm
timeout does not limit chart installs or upgrades: reconciliation runs
asynchronously, and the chart has no install or upgrade hooks.
* Noted that the timeout applies if readiness waiting is re-enabled or a
hook is added, and that the release settles while reconciliation
continues.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 17:55:57 +03:00
Aleksei Sviridkin ec9843be4a docs(virtual-machine): correct persistent EFI migration documentation (#4494)
## What this PR does

Removes the outdated warning that persistent EFI state on default RWO
storage makes a virtual machine node-pinned and prevents live migration.
The warning no longer reflects the supported migration behavior and
could discourage users from persisting Secure Boot keys.

This is a documentation-only change. The chart documentation, JSON
schema, generated API comment, and ApplicationDefinition schema were
regenerated without changing defaults, validation, or runtime behavior.

Fixes #4420

### Validation

- `make -C packages/apps/vm-instance generate`
- `make generate`
- `make manifests`
- `make -C packages/apps/vm-instance test` — 59 tests passed
- `git diff --check`
- A second complete generation pass produced no additional changes

### Screenshots

N/A — documentation-only change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated EFI persistence descriptions to remove claims that using
default RWO storage pins virtual machines to a node or prevents live
migration. The persistence behavior and UEFI-only applicability remain
unchanged.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 17:53:50 +03:00
Aleksei Sviridkin ba1e63c665 ci(pre-commit): detect schema drift across all packages (#4497)
## What this PR does

Extends the `run-make-generate` pre-commit hook to discover package
Makefiles with an actual `generate` target across every package
category.

This preserves existing app and extra coverage while adding system and
library packages such as `talos-log-collector`, `kubevirt-cdi`, and
`cozy-lib`. A focused Bats test verifies category coverage, skips
packages without a generator, excludes nested vendored Makefiles, and
confirms generator failures are propagated.

The newly covered `cozy-lib` package exposed deterministic schema
formatting drift, which is included in this change.

Fixes #4416

### Screenshots

N/A — no UI changes.

### Downstream repositories

The developer-tooling trigger indicates that the website development
documentation should be reviewed. No downstream follow-up was published
as part of this PR.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Pre-commit package generation now checks eligible package directories
across more categories, running generation where a matching target is
available. If generation fails, subsequent packages are not processed.
* **Tests**
* Added checks covering package discovery across categories, exclusion
of ineligible directories, and failure handling that prevents later
packages from being processed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 17:40:19 +03:00
Nikita Aboltin cb8728dde1 docs: note the measured idle usage next to the empty resources defaults
The resources keys stay empty so requests and limits are opt-in, and the comment
records the measured idle usage next to them, so a limit that does get set is not
placed below what the component needs at idle.

Signed-off-by: Nikita Aboltin <aboltin.nikita@rwb.ru>
2026-09-28 16:43:44 +03:00
IvanHunters 0f4d39f1ef test(charts): add empty-apex guard tests for the remaining guarded charts
Add host_guard_test.yaml for postgres, vpn, bootbox, seaweedfs,
monitoring and bucket. Each renders its guarded template with a
non-empty injected _namespace whose host is empty and no per-app host
override, and asserts the ns-host-required failure, plus a positive
render case with the apex set. Completes guard coverage alongside the
existing harbor, qdrant and kubernetes tests.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 16:01:31 +03:00
IvanHunters fe97d4838d test(cozy-lib): exercise the host-override branch of ns-host-required
The override case injected no _namespace, so it passed through the
absent-map branch and never reached the .Values.host override check.
Inject a non-empty _namespace with an empty host so the case actually
covers the override skip.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 15:52:52 +03:00
IvanHunters eedb446e8a docs(tenant): clarify that gateway=false behaves like unset
The rewritten gateway description claimed setting false opts out of
Gateway publishing entirely, including inheritance. It does not:
tenant.gatewayEffective returns "false" for both an explicit false and
an unset key, so namespace.yaml keeps _namespace.gateway pointing at the
nearest ancestor that owns a Gateway and the tenant still publishes
through it. State that false currently behaves the same as unset, and
fix the matching stale comment in gateway_default_test.yaml.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 15:52:46 +03:00
IvanHunters 6602c080a1 fix(kubernetes): drop apex guard on oidc-rbac-job
The oidc-rbac bootstrap Job renders unconditionally, including the
no-etcd bootstrap path and with OIDC disabled, so guarding its
OIDC_SERVER_URL apex failed a render that must succeed: helm template
of the chart with no etcd DataStore (hack/admin-kubeconfig-invariant.bats)
aborted instead of emitting only the awaiting-etcd beacon. A running
cluster's apiserver endpoint is already guarded in cluster.yaml (etcd
present), which is the meaningful malformed-hostname case, so revert the
oidc-rbac-job guard rather than gate it further. Restore the no-etcd
unit fixture to an empty host so it keeps exercising that path.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 15:46:00 +03:00
Nikita Aboltin 8732dc96cf feat(api): add configurable resources and replicas for cozystack-engine components
- cozystack-api, cozystack-controller, lineage-controller-webhook: conditional
  resources block via `{{- with .Values.*.resources }}`, defaulting to `{}` so the
  default render stays unchanged
- cozystack-controller: replicas comes from values (was hardcoded 1); the chart
  now passes --leader-elect unconditionally and grants the Lease verbs through a
  namespaced Role and RoleBinding in cozy-system (the Lease lives in the manager's
  own namespace), so raising replicas adds a standby instead of a second active
  writer over the reconcilers that patch HelmReleases, Gateways, Certificates,
  ResourceQuotas and Secrets

Signed-off-by: Nikita Aboltin <aboltin.nikita@rwb.ru>
2026-09-28 15:19:45 +03:00
IvanHunters 37f59369d3 fix(charts): fail fast on empty tenant apex in host composition
Charts that publish a service compose their hostname as
<sub>.<_namespace.host>. When the injected tenant apex is empty and no
per-app host override is set, they used to emit a malformed trailing-dot
hostname (e.g. "harbor.") into Ingresses, HTTPRoutes and Certificate
dnsNames silently. Source the apex through cozy-lib.ns-host-required at
each hostname-emitting site so the render fails with a clear message
instead, matching the gateway path.

Covered: harbor, postgres (external), qdrant (external), kubernetes
(control-plane endpoint), vpn, seaweedfs, bootbox, monitoring
(grafana, alerta, oidc client), and system/bucket (adds the cozy-lib
subchart symlink it lacked). The guard is placed at the point each
resource is actually emitted, so standalone renders, no-exposure
tenants and the kubernetes no-etcd beacon path are unaffected. The
post-delete cleanup hooks are left unguarded so an empty apex cannot
block uninstall.

Behavior change: a tenant with an empty injected apex and no host
override that enables one of these components now fails its render
(HelmRelease NotReady) instead of deploying a broken hostname. In
practice the apex is always injected (root has root-host, children
inherit it), so this only trips a genuinely misconfigured tenant.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 15:14:45 +03:00
IvanHunters 2dc884fe05 docs(tenant): fix stale gateway comment in namespace.yaml
The inline comment still described the old, incorrect behavior, that
tenant.gatewayEffective returns true for a custom non-derived host.
The resolver keys only on an explicit tenant.spec.gateway=true. Align
the comment with the code and the corrected parameter description.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 15:12:33 +03:00