mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-22 04:33:36 +09:00
52aaffb5bd437abdcf98c2e0278afb744715bc1c
6930
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
52aaffb5bd |
fix(kubernetes): accept a golden worker image that exists only as a PVC
CDI clones the golden's PVC, not its DataVolume, so a golden whose DataVolume was garbage-collected (dataVolumeTTLSeconds) or whose PVC was restored on its own is still a usable clone source. vm-disk already accepts one; a worker pool refused it and failed the whole release. The absence guard now fires only when neither object exists. With no DataVolume the StorageClass and size guards already fall back to the golden's PVC, so a PVC-only golden is still checked against the pool. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
d6e5c1bf2b |
fix(api): preserve foreign HelmRelease metadata on Application update (#2914)
## What this PR does
The aggregated `apps.cozystack.io` API rebuilds the HelmRelease from the
Application on `Update` and issues a full-replace PUT. This strips
metadata owned by other controllers:
- `metadata.finalizers` — including helm-controller's
`finalizers.fluxcd.io`
- `metadata.ownerReferences`
- labels and annotations outside the `apps.cozystack.io-` prefix (e.g.
`reconcile.fluxcd.io/requestedAt`)
Losing the Flux finalizer is the harmful case: helm-controller re-adds
it on the next reconcile, but a delete landing in that window removes
the HelmRelease **without running `helm uninstall`**, orphaning every
resource of the release (VMs, databases, PVCs...). A label patch
immediately followed by a delete — a common client pattern — hits that
window almost every time. We traced ~106 orphaned releases across 39
tenants on a production cluster back to this.
Reproduction:
1. `kubectl patch <any apps.cozystack.io resource> --type merge -p
'{"metadata":{"labels":{"foo":"bar"}}}'` → the backing HelmRelease loses
`finalizers.fluxcd.io`
2. `kubectl delete` the resource right after → HelmRelease is removed
instantly, Helm uninstall never runs, child resources stay alive
Changes:
- `Update` now fetches the current HelmRelease and carries over
`finalizers`, `ownerReferences`, and non-prefixed labels/annotations
before the PUT
- Keys under the `apps.cozystack.io-` prefix remain owned by the API and
are recomputed from the Application, so label/annotation deletions still
propagate
- Regression test `TestUpdate_PreservesForeignHelmReleaseMetadata` pins
the carry-over contract (fails on the previous behavior with exactly the
stripped fields)
## Release note
```release-note
fix(api): updating an Application no longer strips finalizers, ownerReferences, and foreign labels/annotations from the backing HelmRelease, which could cause deletion to skip helm uninstall and orphan the release's resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Application updates now preserve metadata managed by external
controllers, including finalizers, owner references, labels,
annotations, and shard information, when synchronizing related releases.
* Conflict retries now retain the latest release metadata, preventing
changes made between attempts from being lost.
* Creating an application without labels no longer alters shared release
configuration labels.
* **Tests**
* Added coverage for metadata preservation during updates and conflict
retries, and for label handling during application creation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
9b56e70ca6 |
feat(mongodb): support the system-bucket backup flow (useSystemBucket) (#3967)
## What this PR does MongoDB was the only `cozy-default` app with **no `useSystemBucket` flow**: its backup storage was always static on the `PerconaServerMongoDB` CR (bucket/endpoint/credentials from `backup.*` values plus a chart-emitted `<release>-s3-creds` Secret), so a tenant could not back up to the platform system bucket out of the box the way Postgres/ClickHouse/MariaDB/etcd do. This closes that gap so the default MongoDB backup-restore works OOTB. PBM takes `bucket`/`endpointUrl`/`region`/`prefix` as **static CR fields** (not secret refs), and the platform bucket name/endpoint are only known at BackupJob time — so, unlike ClickHouse's runtime `secretKeyRef` env, the chart cannot render them. The implementation mirrors the CNPG driver's pattern: - **Chart** (`packages/apps/mongodb`): a new `backup.useSystemBucket` flag keeps `spec.backup.enabled` (so the pbm agents run) but omits the static `storages`/`tasks`/`pitr` and the `<release>-s3-creds` Secret. Helm never owns `spec.backup.storages`, so a Flux re-render can't fight the driver. A `bootstrap.enabled + useSystemBucket` render guard mirrors Postgres, and `useSystemBucket: true` with `backup.enabled: false` is refused both at render time and by an `x-kubernetes-validations` rule on `backup` in the schema (the same mechanism as the `storageClass` immutability rule). Note on the schema rule: the aggregated apiserver does not evaluate CEL on application-backed resources yet (#2657), so today it is a UI/schema-level marker and the render-time refusal is what stops a Flux release; it also trips the API review gate (“validation rule added”), so this PR needs an API-owner approval on top of the code review. - **Strategy** (`cozy-default-mongodb`): now carries the S3 coordinates (`endpointUrl` already scheme'd by the `backupstrategy-controller.endpoint` helper, `credentialsSecret: cozy-backups-creds`, `prefix: <namespace>/<application>`). - **Driver** (`psmdbstrategy_controller.go`): SSA-injects `spec.backup.storages[s3-storage]` onto the live `PerconaServerMongoDB` at BackupJob time under a dedicated field manager (`cozystack-psmdb-backup-driver`) with `ForceOwnership`. Injection is **gated on the app's `backup.useSystemBucket` flag** (a legacy app leaves it false and is never touched; a useSystemBucket app is re-injected once per mint (once per BackupJob, plus once per storage-race re-mint), before its operator CR is created, so a later coordinate change catches up on the next job while a streaming dump is never patched under). The backup CR is minted only once the operator reports the storage pushed into pbm (`status.observedGeneration` caught up and `PBMReady` True): pbm writes to what its own config says, not to the spec, so a CR minted before that push would land at the previous coordinates while its Backup recorded the new ones; the job waits at `PerconaServerMongoDBPBMConfigPending` meanwhile, bounded by the backup deadline. Once a job has its CR, none of the mint-time reads or holds can fail it: the strategy and app reads the artifact still needs are bounded by the 24h running ceiling instead, and the cluster is not read at all **and on the release having rendered that flag**: the app CR is projected from the HelmRelease's desired values, so the flag reads true before helm-controller renders the revision, while the tenant's own storage, scheduled tasks and PITR stream from the previous render still sit on the cluster. As long as the cluster carries `spec.backup.tasks` or `spec.backup.pitr`, the driver does not inject and holds the BackupJob at `Ready=False PerconaServerMongoDBLegacyRender` (failing it at the deadline), so a wedged legacy revision never has its nightly dump and oplog redirected into the shared bucket. The backup snapshot falls back to the strategy coordinates when the operator status echo is empty; that fallback is gated on the CR's own `percona.com/delete-backup` finalizer (the flow the CR was minted on), not on the app flag re-read at completion, so a legacy backup never records the platform bucket and a flag flipped mid-dump does not change the artifact. The restore path rebuilds `backupSource` from it (the `cozy-backups-creds` reference is projected into the restore namespace before dispatch). - **Retention/cleanup**: on the `useSystemBucket` flow the driver owns the shared-bucket archive (the chart's psmdb `tasks[].keep` retention is gated out, and an on-demand backup is not covered by task retention). It stamps the operator's `percona.com/delete-backup` finalizer on each backup it mints **against a storage that carries the platform coordinates** — the strategy's bucket and `credentialsSecret`, read back from the live cluster after the injection — never on the flag alone: a strategy without an `s3` block, or a storage that points anywhere else, holds the job (`MongoDBStrategyHasNoS3` / `PerconaServerMongoDBStorageForeign`) instead of minting a finalized dump against a bucket the platform does not own. On Cozystack `Backup` deletion (by hand today; by platform retention once it lands, since a `Plan` only creates BackupJobs and never deletes a `Backup`) the driver deletes the operator `PerconaServerMongoDBBackup` CR so that finalizer frees the object — the finalizer's presence is the ownership marker cleanup reads back. That reading is unambiguous because cleanup resolves the CR by the name recorded in `driverMetadata` (the one the driver minted), not by scanning: the psmdb operator does stamp `percona.com/delete-backup` on its own `cron-*` task-minted backups, but those are never in scope. A legacy backup keeps the pre-existing hands-off contract, decided from the snapshot before any read of the operator CR. A terminating namespace or the `backups.cozystack.io/skip-artifact-cleanup: "true"` annotation strips the driver's finalizer and releases; a `Get` of the operator CR that fails with anything other than NotFound/NoMatchError still requeues, and the operator's own `release-lock` finalizer can hold the CR while the operator is down, so those two are what can keep a namespace in `Terminating`. `schedule` and `retentionPolicy` are inert on this flow (there is no engine-side retention and a `Plan` has no retention field yet); platform-level retention is tracked in [#4192](https://github.com/cozystack/cozystack/pull/4192) and teardown GC of orphaned backups in [#3966](https://github.com/cozystack/cozystack/issues/3966), so until those land bound `cozy-backups` with an S3 lifecycle policy. - **Bounds on the waits**: the injection race with the operator's cluster cache (an `unable to get storage` error on a freshly injected storage) is retried by deleting the errored CR and minting a fresh one, with the retry count kept on the BackupJob (`backups.cozystack.io/mongodb-storage-race-retries`) because the deleted CR is reaped before the next poll; after three retries the operator's error is terminal. A system-bucket dump that has started streaming is not cancelled on the 30-minute deadline, but fails after 24 hours in `running` (`BackupAbandoned` warning naming the operator CR, which is left in place as the handle to whatever pbm still writes). The deadline cancel of a not-yet-started CR pins its delete to the resourceVersion the live read answered for. - **RBAC**: `patch` on `perconaservermongodbs` for the injection (server-side apply), and `create`/`delete`/`patch` on `perconaservermongodbbackups` — `delete` to prune the shared-bucket archive on `Backup` deletion, `patch` to strip the driver's finalizer when releasing a wedged cleanup. Coverage: Go unit tests over `internal/backupcontroller` — the injection gate keyed off the app flag (legacy cluster never patched) and off the applied render (a cluster still carrying tasks or pitr is held, untouched), the SSA-injection precondition with the whole entry (per-application prefix included) asserted on the cluster, strategy templating per application, the finalizer refused against a storage without platform coordinates (strategy without `s3` over a legacy entry, foreign coordinates on the merged entry), the retry budget driven with the operator simulated reaping the deleted CR between polls, the restore snapshot fallback and credential swap across all four legacy/system flow combinations, the finalizer-owned cleanup with its escape hatches (skip annotation before any read, `NoMatchError` treated as gone, terminating-namespace release using the pre-read CR, a failed strip surfaced as an Event, a legacy CR never deleted, a legacy snapshot released without a read, a system-bucket snapshot reaching the prune), the deadline cancelling a not-yet-running operator backup only on a positive live read (including no live client at all, and a CR that moved since the read), and the running ceiling. Plus helm-unittest on both charts (chart skips the creds Secret and omits `spec.backup.storages`/`tasks`/`pitr` on `useSystemBucket` and keeps them on the legacy flow; the strategy renders the S3 coordinates and the `insecureSkipTLSVerify` knob). A live `useSystemBucket` backup+restore e2e against the platform bucket is a follow-up (needs the system bucket wired in CI, same status as the Postgres/ClickHouse useSystemBucket e2e). One item to confirm in e2e: the psmdb operator accepting `spec.backup.enabled=true` with an empty `storages` between install and the first BackupJob. ### Screenshots Not applicable (no UI changes). ### Downstream repositories The tenant-facing schema surface is one new field, `spec.backup.useSystemBucket`, on the MongoDB `values.schema.json` / CRD (regenerated), plus a validation rule on `spec.backup`. `terraform-provider-cozystack` hand-models MongoDB and may want to expose it, but it is a new optional boolean that defaults false (no drift for existing users), and the chart `README.md` regenerates on release. Leaving every box unticked per the trigger-map guidance so a maintainer decides on the provider follow-up. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(mongodb): MongoDB backups now support the platform system bucket via backup.useSystemBucket, so the default cozy-default BackupClass works without per-app S3 credentials — the backup driver injects the system-bucket storage onto the cluster at backup time, once the release has rendered the flag. On this flow nothing prunes the shared bucket automatically yet (a Plan carries no retention field; tracked in #4192 and #3966), so an S3 lifecycle policy on cozy-backups is a prerequisite for enabling it, not a suggestion. useSystemBucket requires backup.enabled=true; the chart refuses the combination with backups disabled at render time, and the schema carries the same rule as a marker until the apiserver enforces CEL (#2657). A system-bucket dump still running after 24 hours fails its BackupJob with the operator backup left in place. A MongoDB restore now adopts the target cluster's S3 credentials only for a storage on the same bucket as the archive (it used to adopt them unconditionally); a legacy restore into a target on a different bucket that relied on one key pair covering both must keep or re-create the source release's -s3-creds Secret by name, and is held at Ready=False RestoreCredentialsMissing until it exists. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added opt-in system bucket mode for MongoDB backups. * Automatically injects platform bucket storage without application-managed S3 credentials. * Supports endpoint, region, path prefix, addressing, and TLS settings. * Manages shared-bucket backup artifacts when system bucket backups are deleted. * **Bug Fixes** * Improved restore credential selection to avoid cross-bucket mismatches. * Preserved backup metadata when storage details are unavailable. * **Documentation** * Documented configuration, scheduling, retention, PITR limitations, and restore guidance. * **Validation** * Prevented incompatible bootstrap and system bucket restore configurations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
0a2054e228 |
fix(kubernetes): apply worker TalosConfigTemplate server-side (#4163)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The `talos-reconcile` Job runs under a 256Mi limit and applies its `TalosConfigTemplate` with a client-side apply. Building the merge patch means pulling the cluster's full OpenAPI schema down, which on a management cluster carrying a few hundred CRDs costs around 290Mi of RSS, so the kernel OOMKills kubectl before the Job finishes. Only the merge path pays that cost, which is what makes this hide so well. The first run creates the object and stays cheap, so a fresh install goes green; every reconcile afterwards is an update, so the Job CrashLoopBackOffs on every chart upgrade until it burns its `backoffLimit: 30`. The KamajiControlPlane patch lands before the apply dies, so the cluster keeps working and the failure reads as cosmetic noise. Measured against a 438-CRD management cluster with the Job's own kubectl (`alpine/k8s:1.33.4`), cold discovery cache: | command | peak RSS | | --- | --- | | `get -o jsonpath` | 52Mi | | `apply`, object absent (create) | 52Mi | | `apply`, object present (merge) | **292Mi** — OOMKilled at 256Mi | | `apply --server-side` | 56Mi | `--validate=false` changes nothing, so the cost is the merge strategy lookup rather than validation. Server-side apply leaves the merge to the apiserver and the client never needs the schema. `--force-conflicts` keeps the Job authoritative over an object an earlier client-side apply created. Raising the memory limit instead would work today and rot tomorrow: the cost tracks the cluster's CRD count, so the ceiling would need lifting again as the API surface grows. Includes a regression test asserting the rendered Job applies server-side and carries no client-side fallback. It fails on the pre-fix template. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(kubernetes): apply the worker TalosConfigTemplate server-side so the talos-reconcile Job is no longer OOMKilled on clusters with a large CRD count. The Job previously failed on every reconcile after the first install, which meant on every chart upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved Kubernetes reconciliation reliability by using server-side updates for Talos configuration templates. * Prevented reconciliation jobs from exceeding their memory limit during chart upgrades. * Ensured the reconciliation job remains authoritative when updating previously created resources. * **Tests** * Added coverage verifying server-side application behavior and conflict handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8b861668b8 |
ci(release): build pre-release images for amd64 and arm64 and gate releases on it (#4505)
## What this PR does Pre-release builds now publish every image as an amd64 and arm64 index, and a release fails if anything it ships is single-arch. Main, release lines and pull requests keep building amd64 only. It builds on #4498, which made every image buildable for both architectures. Part of #1961. **This PR is on hold until everything under "Merge after" has landed.** Merged earlier, the new gate fails the next rc on the images those PRs fix. ### How a pre-release is built A pre-release tag gets a second job, for arm64, next to the existing release job. That covers `-rc.N`, `-beta.N` and `-alpha.N` tags. It runs on the CNCF arm64 pool (`oracle-vm-24cpu-96gb-arm64`), builds the same images with `PLATFORM=linux/arm64`, and pushes them as `<tag>-arm64`. `prepare-release` waits for it, and if the arm64 job fails the rc fails with it. There is no amd64-only fallback. Stable tags do not run it: promotion copies the rc digests with `skopeo copy --multi-arch all`, so a stable release inherits the indexes. After the amd64 build, the stitch joins each amd64 image with its arm64 twin into one index. It lives in `hack/stitch-multiarch.sh`. Every tag of the amd64 image moves to the index, the component-versioned ones included. The script then rewrites the digests in the tree and republishes the packages artifact and the installer chart pinned on them. It reads refs through `hack/lib/image-refs.sh`, and it fails if an old digest survives anywhere outside `charts/`. ### The gate A new check fails the rc when a pinned digest in the tree is not an index with both linux/amd64 and linux/arm64. It lives in `hack/verify-multiarch.sh`. It checks first-party and third-party refs alike. Images that are amd64 by nature go into `hack/multiarch-allowlist`, one repository per line, each with a mandatory reason. Today that is only the e2e sandbox. An entry that matches nothing is reported as stale. The static gate only sees digest-pinned refs in the tree. A tag-only ref, an image from a vendored chart default, or one an operator starts at runtime is invisible to it. So the rc e2e also audits every image its nodes pulled, with the same check and the same allowlist. Neither covers a package the e2e never installs. ### Nightly A new nightly workflow builds main for arm64. It keeps the arm64 build cache warm, since the rc job only reads it. It also prints the single-arch refs that would block the next rc. Nothing it builds is published. ### Smaller changes - `CACHE_TAG` in `hack/common-envs.mk` moves the default cache ref, so the arm64 build keeps a cache of its own. - `MATRIX_ARCH=arm64` makes `hack/build-matrix.sh` leave out the amd64-only e2e sandbox. - The kamaji provider image is pushed under the build's `IMAGE_TAG` like every other image. Its Makefile set `IMAGE_TAG` itself, so concurrent builds overwrote each other's component tag. Closes #4503. - `hack/nightly-mirror.sh` verifies a copy against the raw manifest digest. Plain `skopeo inspect` resolves an index to one platform's child. - matchbox is rebuilt for both platforms in the amd64 release job. The arm64 leg skips the talos package, so there is no arm64 half to stitch, and its Dockerfile only copies files, so no QEMU is needed. Releases then network-boot arm64 machines too. Closes #4524. The stitch also rewrites the kamaji ref inside `files/components.gz`, recompressed with `gzip -n` so the bytes are reproducible, so the kamaji control-plane provider ships multi-arch too. keda and kuberture now name the repository next to their pinned digest, so the gate can resolve them; their rendered manifests do not change. ### Merge after - #4552 builds the Talos installer, matchbox, Harbor and the Velero KubeVirt plugin for arm64, and adds flux-plunger, keycloak-operator, kilo and migration-controller to the root `build:` list. It replaces #4507, #4515, #4525 and #4528. - #4549 moves ingress-nginx, cozy-proxy, cozystack-scheduler and keycloak-kms-proxy to their multi-arch releases, which are already published. The rest of the stack is merged: #4485, #4498, #4512, #4517, #4521 and #4522 here, and the multi-arch PRs in ingress-nginx-with-protobuf-exporter, cozy-proxy, cozystack-scheduler and keycloak-kms-proxy. ### Verification The bats suites pass: stitch, verify-multiarch, build-matrix, common-envs, nightly-mirror and the release contracts. The first three also pass under `hack/cozytest.sh` with dash. Each new test was red before its implementation, and the verify-multiarch and stitch tests each have a mutation that turns them red. I ran `verify-multiarch --report` against main. It lists the first-party refs the stitch will fix, plus the third-party and special cases above. The workflows have not run yet. The arm64 job's tools and duration, the stitch against real registries, and the audit are checked for the first time on the nightly and on the next rc. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Nothing under `hack/` is moved or renamed, and no make target changes its default behaviour: `CACHE_TAG` and `MATRIX_ARCH` are opt-in. The satellite repositories listed under "Merge after" are prerequisites, not follow-ups this change forces on them. ### Release note ```release-note ci(release): pre-release builds publish every image as an amd64 and arm64 multi-arch index, and a release fails if any image it ships or pulls in e2e lacks either architecture. Stable releases inherit the indexes through promotion. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Release candidates now build amd64 and arm64 images in parallel and combine eligible images into verified multi-architecture indexes before publication. * A nightly arm64 image build is available for testing and supplies a build cache for release-candidate builds. * **Bug Fixes** * End-to-end checks audit whether pulled images support both architectures. Audit failures block prerelease checks, while stable releases continue with a warning. * End-to-end test artifacts now include collected image references and multi-architecture audit results. * **Documentation** * Updated release and image guidance covers multi-architecture builds, verification checks, and troubleshooting, including arm64 build and stitching failures. <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.7.0-alpha.2 |
||
|
|
31948c8838 |
fix(api): redo the HelmRelease metadata carry-over on a conflict retry
A 409 on the HelmRelease PUT means the live object changed after the read the carry-over was built from, and the writer behind it is normally helm-controller - the controller whose finalizer the carry-over exists to protect, and which re-adds it on the reconcile following a metadata-only update. Retrying with the metadata read before the conflict wrote that stale copy back and dropped the finalizer the reconcile had just restored, reopening the orphaning window the carry-over closes. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
6d541c63eb |
fix(api): make mergeMaps always allocate, never return an input map
mergeMaps returned one of its inputs whenever the other was nil. Create and Update seed the HelmRelease labels with mergeMaps(r.releaseConfig.Labels, ...) and then write the application metadata labels into the result in place, so for an Application with no labels the writes landed on the long-lived config map shared by every request for the kind: requests cross-contaminate (application.name varies per request) and concurrent map writes crash the apiserver. Signed-off-by: mattia-eleuteri <mattia@hidora.io> |
||
|
|
a18cbc581c |
fix(api): preserve foreign HelmRelease metadata on Application update
Update rebuilds the HelmRelease from the Application and issues a full-replace PUT, which stripped metadata owned by other controllers: finalizers, ownerReferences, and any labels or annotations outside the apps.cozystack.io- prefix. Losing helm-controller's finalizers.fluxcd.io finalizer is the harmful case: a delete landing before helm-controller re-adds it removes the HelmRelease without running helm uninstall, orphaning every resource of the release (VMs, databases, PVCs). A label patch immediately followed by a delete - a common client pattern - hits that window almost every time. Fetch the current HelmRelease before the PUT and carry over its finalizers, ownerReferences, and non-prefixed labels and annotations. Keys under the apps.cozystack.io- prefix remain owned by the API and are recomputed from the Application, so label/annotation deletions still propagate. Signed-off-by: mattia-eleuteri <mattia@hidora.io> |
||
|
|
d9ed566326 |
fix(migration): restore the class reclaim policy on imported volumes (#4480)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The VM import handoff sets the transferred `PersistentVolume` to `Retain` so the data survives the swap from the Forklift-populated claim to the VMDisk's claim, and then never gives the policy back. Every imported disk therefore ends up with `persistentVolumeReclaimPolicy: Retain`, while its StorageClass says `Delete` and every VMDisk created directly carries `Delete`. When the tenant later deletes an imported VMDisk through the normal path, its DataVolume and claim go away, but the volume stays `Released` with its backing storage: an orphaned, billed volume that no tenant can see or remove. #### How it was measured On a lab cluster with a LINSTOR-backed `replicated` class (`reclaimPolicy: Delete`): a 16 GiB VM imported from vSphere produced a VMDisk whose PV was `Retain`, while every non-imported VMDisk's PV in the cluster was `Delete`. Deleting the VMInstance and then the VMDisk removed the DataVolume and the claim and left the PV `Released`, still `Retain`, with its LINSTOR resource in place. Patching that PV back to `Delete` made the CSI provisioner delete both the PV and the LINSTOR resource, which confirms the reclaim policy is the only thing keeping it. #### The fix `Retain` is now held only for the window it protects. A new step, `settleReclaimPolicy`, restores the policy the volume's StorageClass declares (`Delete` when the class names none or no longer exists) once the handoff is provably over, and until then reports "not yet" and touches nothing: - the replacement claim is `Bound` and the PV is `Bound` with a `claimRef` naming that claim by UID, as the PV controller reports it rather than as the controller wrote it, so no release is in flight; - the transferred claim is gone; - the claim's one and only owner is the VMDisk's DataVolume, as a controller reference whose UID matches the live DataVolume (CDI's adoption), so the only deletion that can reach the volume is the VMDisk's own; - the VMDisk exists and carries this task's output markers. `fulfill` runs this step after adopting every disk and before creating the VMInstance and deleting the scaffolding VM, so a VM is reported `Succeeded` only once its volumes carry their class's policy, and an import that fails after this point still leaves disks that clean up like any other. The step is a compare-and-set, so repeated passes and restarts are no-ops once settled. It also runs on the path that recovers outputs whose status was never written, for tasks caught mid-handoff by an upgrade. The comment that called permanent `Retain` "part of the contract" and the matching paragraph in `packages/system/migration-controller/README.md` are updated; the README also says how to fix volumes left on `Retain` by imports completed before this change, since those tasks are terminal and are not reconciled again. In practice a VM now stays in `Creating` for one more requeue (about 20 s) after the swap, while the PV controller rebinds and CDI adopts the claim. #### Tests - `TestSettleRestoresTheClassReclaimPolicy`: a finished handoff gets the class's policy back: `Delete`, `Retain` when the class asks for it, and `Delete` when the class names none or is gone. - `TestSettleIsIdempotent`: a second pass over a settled volume reports settled and does not rewrite the PV. - `TestSettleKeepsRetainWhileTheHandoffIsOpen`: thirteen cases, each breaking one condition (claim not created by this import, claim not bound, PV still `Released`, `claimRef` still on the old claim, `claimRef` on the new claim with a stale UID, transferred claim still present, claim not yet adopted by CDI, an extra owner, a DataVolume owner that is not the controller, an owner reference to a previous DataVolume, DataVolume missing, VMDisk missing, VMDisk not created by this import). Every one keeps `Retain`. - `TestFulfillRestoresTheReclaimPolicyBeforeFinishing`: behavioural regression over `fulfill`. The first pass swaps the volume, keeps `Retain`, creates no VMInstance and keeps the scaffolding VM. After simulating the PV controller and CDI, the second pass restores `Delete`, creates the VMInstance and removes the scaffolding VM. With the new step turned into a no-op, this test fails. - `TestFulfillSettlesReclaimedOutputs`: the recovery path for outputs whose status was never written. With the handoff still open the volume keeps `Retain`; once it is finished the policy goes back to `Delete`. - The existing `adoptVolume` tests still assert `Retain` during the swap. `go test ./internal/migrationcontroller/...` and `make test-controllers` pass, `go vet` is clean, and `make generate` leaves the tree unchanged. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only `internal/migrationcontroller/`, `packages/system/migration-controller/README.md` and a comment in `packages/system/migration-controller/templates/rbac.yaml`, which match no entry of the trigger map; the website's VMware migration page does not describe the reclaim policy, so it needs no change. ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(migration): disks imported from VMware no longer keep a Retain reclaim policy after the import. The volume gets its StorageClass policy back once it is bound to its VMDisk, so deleting an imported VMDisk deletes its volume instead of leaving an orphaned Released PersistentVolume. Volumes imported before this fix keep Retain and have to be patched back by hand. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Imported volumes are protected during claim transfers, then returned to their StorageClass reclaim policy after the replacement claim and volume are safely adopted. If the class or policy is unavailable, the policy defaults to `Delete`. * VM imports wait for volume adoption and reclaim-policy settlement before completing, helping prevent premature volume deletion and incomplete imports. Previously unrecorded handoff outputs are also settled before being returned. * **Documentation** * Clarified reclaim-policy behavior during imports and how to restore cleanup behavior for volumes from older imports. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
1728b49e0c |
fix(kubernetes): apply worker TalosConfigTemplate server-side
The talos-reconcile Job runs under a 256Mi limit and applies its TalosConfigTemplate client-side. Building the merge patch means pulling the cluster's full OpenAPI schema down, which on a management cluster carrying a few hundred CRDs costs around 290Mi of RSS, so the kernel OOMKills kubectl before the Job can finish. Only the merge path pays that cost. The first run creates the object and stays cheap, so a fresh install succeeds and the Job then CrashLoopBackOffs on every reconcile afterwards, which means on every chart upgrade. Measured against a 438-CRD cluster with the Job's own kubectl (alpine/k8s:1.33.4), cold discovery cache: get -o jsonpath 52Mi apply, object absent (create) 52Mi apply, object present (merge) 292Mi <- OOMKilled at 256Mi apply --server-side 56Mi --validate=false changes nothing, so the cost is the merge strategy lookup rather than validation. Server-side apply leaves the merge to the apiserver and the client never needs the schema. --force-conflicts keeps the Job authoritative over an object an earlier client-side apply created. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
717d091970 |
fix(backups, postgres): start a Postgres restore from the backup it names (#4443)
## What this PR does A Postgres `RestoreJob` now starts from the base backup its `Backup` refers to, and the PITR documentation describes the window the recovery actually has. **Starting from the named backup.** Without `recoveryTarget.backupID` the barman-cloud plugin picks the base backup itself (`internal/cnpgi/restore/restore.go` in plugin-barman-cloud, then `FindBackupInfo` in barman-cloud `pkg/catalog`): the newest backup of the catalog when no `recoveryTime` is set, otherwise the newest one whose end is not after `recoveryTime`, on any timeline since no `targetTLI` is passed. A `RestoreJob` naming an older `Backup` therefore started from another one, and on an archive that holds sibling timelines — left by in-place restores that archived back under the source `serverName`, before restored clusters got a fresh `newServerName` — it could start from another branch and bring back that branch's data. `FindBackupInfo` gives `backupID` precedence over time-based selection, and CNPG's webhook does not count it among the mutually exclusive targets, so it combines with `targetTime`. - on restore the CNPG driver reads barman's `backupId` and the backup's `stoppedAt` from the `Backup`'s `cnpg.io/Backup` and sets the new chart value `bootstrap.backupID`, rendered as `bootstrap.recovery.recoveryTarget.backupID`; - it does not pin for a `recoveryTime` earlier than one second past `stoppedAt`: the named backup cannot serve an instant before it is consistent, and `stoppedAt` is truncated to the second while the catalog compares barman's end time to the microsecond. Those requests keep the plugin's time-based choice, as today; - it does not pin either when the `cnpg.io/Backup` is gone. plugin-barman-cloud's retention deletes it together with the base backup it describes (`deleteBackupsNotInCatalog`), and the driver deletes one only with its artifact, so a missing one means the catalog has lost that base backup: pinning it would purge the target and then fail the recovery. The plugin's choice recovers to the same end of the archive, and the `RestoreJob` gets a `BaseBackupGone` warning event. A base backup deleted from the object store by other means is still pinned for as long as its `cnpg.io/Backup` stays, and that restore fails after the purge; plugin-barman-cloud deletes such a `cnpg.io/Backup` on its next maintenance pass (`deleteBackupsNotInCatalog`, every `retentionPolicyIntervalSeconds`, retention policy or not), but only from the primary of the cluster that took it, under the same UID. Telling it apart earlier would need reading the barman catalog, which the driver does not do; the documentation says so, and that deleting the `cnpg.io/Backup` by hand is the way out; - a re-restore into the same target clears a `backupID` left by a previous one. **Documentation** (`docs/operations/backup-classes.md`, PostgreSQL PITR): - the upper bound of the window is the last transaction (commit or abort) in the archived WAL, not the newest segment: recovery ends just before the first transaction after the target, and a segment switch or a base backup is not one. On a database at rest every instant after its last write fails with `recovery ended before configured recovery target was reached`, including "at/before the most recent completed backup", which the page recommended as the safe choice; the state as of the last write is obtained by omitting `recoveryTime`; - aiming at a backup's `stoppedAt` misses that backup (the plugin keeps only a backup whose end is not after the target), falling back to an older one or to `no target backup found`; the page now says to aim one second past it; - when a base backup removed from the bucket outside retention is still pinned, and how to unpin it; - how to read the upper bound from the archive, what pinning the base backup changes, and a section on archives with several timelines: PostgreSQL follows `recovery_target_timeline: latest` from the backup's timeline through the consecutively numbered history files it can find, so a missing `.history` confines the replay to the backup's own timeline and the bound is the last transaction on that timeline, while a targeted timeline that does not descend from the backup stops recovery with `requested timeline N is not a child of this server's history`. **Tests.** `TestCNPGRestoreBackupID` (the pinning rule), `TestReconcileCNPGRestore_PinsTheRequestedBackup` (drives a restore through the purge and reads back the patched `Postgres` app, including a `cnpg.io/Backup` removed by retention, with and without `recoveryTime`), `TestBuildPostgresAppRestorePatch_ReplacesBackupID`, a deep-copy test for the new `stoppedAt` field, and three `helm unittest` cases for `recoveryTarget`. With the rule disabled, 4 of the rule cases and 3 of the reconcile cases fail; with the template change removed, 2 chart cases fail; the retention case fails against the first version of this PR, which pinned an id recorded on the artifact; the deep-copy test fails once `StoppedAt` is dropped from the generated `DeepCopyInto`. `go test ./internal/backupcontroller/...` and `helm unittest packages/apps/postgres` (139/139, after rebasing on main) pass. Not run: the e2e suite (`hack/e2e-chainsaw`), and a restore against a live cluster with this driver. ### Screenshots No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/issues/716. `content/en/docs/next/operations/services/backup-classes.md` (and `v1.6/`) carries a copy of the corrected "The recoverable window" and "Discovering the earliest / latest restorable time" sections; the window and `stoppedAt` corrections apply to released versions too, the `backupID` part only once this is released. - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: none expected. The diff adds `bootstrap.backupID` to the postgres `values.schema.json`, but the provider does not manage the `bootstrap` block (`internal/provider/postgresql_schema.go`: "bootstrap blocks use server defaults"), and the `RestoreJob` CRD is unchanged. Left unticked for a maintainer to confirm. - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(backups): a Postgres RestoreJob now starts from the base backup its Backup refers to (recoveryTarget.backupID) instead of whichever backup the barman-cloud plugin picks, which on an archive with several timelines could restore another branch's data; a recoveryTime before that backup's end keeps the previous time-based choice. The PITR documentation now bounds the recoverable window by the last archived transaction rather than the newest WAL segment. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Restores can target a specific base backup and optionally replay to a requested recovery time. If no backup is specified, the newest eligible backup is selected automatically. * **Bug Fixes** * Restore handling warns when the referenced base backup is unavailable and allows the plugin to select a backup from the catalog. * **Documentation** * Clarified recovery-window boundaries, timestamp precision, idle databases, timeline branches, and behavior when a requested target cannot be reached or predates the earliest base backup. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
ff0752d8e9 |
fix(kubernetes): probe Keycloak CRDs by output in oidc-bootstrap cleanup (#3520)
## What this PR does Fixes #3516. - `packages/apps/kubernetes/templates/oidc-rbac-job.yaml`: the `mode=None` cleanup path of the `oidc-bootstrap` Job guarded its Keycloak deletes with `if kubectl api-resources --api-group=v1.edp.epam.com >/dev/null 2>&1`. That command **exits 0 for an absent API group** (it simply prints an empty list), so the guard always fell through and the delete failed with `error: the server doesn't have a resource type "keycloakclient"` — `--ignore-not-found` covers a missing *object*, not an unknown *type*. Since the Job is a `post-install,post-upgrade` Helm hook with `backoffLimit: 6`, this blocked the Helm upgrade of every `kubernetes-<cluster>` release on any cluster without the EDP Keycloak operator CRDs, i.e. at the chart default `oidc.mode: None`. The probe now captures the discovery output once and gates each delete on its own resource type being present, matching the idiom already used in `packages/core/platform/images/migrations/migrations/{34,38,39}` and `packages/system/ouroboros/.../cleanup-hook.yaml`. `api-resources` is kept (rather than `get crd`) on purpose: it hits discovery, which `system:discovery` grants to every authenticated principal, so the namespaced Job Role needs no cluster-scoped CRD read. - `packages/apps/kubernetes/tests/oidc_test.yaml`: new unit test asserting the rendered script probes by output and that the exit-code form does not come back. All four of its assertions fail against the pre-fix template. `cmd/check-readiness` and the platform migrations were checked too: they already test the output, not the exit code. ## Validation - `helm unittest packages/apps/kubernetes` on the branch rebased onto current `main`: 317/317 pass. Putting `main`'s template back fails only the new case, all 4 assertions, so it is a real regression guard. - `make generate` in `packages/apps/kubernetes`: no diff. - Extracted the rendered `mode=None` script and ran `sh -n` / `shellcheck -s sh` — clean (only pre-existing `SC3040` for `pipefail` and one pre-existing `SC2086`). - Executed the extracted script against a stub `kubectl` reproducing the two real behaviors (`api-resources` exits 0 with empty output for an absent group; `delete <unknown type>` exits 1 despite `--ignore-not-found`): the pre-fix script exits 1 with `the server doesn't have a resource type "keycloakclient"`, the fixed script exits 0 with the CRDs absent and still issues both deletes with the CRDs present. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: The diff touches no `values.schema.json` and no behaviour the docs describe: the template change restores the documented `oidc.mode: None` path rather than altering it. ## Release note ```release-note fix(kubernetes): the oidc-bootstrap hook no longer fails on clusters without the Keycloak operator CRDs, which blocked Helm upgrades of kubernetes-* releases at oidc.mode=None ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * OIDC cleanup now skips deleting Keycloak resources when their resource types are unavailable, preventing cleanup failures. * **Tests** * Added coverage for resource discovery and conditional cleanup behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
abc07a3927 |
fix(backups): hold a BackupJob while its application is being restored (#4544)
## What this PR does Nothing on the backup path looks at restores. A Plan creates its BackupJobs on schedule (`plan_controller.go`), and `BackupJobReconciler` dispatches them to their driver whatever else is happening to the application. A backup taken during a restore captures a half-restored state at best, and for CNPG it does worse. We hit it on a v1.6.3 cluster. helm-controller applied the restored Cluster with `bootstrap.newServerName`, and four seconds later a BackupJob of the same application re-applied the barman plugin with the source's `serverName`. The restored cluster archived onto the source's WAL prefix, and its recovery job died with `WAL archive check failed for server <cluster>: Expected empty archive`. Correcting `serverName` by hand let the same restore finish, which is what identified the writer. `currentBarmanServerName` preserves the live value on most paths, but not in the window where the restored Cluster is being re-rendered. An on-demand run can be refused by whoever creates it, but a scheduled one cannot, and there is no `suspend` on a Plan. With an hourly plan and a CNPG restore allowed thirty minutes, the overlap happens about half the time. A BackupJob that has not started yet (`status.startedAt` unset) now waits while a non-terminal RestoreJob in its namespace writes into the same application. The target is resolved as the drivers resolve it (`resolveCNPGRestoreTarget` and the Job-based paths): the referenced Backup's `applicationRef`, with the target's `kind` and `name` when a `targetApplicationRef` is set, and the target's API group only when it names one, so a target without one stays in the Backup's group. A Velero restore always stays in the Backup's group, since the Velero driver drops the target's API group and restores the Backup's resources. A RestoreJob whose Backup is gone is skipped, since the restore controller fails it on its next pass. `options.targetNamespace` is honoured only when the Backup was taken by the Velero strategy, the one driver that reads it: the CNPG and Job-based drivers ignore the option and restore in place, so their restores hold whatever it says. The held job stays `Pending` with a `Ready=False` condition, reason `RestoreInProgress`, naming the restore, and is requeued every 30s. It is not failed: a scheduled run should still happen once the restore ends. A failed write of that status is returned, so the manager retries it, and once the restore ends the condition is removed before the job starts, since it would otherwise read "waiting" until the driver next writes `Ready`, which for a healthy run is when the backup completes. The gate sits after BackupClass resolution and strategy validation, so third-party strategies are unaffected, and before credentials projection, so a held job projects nothing. What happens to a delayed scheduled run: the Plan controller only decides when to *create* a BackupJob (a slot within `startingDeadline`, 300s, of now). Once created, the job is not subject to that deadline, so a held run keeps its `<plan>-<unix-minutes>` name and runs once the restore ends, later than its slot. If the restore outlasts the next slot, that slot's job is created and held too, and both run after the restore. Nothing is skipped, and nothing is deduplicated. Not covered, deliberately: - A backup already running when a restore starts is not interrupted. Its driver owns state the gate would strand. The reverse guard, a RestoreJob waiting for a running BackupJob, would belong in the restore path, together with a per-application claim that closes the race between the two checks, and is tracked in #4559. - A Velero RestoreJob in another namespace that targets this one through `options.targetNamespace` does not hold this namespace's backups. The lookup is namespaced. - The Velero restore path has no deadline, so a stuck VM restore holds that application's backups until the restore is ended or deleted. That matches the restore already blocking the application. Tests: `TestActiveRestoreTargeting` covers the resolution matrix (in place, to-copy into and out of the application, terminal phases, another kind or another API group under the same name, a to-copy target inheriting the Backup's API group, a Velero target naming another API group, a Velero restore sent to another namespace or naming its own, a Postgres restore carrying `targetNamespace`, a missing Backup with or without a target). `TestReconcile_RetriesAHoldItCouldNotRecord` and `TestReleaseRestoreHold` cover the status writes, and the held-job test reconciles again after the restore ends to check the condition is gone. `TestReconcile_HoldsABackupJobWhileItsApplicationIsRestored` is the behavioural regression: it fails with the gate removed (the job reaches projection and dispatch). `TestReconcile_DoesNotHoldARunningBackupJob` checks that a started job is not held. Backport: a candidate for `release-1.6`, where the reproduction was seen. `backupjob_controller.go` differs from `main` only in the dispatch switch and the Go 1.27 modernizers, so the cherry-pick should be clean or close to it. ### Screenshots N/A, no UI change. ### Downstream repositories The diff touches only `internal/backupcontroller/` Go code. It changes no CRD, API type, chart, values file, hack script or naming, so the trigger map comes out empty. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backups): a BackupJob that has not started yet, scheduled or on demand, now waits while a RestoreJob is restoring the same application, instead of running against it. It stays Pending with a Ready=False RestoreInProgress condition and runs once the restore ends. This stops a CNPG backup that has not started yet from re-pointing an in-place restored cluster at the source's WAL archive; a backup already running when a restore starts is not held. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Unstarted backup jobs now wait when an active restore targets the same application. They remain pending and are retried after a short delay, then can proceed once the restore finishes. * Backup jobs that have already started continue without being held by an active restore. Completed or failed restores do not hold pending backup jobs. * Restore matching accounts for the specified target and, for Velero restores, the target namespace. A backup job’s restore hold is cleared when no matching active restore remains. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
afe23a2c3f |
ci(release): pin flux where the release pushes the packages artifact
prepare-release now runs flux push artifact to publish the rc's packages artifact, and it installed whatever flux was latest, while build-amd64 and the candidate checks pin one version. It installs that version now. The arm64 leg and the nightly never run flux, so they no longer install it. The rc-e2e comment gave a stable tag as the reason for its !cancelled(), but the job only runs for an rc; it now says it follows the rule every job below build-arm64 has. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
3f6de5eac0 |
ci(release): keep every installer tarball out of the release assets
The release job now builds the Talos installer for both arches, so _out/assets holds an arm64 tarball next to the amd64 one. Both are the image the build pushed, not something upload-assets publishes, and the artifact only excluded the amd64 one. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
51a2d859ac |
docs(release): describe what the release job builds and what blocks an rc
The amd64 release job builds the Talos installer and matchbox for both arches, so the pages that said CI never builds both in one job, and that build-amd64 builds amd64 only, were wrong. Every rc stitch also skips matchbox and the e2e sandbox, and the nightly report lists matchbox and the refs an in-tree rebuild stamps over; the page said every such skip and line fails the next rc. Name the ones that do not. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
71144d4fd6 |
ci(release): pin the matchbox rebuild in the release job
tags.yaml only runs on tags, so no pull request exercises it. The matchbox rebuild is the only thing that makes its ref an index, and losing the step, moving it past the tree capture or letting it switch the default builder would surface as a failed gate on the next rc. The comments that called the talos images amd64 only now say they are built for both arches in the amd64 job. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
b314514950 |
chore(hetzner-robotlb): update robotlb to 0.1.0 (#4557)
## What this PR does Updates the bundled robotlb to 0.1.0. The upstream project moved from the `intreecom` GitHub organization to `treetscom`, so the chart and the image are now pulled from the new location. Upgrade note from upstream: balancer targets now follow the Service's `externalTrafficPolicy`. A Service that leaves it unset gets the Kubernetes default `Cluster`, and its balancer targets every eligible node instead of only the nodes running its pods. Set `externalTrafficPolicy: Local` to keep the previous targeting. ### Screenshots Not applicable, no UI changes. ### Downstream repositories - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#715 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The website still links to the old `Intreecom/robotlb` repository in the Hetzner install guide, the platform-stack guide and the licenses page. GitHub redirects those links, so nothing is broken; cozystack/website#715 updates them. ### Release note ```release-note chore(hetzner-robotlb): update robotlb to 0.1.0 and pull it from the new treetscom organization. Load balancer targets now follow the Service's externalTrafficPolicy; set it to Local to keep targeting only the nodes that run the Service's pods. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Updates** * Updated the RobotLB installation to use the `treetscom` chart and container image sources. * Updated the installation notes to link to the RobotLB project’s current documentation location. * Updated the RobotLB chart’s application version to 0.1.0. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6ef23da6e7 |
ci(release): publish matchbox as an amd64 and arm64 index
matchbox already carries the kernel and initramfs of both arches, but the release built it for amd64 only: the arm64 leg skips the talos package, so there is no arm64 half to stitch, and the gate let it through on an allowlist entry. Its Dockerfile only copies files, so the amd64 job now rebuilds it for both platforms on a docker-container builder and the stamped ref pins the index. The allowlist entry goes. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c740031453 |
Merge remote-tracking branch 'origin/main' into ci/build-arm64-leg
Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
1cefadcef6 |
docs(backups): say when a base backup gone from the bucket is still pinned
A restore leaves the base backup to the plugin only once the cnpg.io/Backup is gone. A base backup removed from the bucket by a lifecycle rule or by hand keeps its cnpg.io/Backup until the plugin's next maintenance pass on the source cluster, and indefinitely once that cluster no longer runs under the same UID. A restore in that window fails after the purge and on every retry, with nothing on the RestoreJob to unpin it; deleting the cnpg.io/Backup is the way out and does not touch the object store. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
28e42fbe4d |
fix(backups): match a Velero restore in its Backup's API group
The Velero driver resolves a restore's target from the Backup's applicationRef, overriding only the kind and name, and restores the Backup's resources. A targetApplicationRef.apiGroup never reaches it. The restore gate honoured that group all the same, so a Velero RestoreJob naming another group let a BackupJob of the application actually being restored start against it, and held one of an application the restore never touches. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
64e837727c |
docs(backups): say what recovery does when the latest timeline is a sibling
recovery_target_timeline latest targets the highest consecutively numbered history file, without checking that it descends from the base backup. On an archive holding sibling branches that can be another branch, and recovery then stops with "requested timeline N is not a child of this server's history" rather than following the backup's own line of descent. The page also names the second case where a restore does not pin its base backup: a cnpg.io/Backup removed by retention. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
6999609585 |
docs(backups): bound Postgres PITR by the last archived transaction
The recoverable window was described as ending at the newest archived WAL segment, and the safe fallback as "at/before the most recent completed backup". A time-targeted recovery ends just before the first transaction after the target, and a segment switch or a backup is not one, so on a database at rest every instant after its last write fails, including the end of a later backup. Aiming at a backup's stoppedAt also misses it: stoppedAt is truncated to the second and the plugin compares barman's end time to the microsecond. Describe the bound the recovery actually has, how to read it from the archive, the one-second margin on stoppedAt, what pinning the base backup changes, and why archives with several timelines need it. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
497dd6fc4f |
fix(backups): start a Postgres restore from the backup it names
Without recoveryTarget.backupID the barman-cloud plugin chooses the base backup itself: the newest one in the catalog when no recoveryTime is set, and the newest one ending by recoveryTime otherwise, on any timeline since no targetTLI is passed. A RestoreJob naming an older Backup therefore started from another one, and on an archive holding sibling timelines - left by in-place restores that archived back under the source serverName - from another branch, bringing back that branch's data. The restore reads barman's backupId and the backup's stoppedAt from the live cnpg.io/Backup behind the Backup, and passes the id through the chart as bootstrap.recovery.recoveryTarget.backupID. The plugin gives backupID precedence over time-based selection, and CNPG does not count it among the mutually exclusive targets, so it combines with targetTime. Two cases keep the plugin's choice. A recoveryTime earlier than one second past stoppedAt: the named backup cannot serve an instant before it is consistent, and stoppedAt is truncated to the second while barman compares its end time to the microsecond. And a cnpg.io/Backup that is gone: plugin-barman-cloud's retention deletes it together with the base backup it describes, and the driver deletes one only with its artifact, so pinning its id would purge the target and then fail the recovery on every retry. That restore records a BaseBackupGone warning event instead. A re-restore into the same target clears a backupID left by a previous one. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
c6a4933cf6 |
test(migration): cover the recovery settle and the handoff guards
Recovering outputs that no status recorded goes through fulfill's vm == nil branch, which settles the reclaim policy of volumes an older controller left on Retain. Nothing drove that branch, so the write path could be removed with the suite still green. A fulfill test now runs it with the handoff open and finished. The guards that stop the controller from rewriting the policy of a volume it did not produce, or before CDI controls the claim, get one case each: a claim or VMDisk marked by another import, a non-controller DataVolume owner reference, and a claimRef naming the replacement claim under a stale UID. The storageclasses RBAC comment now also names the reclaim policy read. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
06400ac033 |
fix(kubernetes): probe Keycloak CRDs by output in oidc-bootstrap cleanup
`kubectl api-resources --api-group=<absent group>` exits 0 and prints an empty list, so guarding on the exit code always fell through. The delete then failed with `the server doesn't have a resource type "keycloakclient"` — `--ignore-not-found` covers a missing object, not an unknown type. Because the oidc-bootstrap Job is a post-install/post-upgrade Helm hook, that failure blocked the Helm upgrade of every `kubernetes-<cluster>` release on any cluster without the EDP Keycloak operator CRDs, which is the default configuration (`oidc.mode: None`). Capture the discovery output once and gate each delete on its own resource type, matching the probe idiom already used by the platform migrations and the ouroboros cleanup hook. Fixes #3516 Signed-off-by: Mattia Eleuteri <mattia@hidora.io> Assisted-by: LLM |
||
|
|
6555ae16b8 |
fix(mongodb): start a backup only once the operator has pushed the storage into pbm
The wait after a storage injection watched the cluster's spec, which is not what pbm writes to. At the pinned operator version the backup reconciler hands pbm only a storage name and records status.s3 from the spec entry it read, while the storage pbm uses is whatever the cluster reconciler last pushed into pbm's own config; that push flips PBMReady False and back to True and the reconcile stamps observedGeneration when it ends. A backup started between the apply and that push dumped to the previous coordinates while its Backup recorded the new ones, so both its restore and its prune pointed at the wrong bucket. The wait was also no wait at all: the status write that named it re-queued the job at once, and the live read always showed the apply. The mint now waits for observedGeneration to reach the cluster's generation and for PBMReady to be True, and fails at the backup deadline if the operator never gets there. The strategy, app and cluster reads still ran ahead of the operator CR lookup and were bounded by the backup deadline, so a strategy absent for one reconcile 40 minutes into a streaming dump failed the job and left the finalized CR unreachable. The CR is resolved first now; with one in hand the cluster is not read at all, and the reads the artifact needs are bounded by the running ceiling, past which the job fails naming the CR left in place. The settle check treated a field the entry lacked as agreement, so a strategy that newly set forcePathStyle minted in the same pass as the apply that changed addressing. A missing field now counts as its zero value, which keeps an entry the operator rewrote without an empty region settled while catching a value the apply adds; the storage type is compared outright. The runbook still said the finalizer keeps a Plan from accumulating objects, the RBAC comment still said the timeout cancel runs on either flow, and four wait messages read a duration as a delay. All corrected; the runbook also describes the pbm wait. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
841933dcec |
docs(mongodb): stop saying a Plan prunes the shared bucket
A Plan creates BackupJobs and never deletes a Backup, and PlanSpec has no retention field, so nothing on the platform bounds a MongoDB archive on the shared bucket today except an S3 lifecycle policy or a manual Backup delete. The runbook's tuning section said a Plan does, two hundred lines after the same page said a Plan carries no retention, and three driver comments named "Plan retention" as the path that reclaims the bucket. An operator reading the first sentence would skip the lifecycle policy the release note calls a prerequisite. All of them now say what deletes a Backup today, and that the finalizer frees the object whenever one is deleted, whatever does the deleting. The settle check after an apply compared only the fields the desired entry carries, so an entry written with forcePathStyle while the strategy still set it counted as settled once the strategy dropped the field, and the apply that removed it minted in the same pass. The check now treats a field the apply is about to remove as a change too. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
707e38d17d |
fix(mongodb): keep every mint-time hold off a job that already has its CR
The injection and the ownership switch were gated on the job having no operator CR yet; the precondition between them was not, and neither was the pair of holds ahead of them. Each of those fails the job at the 30-minute deadline, so a tenant disabling backups, or the platform dropping the strategy's s3 block, while a system-bucket dump streamed failed the BackupJob at 30 minutes with the CR left finalized and no Backup object to reach it, bypassing the 24-hour ceiling that exists so a shared-bucket archive is never stranded. Everything up to the mint now runs only while there is no CR; a job with one goes straight to the state switch. The settle wait after the apply was keyed on the storage key being present, so a change to the strategy coordinates rewrote an entry the operator had cached and minted in the same pass, the case the wait exists for. It is now keyed on the entry's content: only an apply that changes what the entry says costs a poll. The mint-time reads (the render's tasks and pitr, the storage entry) came from the manager's cache, which can lag a re-render back to the legacy flow by one reconcile and show the flag true with the tasks not yet visible. With a dynamic client they now come from the apiserver, the way the deadline cancel already reads the operator CR. Tests age the existing-CR case past the deadline and vary the three holds, pin the coordinate-change wait, the live read and the term that keeps the storage-race retry from swallowing an unrelated operator error. The runbook's tuning section no longer summarises the defaults as 30-day retention: only the CNPG and Velero strategies carry one. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
46a8bd7363 |
fix(mongodb): apply the storage once per job, and pin the terms that decide ownership
The system-bucket storage was applied on every reconcile of a job, not once per job: with the operator CR already minted nothing returned before the apply, so a streaming dump saw an SSA patch every poll for up to a day, and the only thing keeping that patch off a cluster that had re-acquired the chart's tasks and pitr since the mint was a predicate term no test held. The storage a CR names is fixed for its dump, so the apply now runs only before the CR exists; the legacy-render hold covers that window, and a test pins that an existing CR is never injected over. The other predicate that decides whether a Backup owns an archive compared the live entry's bucket and credentialsSecret against the strategy, and only the bucket half had a fixture. The credential is what tells a tenant's entry from the platform's when a bucket name collides, so a case varying only that term joins the foreign-bucket one. The cleanup shortcut's snapshot-kind term and the 24h running ceiling are pinned the same way, the latter as an absolute boundary pair. The generated field description said the schema rejects useSystemBucket without backup.enabled. The aggregated apiserver does not evaluate CEL on application schemas yet, so the render-time refusal is the one that holds; the description and the runbook now say so. The runbook's restore paragraph still described the unconditional credential swap this branch narrowed, exempted MariaDB from a retention warning its shipped strategy does not earn, and pointed at a Plan retention field that does not exist; the example README and a dispatcher test's comment still carried the pre-branch claim that the driver never injects and never deletes. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
a6bb0556fc |
fix(mongodb): own archives by where the storage points, and bound the open loops
The prune finalizer on a driver-minted PerconaServerMongoDBBackup means the platform owns the archive: cleanup deletes the operator CR on that marker and the operator removes the object from storage. It was stamped from the app's useSystemBucket flag, which says nothing about where the storage the CR names actually points. With a strategy carrying no s3 block nothing was injected, the presence-only precondition passed on a legacy tenant-bucket entry, and the CR went out finalized against the tenant's own bucket; deleting the Cozystack Backup then deleted the tenant's archive. The finalizer now follows the storage's coordinates as read back from the live cluster after the injection, and a strategy without s3 coordinates, or a storage pointing anywhere else, holds the job instead of minting. The injection keyed on the same desired value. The app CR is projected from the HelmRelease's spec.values, so the flag reads true before helm-controller renders the revision, while the tenant's own storage, scheduled tasks and PITR stream stay on the cluster. Injecting over that entry redirected the tenant's nightly dump and oplog into the shared bucket under platform credentials. A cluster still carrying tasks or pitr is now left alone and the job held until the release renders the flag, and the schema carries a rule marking useSystemBucket without backup.enabled invalid, the pair the chart already refuses at render. The storage-race retry counted surviving operator CRs against its budget, but an errored CR is reaped the moment it is deleted (the operator drops delete-backup without pruning on a non-ready CR), so the count never left one and the job minted a CR per poll for the whole deadline. The count now lives on the BackupJob and survives the reap. A system-bucket dump in running had no bound at all: a wedged pbm agent left the job Running forever and a nightly Plan stacked another beside it. It now fails after 24 hours with the operator CR left in place as the handle to whatever pbm still writes. The deadline cancel pins its delete to the resourceVersion the live read answered for, and the artifact-retry failure names the window that was actually in force. Tests pin the guards that were exercised only in the harmless direction: a system-bucket snapshot reaching the prune, the resolved prefix reaching the cluster, a missing live client deferring the cancel, and the finalizer refused against foreign coordinates. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
5d02c94e71 |
chore(hetzner-robotlb): update robotlb to 0.1.0
The upstream project moved from the intreecom GitHub organization to treetscom, so the chart and image are now pulled from the new location. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
5e32ef35b0 |
feat(build): build Talos, Harbor, the Velero KubeVirt plugin and the unlisted images for arm64 (#4552)
## What this PR does This brings together four approved PRs that give the remaining first-party images an arm64 build. Each of them lost its approval when #4498 merged and GitHub retargeted it to main, so I merged them into one branch to review and approve once. Each original PR has the full description and review history, and I'll close them in favour of this one. - #4507 builds the Talos installer and matchbox for arm64. Releases also publish the arm64 ISO, disk images, kernel and initramfs. - #4515 builds the KubeVirt Velero plugin in-tree for amd64 and arm64. - #4525 adds flux-plunger, keycloak-operator, kilo and migration-controller to the root `build:` list, so CI rebuilds them like every other image. A test keeps any new image target from being left out. - #4528 rebuilds the Harbor v2.15.1 images for amd64 and arm64, because upstream publishes v2.15 for amd64 only. Release builds still run with `PLATFORM=linux/amd64`, except the Talos installer, so these images ship arm64 only once #4505 adds the arm64 leg. This PR makes that leg buildable. A few commits are new on top of the originals. The velero test no longer pins the plugin's image string, because the release build stamps its own rebuild over it. It now checks the plugin init container by repository name. A promote-retag test comment now says what it guards, and the image-refs doc no longer calls its fixed gaps open. The merge with main keeps the Harbor portal in the list of stages whose output has no architecture. ### Screenshots Not a UI change. ### Downstream repositories I left every box empty and opened nothing in other repositories. Two repositories are touched, and neither follow-up is needed before this merges: - cozystack/website names the amd64 assets in `content/en/docs/next/install/talos/iso.md` and `install/providers/oracle-cloud.md`. The install pages could offer the arm64 downloads once a release carries them. No asset is renamed, so the current links keep working. - cozystack/talm points at `ghcr.io/cozystack/cozystack/talos:<version>` in `charts/cozystack/values.yaml`. Once a release publishes that tag as a two-arch index, the same reference works on arm64 nodes, so nothing there has to change. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(talos): the Talos installer image is built for amd64 and arm64, with arm64 profiles that carry drbd and zfs, and the bootbox matchbox image network-boots arm64 machines as well as amd64 ones. Releases publish the arm64 Talos ISO, disk images, kernel and initramfs. feat(velero): the KubeVirt Velero plugin is now built in-tree for amd64 and arm64, so the velero server starts on arm64 clusters. build: flux-plunger, keycloak-operator, kilo and migration-controller are now rebuilt by CI like every other first-party image, so their pinned images pick up source patches and base-image updates and are published for amd64 and arm64. feat(harbor): the Harbor application now runs on arm64 nodes. Its component images are rebuilt from the pinned upstream v2.15.1 release for amd64 and arm64, because upstream publishes v2.15 images for amd64 only. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Talos images and boot assets now support both AMD64 and ARM64, including architecture-specific installation and cloud images. * Added build support for Harbor components and the KubeVirt Velero plugin. * Nightly builds now include ARM64 release assets. * **Improvements** * Harbor uses pinned chart and image versions, with built image references recorded automatically. * Added checks to catch image packages missing from the main build and to verify multi-architecture installer output. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
689b7eefb9 |
feat(system): update ingress-nginx, cozy-proxy, cozystack-scheduler and keycloak-kms-proxy to multi-arch releases (#4549)
## What this PR does Four components move to their first releases built for both linux/amd64 and linux/arm64. The current pins are amd64 only, so these pods cannot start on an arm64 node. - ingress-nginx: the cozystack controller and protobuf-exporter rebuild goes to v1.11.6, and the vendored chart moves to the matching 4.11.6. Upstream 1.11.6 is a maintenance release (Go, dependencies, NGINX base). The template and Lua code the rebuild patches are unchanged since 1.11.5. - cozy-proxy v0.4.0 - cozystack-scheduler v0.4.0 - keycloak-kms-proxy 0.2.3 The charts were re-vendored with `make update`. Every digest resolves to an index that has both platforms. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(system): ingress-nginx v1.11.6, cozy-proxy v0.4.0, cozystack-scheduler v0.4.0 and keycloak-kms-proxy 0.2.3 now ship linux/arm64 images. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Updates** - Updated Cozy Proxy and Cozystack Scheduler to version 0.4.0. - Updated Ingress-NGINX to 1.11.6, including its certificate-generation component. - Updated Keycloak’s KMS proxy to version 0.2.3. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
88b61c96de |
docs(kubevirt): state the preconditions of the eviction comments (#4441)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does This fixes two comments in the KubeVirt CR that #4403 got wrong. It follows up on the [review of #4403](https://github.com/cozystack/cozystack/pull/4403#pullrequestreview-5301013108), and I checked both findings against the source first. Only comments change, the rendered CR is the same. The comments didn't say that the new eviction behaviour applies only to VMIs created after the switch. The VMI mutator copies the cluster `evictionStrategy` into the VMI spec at creation (KubeVirt v1.9.0 `pkg/defaults/defaults.go:208-211`), and the eviction admitter reads the spec first. So a non-migratable VMI created before the switch still has `LiveMigrate` and keeps blocking the drain. The workload updater can't evict it either, and it stays on the old launcher until its VM restarts. The worker comment described the wrong mechanism. When a VMI can't migrate and has `evacuationNodeName` set, CAPK's `IsTerminal()` returns true and the controller sets `FailureReason` on the KubevirtMachine. This is upstream CAPK code at the pinned commit, the cozystack patch doesn't touch it. Every kubernetes-nodes group has a MachineHealthCheck, and CAPI v1.10.1 treats `FailureReason` as unhealthy, so the Machine gets replaced. If `maxUnhealthy` holds remediation back (at the default `50%`, a group of one), the VM starts a new VMI in place. CAPK never clears `FailureReason` though, so the Machine stays failed until remediation is allowed. The comment stays on the `evictionStrategy:` line, because `make update` inserts blocks right before it. `helm unittest` passes 33/33 and `update_idempotency_test.sh` passes. `make update` leaves the file byte-identical. The third finding (tests pin values, not behaviour) is tracked in #4422. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note docs(kubevirt): the KubeVirt CR comments now say that a VMI created before the evictionStrategy change keeps LiveMigrate until its VM restarts, and that the MachineHealthCheck replaces a non-migratable kubernetes-nodes worker when the workload updater evicts it. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified that existing VM instances retain the eviction strategy they were created with, and may need to be restarted before they can migrate during a drain. * Expanded guidance on how eviction of non-migratable workers affects machine health and replacement, including when remediation is delayed by the `maxUnhealthy` limit. * No rendered configuration values changed. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8d547be7fe |
fix(tests): reject unreadable RD preset schemas (#4191)
## What this PR does `rd-presets-check` discards `jq` errors, so malformed schema can pass the check or appear as missing presets. It now fails with the schema path and `jq` diagnostic. An absent schema or a readable schema without `resourcesPreset` is still skipped. Regression tests use real `jq` for malformed input and partial output. They also cover the canonical preset set, exact matching and the long-list case fixed by [#4204](https://github.com/cozystack/cozystack/pull/4204). ### Testing - `make unit-tests -j4 --output-sync=target` passed. - Nine cases in `hack/check-rd-presets_test.bats`, run with `bats` and `dash hack/cozytest.sh`. - Both read-failure cases fail against `main`. Eleven deliberate code mutations also fail the expected test. - `shellcheck hack/check-rd-presets.sh hack/check-rd-presets_test.bats`. ### Screenshots Not applicable. ### Downstream repositories - [x] No downstream repository is affected by this change No files or make targets were moved or renamed. The change is limited to the RD checker and its tests. ### Release note ```release-note fix(tests): Fail the RD preset check when a schema cannot be read, with the schema path and jq diagnostic. ``` |
||
|
|
cb035f8eac |
docs(agents): stop calling the image-ref known gaps unfixed
Two of the gaps listed there are handled in the tree now, and each entry already says whether it is. The heading sentence still claimed all of them were open. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
029707f0b2 |
test(hack): say what the promote-retag ownership check guards
The comment above the ownership loop described how both images used to be dropped. Describe the reason the check matters instead: neither image has a release tag to fall back on, so a miss would go unnoticed. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
d190dcec74 |
test(velero): check the KubeVirt plugin init container by repository
The release build stamps its own rebuild of the plugin over the upstream ref, so the test matches the repository name alone. It still fails when the init container is dropped or points at another image. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
02461acdde |
Merge remote-tracking branch 'origin/main' into feat/arm64-bundle
Signed-off-by: Aleksei Sviridkin <f@lex.la> # Conflicts: # hack/image-builders-cross-compile.bats |
||
|
|
0040d6aa3d |
feat(keycloak): update keycloak-kms-proxy to 0.2.3 with arm64 images
0.2.3 publishes the proxy as a linux/amd64 and linux/arm64 index; earlier releases were amd64 only. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
a0d3dba07f |
feat(cozystack-scheduler): update to v0.4.0 with arm64 images
v0.4.0 publishes the scheduler as a linux/amd64 and linux/arm64 index; earlier releases were amd64 only. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c22cc2851e |
feat(cozy-proxy): update to v0.4.0 with arm64 images
v0.4.0 publishes cozy-proxy as a linux/amd64 and linux/arm64 index; earlier releases were amd64 only. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
75d7573942 |
feat(ingress-nginx): update controller to v1.11.6 with arm64 images
The cozystack controller and protobuf-exporter rebuilds are now published for linux/amd64 and linux/arm64, so the ingress controller can start on arm64 nodes. The rebuild is tagged by its upstream version, and v1.11.6 is the first multi-arch one, so the chart moves to the matching 4.11.6. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
b2ae6b773a |
fix(tenant): fail fast on an empty tenant apex and correct gateway docs (#4538)
Closes #4536 ## What this PR does Two fixes to how tenant charts handle the tenant apex host (`_namespace.host`), from an audit of what happens when a tenant's `host` is changed or left empty. **1. Fail fast on an empty tenant apex (was: silent malformed hostname).** Charts that publish a service compose their hostname as `<sub>.<_namespace.host>` (for example `harbor.<apex>`, `s3.<apex>`, `grafana.<apex>`). When the injected apex was empty and no per-app `host` override was set, they emitted a malformed trailing-dot hostname (`harbor.`) into Ingresses, HTTPRoutes and Certificate dnsNames with no error. Only the gateway path guarded against this. A new library helper `cozy-lib.ns-host-required` now sources the apex at each hostname-emitting site and fails the render with a clear message when the apex is empty and no override is set. It is scoped to the injected state, so standalone renders (`helm template` / `helm lint` with no `_namespace`, or an empty `_namespace` map), no-exposure tenants, and conditionally emitted resources (external=false, the kubernetes no-etcd beacon path) are unaffected. The two post-delete cleanup hooks are left unguarded so an empty apex cannot block uninstall. Charts covered: harbor, postgres (external), qdrant (external), kubernetes (control-plane endpoint), vpn, seaweedfs, bootbox, monitoring (grafana, alerta, grafana OIDC client), and system/bucket (which also gains the `cozy-lib` library it was missing, both the local subchart symlink and its package-source registration). **2. Correct the tenant `gateway` parameter documentation.** The `gateway` description (values.yaml, values.schema.json, README, the Go type, the tenant-rd CRD, and an inline comment in namespace.yaml) claimed the chart auto-enables a Gateway for tenants whose apex is derived from the parent (empty `host`) and leaves it off for custom-apex tenants. The resolver `tenant.gatewayEffective` never inspects `host`: an unset `gateway` key always means the tenant owns no Gateway and inherits from the nearest ancestor that owns one. The unit tests already pin this behavior. The description is rewritten to match and regenerated. **Behavior change.** A tenant with an empty injected apex and no `host` override that enables one of the covered components now fails its render (the HelmRelease reports NotReady) instead of deploying a broken hostname. In practice the apex is always injected (the root tenant has `root-host`, children inherit it), so this only trips a genuinely misconfigured tenant. Tests: a new `cozy-lib.ns-host-required` unit test, plus per-chart guard tests for harbor, qdrant, kubernetes, postgres, vpn, bootbox, seaweedfs, monitoring and system/bucket. All affected chart suites pass. ### Screenshots Not applicable (no UI change). ### Downstream repositories I walked the trigger map against the diff. The only touchpoint is the tenant `README.md` gateway description, which the website release bot regenerates automatically from `README.md` on a stable tag (no manual follow-up). The terraform provider is not affected: this is a description-only change to the existing `gateway` field, with no field, default or enum change, and the tenant-rd CRD schema reaches the provider as opaque JSON. Boxes left for a maintainer to confirm. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(tenant): tenant charts now fail rendering with a clear error when a tenant's apex host is empty and no per-app host override is set, instead of silently emitting a malformed hostname; the tenant `gateway` parameter description is corrected to match actual behavior (an unset `gateway` never auto-enables a Gateway). ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Host-dependent services now show a clear rendering error when neither a tenant host nor an app host is configured, instead of producing an invalid hostname. * Tenants without `gateway: true` use the nearest ancestor’s Gateway, whether their host is inherited or custom. Enable `gateway: true` for a tenant’s own Gateway, IP, and certificate; this is required to make a custom apex publicly reachable. * **Documentation** * Clarified Gateway inheritance behavior and when a tenant needs its own Gateway. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
3d2dfb1ed4 |
fix(build): let a builder stage inherit its base stage's platform (#4545)
## What this PR does The check that builder stages compile natively is red on main, and this is a false positive. The shadowbox Dockerfile builds its webpack stage as `FROM source AS app`, where `source` is a `--platform=$BUILDPLATFORM` stage. The check reads only the literal FROM line, so it reports `app` as emulated. BuildKit gives a stage the platform of the stage it starts from, so `app` already runs natively. The check now tracks which named stages are native and carries that to any stage built from one of them. A stage built from an emulated stage stays emulated. The arch-neutral list now also takes a `<file>:<stage>` entry. With it only the shadowbox JS stage is excused, and its Go stage still has to use TARGETARCH. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` |
||
|
|
f301cb551b |
feat(api): add configurable resources and replicas for cozystack-engine components (#2956)
## What this PR does Exposes standard Helm `resources` values for the three core workloads in the `cozystack-engine` package, and makes `cozystack-controller` replicas configurable (was hardcoded to `1`). **Changes by component:** | Component | `resources` | `replicas` | |---|---|---| | `cozystack-api` | ✅ NEW (`cozystackAPI.resources`) | already configurable | | `cozystack-controller` | ✅ NEW (`cozystackController.resources`) | ✅ NEW (`cozystackController.replicas`, default `1`) | | `lineage-controller-webhook` | ✅ NEW (`lineageControllerWebhook.resources`) | already configurable | Each template uses `{{- with .Values.xxx.resources }}` so the `resources:` block is only rendered when values are provided. Default `resources: {}` produces no output, so resource requests/limits stay opt-in. **Leader election:** `cozystack-controller` now always runs with leader election — the chart passes `--leader-elect` unconditionally and ships a namespaced `Role`/`RoleBinding` for the Lease in `cozy-system` (where controller-runtime creates it). Raising `cozystackController.replicas` therefore adds standbys instead of a second active writer over the reconcilers that patch HelmReleases, Gateways, Certificates, ResourceQuotas and Secrets; this holds whatever `replicas` is. A restart can pause reconciliation for up to the Lease duration (15s by default), because `LeaderElectionReleaseOnCancel` is off. With `replicas: 1` the change for an existing install is the flag plus the two RBAC objects. **Why:** Without these values, operators cannot set QoS guarantees and must resort to `kubectl patch`, Flux Kustomize SSA patches, or Kyverno mutations — all fragile under Helm upgrades. Closes #2955 ### Release note ```release-note feat(api): add configurable `resources` (requests/limits) for cozystack-api, cozystack-controller, and lineage-controller-webhook. `cozystack-controller` replicas is now configurable via `cozystackController.replicas` (default: 1). The `cozystack-controller` Deployment now always runs with leader election and creates a Lease in `cozy-system`, so raising `replicas` adds standbys rather than a second active controller; a restart can pause reconciliation for up to the Lease duration (15s by default). An empty `resources: {}` renders no resources block, so requests and limits remain opt-in. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Container resource requests and limits can now be configured for the API, controller, and webhook services; defaults remain unchanged. * The controller replica count is configurable, and leader election is enabled for multi-replica operation. * Controller leader-election access is limited to the `cozy-system` namespace. * **Documentation** * Documented the webhook resource configuration option. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
af8cf743cd |
fix(backups): hold a BackupJob while its application is being restored
Nothing on the backup path looks at restores. A Plan creates its BackupJobs on schedule, and the BackupJob reconciler dispatches them to their driver whatever else is happening to the application. For CNPG that is not merely a backup of a half-restored state: every BackupJob reconcile re-applies the Cluster's barman plugin, and on a cluster under an in-place restore it can land after the chart has re-rendered the Cluster with its new serverName and put it back on the source's. The restored cluster then archives onto the source's WAL prefix and its recovery fails with "WAL archive check failed ... Expected empty archive". This was reproduced with a backup triggered a few seconds after an in-place restore, and an hourly plan overlaps a restore of up to thirty minutes about half the time. A BackupJob that has not started yet now waits while a non-terminal RestoreJob in its namespace writes into the same application, resolved as the drivers resolve it: the Backup's applicationRef, with the target's kind and name when a targetApplicationRef is set, and the target's API group only when it names one. A RestoreJob whose Backup is gone holds nothing, since the restore controller fails it on its next pass. options.targetNamespace is honoured only for a Backup taken by the Velero strategy, the one driver that reads it; the CNPG and Job-based drivers ignore the option and restore in place, so their restores hold whatever it says. The held job stays Pending with a Ready=False RestoreInProgress condition naming the restore and is requeued, rather than failed, so a scheduled run still happens once the restore ends. A failed write of that status is returned, so the manager retries it. When the restore ends the condition is removed before the job starts, since it would otherwise read "waiting" until the driver next writes Ready, which for a healthy run is when the backup completes. A run that has already started is left alone: its driver owns state the gate would strand. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
|
4c945dbc40 |
fix(build): let a builder stage inherit its base stage's platform
The cross-compile check read only the literal FROM line, so a stage built FROM an earlier --platform=$BUILDPLATFORM stage counted as emulated even though it runs natively. The shadowbox webpack stage is such a stage, and it failed the check on main. The check now follows FROM <stage> to its base and inherits its platform. ARCH_NEUTRAL also accepts "<file>:<stage>", so the webpack stage can be marked as producing no architecture without excusing the Go stage in the same Dockerfile. A lookup failure now fails the stage instead of passing it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |