mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-10 06:24:09 +09:00
3e4295b4a659fe1186c0ff079d21cd7af89148ee
7551
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 myasnikovdaniil
|
3e4295b4a6 |
[Backport release-1.7] fix(backups): gate every default Strategy CR on its CRD (#4935)
# Description Backport of #4929 to `release-1.7`. |
||
|
myasnikovdaniil
|
587ec5913f |
[Backport release-1.7] fix(fluxcd): keep Helm uninstall working during flux-plunger recovery (#4995)
# Description Backport of #4293 to `release-1.7`. |
||
|
Alexey Artamonov
|
021040795b |
fix(fluxcd): keep Helm uninstall working during flux-plunger recovery
flux-plunger recovers a "has no deployed releases" HelmRelease by
briefly suspending it, deleting the stale storage Secret and
unsuspending. helm-controller v1.5.x skips the uninstall and drops its
finalizer when it handles a delete on a suspended HelmRelease, and it
wakes on the same delete event flux-plunger does, so a delete inside
that window orphaned the release's DataVolume, PVCs, RBAC and storage
Secret. Lifting the suspension on the delete event cannot win that race
reliably.
Hold the release with a flux-plunger finalizer while it is suspended.
The finalizer is applied together with the suspension in one server-side
apply. On a delete flux-plunger lifts its suspension and keeps the
finalizer, helm-controller reconciles the unsuspended generation and
runs the uninstall, and flux-plunger removes the finalizer once
helm-controller's own is gone and that generation is observed. This
holds whichever controller handles the delete first. A completed
recovery lifts the suspension and the finalizer together, and a
terminating release never begins recovery.
Decide ownership of a suspension by which field manager owns
spec.suspend in metadata.managedFields rather than by an annotation. The
apiserver transfers the field away the moment an operator, cozyhr resume
or a backup controller writes a different value to it, so a later
suspension is never misread as flux-plunger's, and one set through the
shared flux-client-side-apply manager (platform migrations, the tenant
delete hook, a manual kubectl apply) is never claimed. A release deleted
under another manager's suspension is left to that manager's decision.
Acquire applies without forcing first and, on a conflict, re-fetches and
forces only if spec.suspend is still false: a backup controller owns the
field at false through a dynamic-client Update, which a non-forced apply
can never take, while true is an active suspension to leave alone. Every
apply is preconditioned on the resourceVersion and uid of the read that
decided on it. Without the resourceVersion, the unforced apply of true
over a foreign true is accepted as shared ownership, a window the
optimistic-locked merge patch this replaces had closed, and lifting the
suspension and the finalizer from a release read as live would delete it
without an uninstall if helm-controller had meanwhile handled a delete.
Without the uid, an apply to a release that has gone would create an
empty one. The apply carries only spec.suspend and the finalizer, so it
cannot leak the required spec.interval.
Read the release, every re-check and the storage Secret list from the
apiserver rather than the informer cache. A lagging cache would let the
processed-version guard delete the revision below the one just
reclaimed, and the ClusterRole grants secrets get and list but not the
watch an informer needs.
Emit a Warning when flux-plunger holds a suspension it failed to clear,
and Normal events when it leaves an external suspension alone or defers
a recovery. An external suspension on a release flux-plunger has
recovered before is a Warning instead: it is most likely one an earlier
flux-plunger left under the shared manager, which nothing lifts unless
an operator does. A release deleted under another manager's suspension
is a Normal event when that was a migration's choice, and a Warning when
the release carries flux-plunger's processed-version annotation, since
then the skipped uninstall is an accident.
Keep a single writer with leader election. A second process able to act
under the same field manager could clear a suspension the first is still
recovering, and no rollout strategy prevents one: the next pod starts as
soon as the old one is terminating, and an eviction or a node drain
replaces a pod the same way. The old pod keeps its Lease, unreleased,
through its stop and drain, and the Lease outlasts both, so the next pod
acts only after the old one is done; the price is that a new pod waits
for the Lease to run out. The strategy sets maxSurge 0 with
maxUnavailable 1 and stays RollingUpdate because existing installs carry
the apiserver-defaulted rollingUpdate block and helm-controller applies
server-side, where type Recreate over it is rejected on upgrade.
Give the finalizer a way out when flux-plunger itself goes. Once the
manager has stopped, within a bounded time, and with it every reconcile,
flux-plunger drains its finalizers under a deadline that, with the stop,
fits inside the pod's termination grace period. A stop that overran its
bound is drained too and still reported, and only a pod that led drains.
The drain writes its events straight to the apiserver, since the
manager's recorder stops with the manager. A recovery in progress is
aborted, the suspension and the finalizer lifted together for the next
flux-plunger to start again, and a release being deleted has the
suspension lifted and keeps the finalizer until helm-controller has run
the uninstall. A release helm-controller does not handle in time keeps
it. The waiting lives in the controller, where the envtest suites hold
it against a real apiserver's generations and finalizers, rather than in
a shell script against a model of them.
A pre-delete hook on the chart is the backstop for what the drain could
not finish, so uninstalling the component never leaves a release that
tenant teardown waits on forever. It stops flux-plunger, waits out its
grace period, then lifts any remaining suspension and finalizer without
waiting again, naming each release it lets go of before helm-controller
uninstalled it. It lifts a suspension only where flux-plunger alone
holds it, with a JSON patch preconditioned on the resourceVersion and
uid it read: kubectl apply --server-side on a release last written by a
client-side apply moves that apply's fields to the applying manager and
strips every one the body leaves out. The Job's deadline sits under the
300s helm-controller gives a pre-delete hook by default. A Deployment
that cannot be stopped or a listing that never succeeds ends the hook
before any finalizer is touched, and a finalizer it cannot remove ends
it after the sweep; either way it exits non-zero: a failed pre-delete
hook leaves the release untouched and helm-controller retries the
uninstall, hook included, from a state it can still converge from, where
exiting 0 would remove the component and strand the finalizer. A
flux-plunger pod still present after its grace period, typically one
stuck Terminating on a NotReady node, is only reported, since failing on
it would fail every retry until the node returns; because it may still
be setting finalizers, they are listed again after the sweep and any
found fails the hook. The Job runs once, since helm-controller's retry
is the one that starts afresh, and is kept after success so that its
report stays readable; helm.sh/hook-output-log-policy would make
helm-controller call a nil output writer (fluxcd/helm-controller#1562).
A delete still held after ten minutes raises a Warning, and the held
state is re-checked on a timer instead of only on helm-controller's next
status write. The chart README documents the finalizer and how to clear
it.
Run the envtest suites on every pull request through a new make target
that fetches a kube-apiserver and etcd with a pinned setup-envtest and
takes the HelmRelease CRD from the in-tree Flux manifests: the unit
tests drive a fake client, and everything the finalizer rests on (apply
preconditions, the finalizer set-merge, real managedFields) only holds
against a real apiserver. CI caches those binaries, and the target uses
them without asking the release index, so a pull request does not depend
on their sources being reachable; the control plane gets a longer start
timeout because it starts beside the other test goals.
Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
dee1eef559 |
test(backups): fail CI on any default Strategy CR rendered without its CRD gate
The CRD gate has now been missed twice: MongoDB shipped without it, and
Kafka, RabbitMQ and Redis followed after it existed. helm-unittest only
checks the templates a case lists, so a new strategy-*-default.yaml
that forgets the gate stays green until an upgrade wedges.
Render the chart with the bucket gate open and no CRD served, and fail
on any strategy.backups.cozystack.io object from any template. Also
check that every rendered Strategy CR gates on the CRD this chart ships
for its kind, because a gate naming the wrong CRD passes the first
check and keeps the CR out of every live render.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
a29fd4814c |
fix(backups): gate every default Strategy CR on its CRD
The chart ships its strategy CRDs as ordinary templates, and Helm maps
every document of a release through the RESTMapper before it applies
any. A default Strategy CR gated only on the bucket name therefore
fails Build() with "no matches for kind" whenever the bucket name
resolves before the CRD is served, the release applies nothing, the
CRDs are never created, and every retry renders the same manifest.
That happens on the upgrade that first ships a kind: the Kafka,
Rabbitmq and Redis CRs leave a v1.6 cluster's HelmRelease stuck in
UpgradeFailed on the way to v1.7. It also happens on the first render
of an external-S3 install (provisionBucket=false), where the bucket
name comes from values and every older default CR renders next to its
absent CRD.
MongoDB already gates on crdEstablished for this reason. Give every
default Strategy the same gate. On an installed cluster the CRDs are
established and the render is unchanged; otherwise the first revision
creates the CRDs and skips the CRs, and DefaultObjectsGate, which reads
the cozy-default BackupClass routes, forces the follow-up upgrade that
renders them once the kinds are served.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Aleksei Sviridkin
|
38d6ac7efb |
feat(redis): expose min-replicas-to-write and min-replicas-max-lag (#4848)
## What this PR does Adds two optional values to the Redis app, `minReplicasToWrite` and `minReplicasMaxLag`, rendered as `min-replicas-to-write` and `min-replicas-max-lag` in the RedisFailover `spec.redis.customConfig`. Both are unset by default, so the rendered config and behaviour are unchanged for existing instances. Replication is asynchronous, so a master that is isolated from its replicas keeps accepting writes until Sentinel fails over, and those writes are lost. With `minReplicasToWrite: 1` the isolated master stops accepting them after `minReplicasMaxLag` seconds, which bounds the loss. Trade-off: with the default `replicas: 2` there is a single replica, so the master also refuses writes whenever that replica is down (node drain, restart, upgrade). Users who turn this on accept lower write availability for a bounded loss window. Includes helm-unittest cases and the regenerated schema, README, Go types and ApplicationDefinition. ### Screenshots ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/68 - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(redis): add optional minReplicasToWrite and minReplicasMaxLag values to bound writes lost on failover; unset by default ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional Redis settings to require a minimum number of replicas to acknowledge writes and to set the maximum age of those acknowledgements. * Invalid settings are rejected: the required replica count must be nonnegative and below the configured replica count, and the acknowledgement age must be at least 1 when a replica requirement is set. Replication remains asynchronous, so these settings do not eliminate potential write loss during failover. * **Documentation** * Documented the settings, defaults, validation, and behavior in Redis configuration guidance. <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.7.0-rc.1 |
||
|
myasnikovdaniil
|
ebceef4105 |
ci(retention): sweep packages named by the published nightly tree (#4826)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The Artifact Retention workflow has failed every day since mid-August, and it never pruned anything. It lists the org's container packages with `GET /orgs/cozystack/packages?package_type=container`, and that endpoint answers `400 Invalid argument` to the app installation token the job uses. Others hit the same in actions/github-script#449 and cli/cli#9606. An old `|| true` used to hide it. This is item A19 in #3278. The per-package endpoints accept the same token: a probe run confirmed it, and the dry-run below read every package with it. So the job no longer lists the org. The new `hack/retention-packages.sh` takes the package names from the published `cozystack-packages:nightly` tree, reading refs through the shared `hack/lib/image-refs.sh`. It also adds `cozy-installer` and `cozystack-nocloud`, which the nightly publishes outside the tree. The committed tree can't be the source: it misses images whose digest only the build pins, and those still get dated nightly tags every day. Failures now turn the run red instead of passing silently: - if the name list is empty, the nightly sweep is skipped and the run fails, but abandoned promotion candidates are still handled, because their protection doesn't depend on that list - if a package's versions can't be read, that package is skipped and the run fails at the end - if a DELETE is refused, `hack/retention-delete.sh` logs an error with the HTTP status, goes on with the remaining versions and fails the run (before, this was only a warning) - if names resolve but none of the packages is linked to this repository, nothing gets swept, so the run fails too, after the promotion sweep has run (that sweep itself is skipped only when candidate protection can't be resolved) Promotion-candidate protection itself is unchanged. The contract test that pinned the old org listing now pins the new behaviour, and both new scripts have bats tests. Known gap: a component that disappears from the nightly tree drops off the list, so its old dated nightlies are never pruned. Today none of the packages outside the list has more dated nightlies than the keep window. Dry-run on this branch is green: https://github.com/cozystack/cozystack/actions/runs/38075159900. It resolves 59 packages, skips `cozystack/cozystack-ui` with the warning, and would delete 418 old nightly versions plus one abandoned promotion candidate. Deleting needs a different token. The app token lists versions, but a one-version apply run (https://github.com/cozystack/cozystack/actions/runs/37939905343) got `404 Package not found` on DELETE for a version it had just listed. The same DELETE with this repository's `GITHUB_TOKEN` and the job's `packages: write` worked in a second run (https://github.com/cozystack/cozystack/actions/runs/37941084456), with no change to package settings. So `hack/retention-delete.sh` now deletes with `GITHUB_TOKEN`, and listing stays on the app token. That delete was shown on a package linked to this repository. One of the swept packages, `cozystack/cozystack-ui`, is linked to another repository, and I didn't try deleting there. Such a package is skipped before its versions are read and named in a warning, so it can't turn every apply run red. The follow-up for it is in #3278. `RETENTION_APPLY` stays off, so the scheduled run is a green dry-run until someone turns it on. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> N/A ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ci(retention): the GHCR retention workflow takes package names from the published nightly tree instead of the org package listing, which rejects the app token, and fails the run whenever names, versions or deletions cannot be resolved. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Nightly package retention now targets packages referenced by the published nightly tree and keeps the 14 newest eligible versions per package. * Promotion candidates older than 30 days are eligible for pruning only when their digests are not protected by qualifying release pull requests or branch tips. * **Bug Fixes** * Packages linked to another repository are skipped, and sweep failures are reported without preventing other eligible packages from being processed. * **Documentation** * Clarified which nightly versions and promotion candidates qualify for retention. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
34488db766 |
ci(e2e): never tear down the Proxmox tenant on the runner's default context (#4807)
## What this PR does The Proxmox e2e teardown could delete from the wrong cluster. It runs under `always()` and got its kubeconfig only from `GITHUB_ENV`, which the "Stage credentials" step writes. If a step before that fails, for example a missing stand variable, `KUBECONFIG` is never set. The three `kubectl delete` calls then go to whatever cluster the self-hosted runner's own kubeconfig selects. Every teardown call now passes the staged file with `--kubeconfig`. Stage fails unless that file's current context resolves to a server, and prints kubectl's own error when it does not. Before, an empty `COZY_KUBECONFIG` secret let client-go fall back to its defaults, in-cluster credentials among them. The suite step gets the staged path in its own `env`. The `GITHUB_ENV` export is gone, so no step in this workflow picks its cluster up implicitly. Tear down decides whether to skip from how Stage ended, not from the file. If Stage did not succeed, the suite never ran and there is no tenant to remove, so the step prints a notice and exits 0. If Stage succeeded and the file no longer resolves, the step fails with kubectl's error, because the tenant and its VMs may still be there. A side effect: a run whose Stage fails no longer cleans up a tenant left by an earlier `keep_tenant` run. The next run that passes Stage without `keep_tenant` removes it. The Stage and Tear down scripts now have unit tests in `hack/proxmox-teardown-contract.bats`, run against a stub `kubectl`. I also ran both against a real kubectl v1.37.1 for every Stage outcome. The files were missing, empty, garbage YAML, without a current context, with a dangling context, without a server, and valid. One more case was a runner without kubectl. The deletes also no longer stop at the first failure. Under `set -e`, a failed first delete skipped the tenant delete, and the tenant's VMs stayed on the hypervisor. Now each delete runs, and the step fails at the end if any of them failed. With a recording `kubectl` that fails one chosen delete, all three ran in every case and the step exited 1. With the change reverted, a failing first delete was the only one that ran. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note ci(e2e): the Proxmox e2e teardown always uses the staged kubeconfig, so it can no longer delete from the runner's default cluster. It skips only when Stage did not succeed and fails when the kubeconfig breaks after Stage. A failed delete no longer skips the tenant delete. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Improved reliability of automated Proxmox test setup and cleanup. Tests now stop setup when Kubernetes configuration is unavailable, skip cleanup when setup fails, and attempt all resource deletions before reporting cleanup failures. * **Tests** * Added checks for configuration validation, cleanup behavior, and handling of failed deletion attempts. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
2db18a0c86 |
test(e2e): record ZFS datasets and LINSTOR volumes in cozyreport (#4657)
## What this PR does The kubernetes suites' LINSTOR cleanup check now compares dataset names per node, and prints the leftover names when it fails (#4651). This PR puts the rest of what a triager needs into `cozyreport.tgz` on every run, green or red: the size of each dataset, its `origin` and `creation`, the snapshots, the pool's pending frees, and the LINSTOR volume list. What `hack/cozyreport.sh` now collects: - `linstor/zfs-pools.txt`: the zpool behind each ZFS storage pool, per node, taken from `StorDriver/StorPoolName` in `linstor --machine-readable sp l`. - `linstor/zfs/<node>.txt`: that node's satellite runs `zpool get -p freeing,leaked,allocated,free` and `zfs list -p -t all -o name,used,avail,refer,origin,creation -r` over the node's pools. - `linstor/volumes.txt`: `linstor v l`, next to the existing `resources.txt`. Sizes are exact bytes, so the sizes of leftover datasets can be summed against the pool root's `avail`. That `avail` can differ from `storage-pools.txt` without anything leaking: on a node with only thick pools, `linstor sp l` serves the free capacity cached at the satellite's last report. A dataset LINSTOR could not destroy because snapshots depend on it is renamed to `_deleted_<name>` in the same pool, so a search for leftovers has to look for that prefix as well as `pvc-`. A pool that `zpool get` or `zfs list` cannot open does not cost the listing of the other pools. `zpool get`'s message and exit status go into the listing, and `zfs list` lists the rest. A node LINSTOR names no ZFS pool for gets a note instead of a listing, because `zfs list -r` with no pool argument lists every pool on the machine. A satellite whose `spec.nodeName` cannot be read gets a note that says so. All reads go through the bounded exec reader, like the other LINSTOR reads, and the time budget in the script header counts them. The second commit changes how that reader marks a read that wrote output and then exited non-zero without hitting the timeout. It used to append `TRUNCATED ... part way through, not because the command finished`. But `kubectl exec` passes the remote command's status through, and `zfs list` over one missing pool prints a complete listing of the others and exits 1. The marker now names the exit status and says the status can't tell an error after complete output from a cut stream. This changes the marker for every LINSTOR exec read. The timeout marker stays as it was. Tested with `hack/cozytest.sh hack/cozyreport.bats`. The new tests run the controller and satellite shells against stub `linstor`, `zpool` and `zfs`, with the real `jq`. They cover a failed and an unparsable storage-pool read, a pool row with a missing, empty or blank backing name, a non-ZFS pool, a pool name with a space, a pool that `zpool` and `zfs` cannot open, a failing `zfs`, and an unreadable node name. A test goes red if any of these break: the filter that keeps non-ZFS pools out, the filter that drops a missing, empty or blank pool name, the jq exit status, the `zpool` fallthrough, the node-name note, or the reader's new wording. Refs #4653 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note test(e2e): cozyreport records the ZFS datasets of each node's LINSTOR pools in exact bytes, and the LINSTOR volume list, on every e2e run. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Collection reports now include LINSTOR volumes, ZFS pool capacity and usage, and per-node ZFS datasets and snapshots with exact-byte details. Dataset listings are scoped to the ZFS pools LINSTOR reports for each node. * Reports include notes or warnings when node, pool, or dataset information is missing or cannot be collected. * **Documentation** * Clarified storage report contents, artifact markers, bounded reads, and how unsupported data, incomplete results, collection limits, and per-object warnings are reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
95c2d85b5b |
fix(etcd): leave compaction to etcd and pass the real quota to defrag (#4873)
## What this PR does The hourly defrag CronJob ran `etcd-defrag` with its defaults, and two of them do not fit the cluster the chart creates. - `--compaction` (default `true`) compacts to the latest revision before every defrag. The EtcdCluster already compacts itself (`autoCompactionMode: periodic`, `autoCompactionRetention: 5m`), so the extra compaction either fails with `mvcc: required revision has been compacted` when nothing was written since the last auto-compaction, which is what every defrag Job log shows on our clusters, or, on a busy cluster, drops the history the retention keeps for watchers to resume from. - `--etcd-storage-quota-bytes` defaults to 2 GiB, so `dbQuotaUsage` in the defrag rule is measured against 2 GiB rather than the `quotaBackendBytes` the chart sets (95% of `size`, about 3.8 GiB for the default `4Gi`). The CronJob now passes `--compaction=false` and the chart's own quota through the existing `calculateQuotaBackendBytes` helper. The new unit tests check both arguments and that the defrag quota equals the EtcdCluster's `quotaBackendBytes` for a non-default size. Without the template change all three fail. The same templates are on `release-1.6` and the change applies there cleanly. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff is two template arguments and a unit test in `packages/extra/etcd`, with no change to values, schema, defaults or package names, so nothing in the trigger map applies. ### Release note ```release-note fix(etcd): the defrag job no longer compacts on top of etcd's periodic auto-compaction, and it evaluates its 80% quota threshold against the cluster's real quotaBackendBytes instead of 2 GiB. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * The etcd defragmentation job now uses the cluster’s configured backend storage quota and skips compaction before defragmenting. This keeps the defragmentation operation aligned with the cluster’s storage settings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Yasar Rahman
|
8be7fb2e66 |
feat(redis): expose min-replicas-to-write and min-replicas-max-lag
Lets an operator bound the writes lost when an isolated Redis master is failed over: the master refuses writes while fewer replicas than configured are connected and recent. Both values are unset by default, so the rendered Redis config is unchanged. Redis treats 0 in either directive as disabling the check, so the chart refuses minReplicasMaxLag 0 next to a replica requirement instead of silently turning the protection off. Negative values are refused by the schema (minimum 0), since Redis rejects them at CONFIG SET and the operator would stop healing the Sentinels of that instance. A requirement of replicas or more is refused at render: a master with replicas pods has replicas-1 replicas, so it would refuse every write for good. Assisted-by: LLM Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> |
||
|
Aleksei Sviridkin
|
33ca8b67bb |
feat(redis): run Sentinel on the same image as the Redis pods (#4831)
## What this PR does
The Redis application pins its Sentinels to `redis:6.2.6-alpine`
(`packages/apps/redis/templates/redisfailover.yaml`), while the data
pods run the version selected by `version` (`redis:8.4.0` for `v8`,
`redis:7.4.7` for `v7`). The pin was added in
|
||
|
Yan Bondarenko
|
fe59a6bf91 |
fix(etcd): leave compaction to etcd and pass the real quota to defrag
The hourly defrag CronJob runs etcd-defrag with its defaults, which compact to the latest revision before defragmenting. Every cluster already compacts itself with a periodic five-minute retention, so the second compaction either fails with "required revision has been compacted" when nothing was written since, or drops the history that retention keeps for watchers to resume from. The same defaults evaluate dbQuotaUsage in the defrag rule against 2 GiB rather than the quotaBackendBytes the cluster runs with, so the 80 percent threshold fires at a different database size than intended. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Aleksei Sviridkin
|
9a348eeb50 |
fix(kubernetes): delete orphaned resources in new post-delete job (#3856)
## What this PR does When tenant deletes the kubernetes cluster, 3 resources are orphaned and leftover in the tenant namespace. - jobs/<name>-oidc-bootstrap - secrets/<name>-talos-ca - secrets/<name>-talos-tls-cert This PR adds a new "step 7" in existing `packages/apps/kubernetes/templates/delete.yaml` in order to delete these orphaned resources manually. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(kubernetes): Fixed the `<name>-talos-ca` and `<name>-talos-tls-cert` Secrets and the `<name>-oidc-bootstrap` Job being left in the tenant namespace after a Kubernetes cluster is deleted. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Kubernetes release cleanup now removes the Talos CA and TLS certificates before their associated Secrets, and also removes the OIDC bootstrap Job. * The final cleanup report now includes best-effort failures from Step 7, alongside failures from Steps 3, 5, and 6. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
84e5e949fe |
test(ci): pin the label-writing workflows with contract tests (#3664)
Closes #3589 Adds `hack/label-workflows-contract.bats`, following the pattern of `hack/promote-gate-contract.bats` / `hack/release-freeze-contract.bats` (same helpers: comment-stripped pins over job/input blocks, one filter per language since `pr-labeler.yaml` mixes YAML `#` and JavaScript `//` comments). ## What it pins **Cross-file label resolution** (the headline check from the issue): - every value in `typeToKind` and `scopeToArea`, plus the three literal `toAdd.add(...)` labels (`area/release`, `kind/breaking-change`, `area/uncategorized`), resolves against a `name:` in `.github/labels.yml` - `stale.yaml`'s four label inputs and all 16 exempt entries (12 issue + 4 PR) resolve the same way; failures name the missing label **Structure:** - trigger list stays `[opened, reopened, synchronize]` — `edited` cannot creep back in as an executable line - exactly the five documented type mappings; a mapping demoted to a `//` comment stops counting - additive-only: `addLabels` once, no `removeLabel`/`setLabels`, `!existing.has(l)` filter present, `area/uncategorized` fallback present - stale policy: label choices, exempt-list sizes, `days-before-stale: 60`, `days-before-close: 14`, `operations-per-run: 100`, `remove-stale-when-updated: true`, and the specific renewal-path exemptions (`lifecycle/frozen`, `triage/accepted`, `do-not-merge/hold`) ## Verification - Passes under both `bats` and `hack/cozytest.sh` (POSIX-only constructs — no process substitution; the first draft used one and cozytest.sh caught it, which is a nice validation of running both) - Mutation-checked per the issue's bar — each of these went red and was restored: typo in a `scopeToArea` value, `edited` added to the trigger list, fallback line demoted to `//`, `days-before-stale` changed, typo in an exempt label, `feat` mapping demoted to `//`, typo in the literal `kind/breaking-change` - Picked up automatically by `BATS_UNIT_FILES` (wildcard) — no Makefile change - The two documented drift examples (the `edited` event claim and the `revert` type in `docs/agents/contributing.md`) are left as-is: they are doc statements, and which side is authoritative is a maintainer call. The tests pin the workflow's current executable behavior. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added automated checks for pull request labeling and stale-item workflows. * Validates workflow labels against defined labels and pull request type mappings against the contributing guide, including documented unmapped types. * Checks pull request event triggers and label-writing behavior. * Verifies stale-item labels, exemptions, policy values, operation limits, and scheduled and manual triggers, including the absence of user-activity triggers. * Ensures commented-out workflow declarations do not satisfy these checks. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
76a4af95c0 |
docs(monitoring): correct the release name the OIDC comment, guide and test fixture encode (#3468)
## What this PR does
The Grafana OIDC `clientId`, audience-scope name, client-secret name and
users-Job name are all derived from the Monitoring release name. In a
running cluster that release is always `monitoring-system`: the tenant
chart's outer HelmRelease `monitoring` points at `extra/monitoring`,
whose `templates/helmrelease.yaml` creates the inner HelmRelease `{{
.Release.Name }}-system` that carries `system/monitoring` and renders
OIDC. There is no release named `monitoring` that renders OIDC.
Three places encoded a release named `monitoring` (i.e.
`<ns>-monitoring`, where a cluster renders `<ns>-monitoring-system`),
per #3338:
- `packages/system/monitoring/templates/_helpers.tpl` header comment —
both example lines were wrong. The per-tenant release is
`monitoring-system`, and the platform Grafana lives in the `tenant-root`
namespace, not `cozy-monitoring`. (`cozy-monitoring` only hosts the
ExternalName services that redirect to the platform stack the root
tenant hosts in `tenant-root` — see
`packages/system/cozystack-basics/templates/monitoring-external-services.yaml`.)
- `docs/oidc-grafana.md` — placed the platform release in
`cozy-monitoring`; it lives in `tenant-root`. It also named the
per-tenant release `monitoring`; that is `monitoring-system` too, so
both instances now read `monitoring-system` and differ only by
namespace.
- `packages/system/monitoring/tests/oidc_test.yaml` — the suite pinned
`release.name: monitoring`, so its clientId / audience-scope /
client-secret / users-Job assertions pinned names no cluster renders.
Both fixtures are set to `monitoring-system` and the derived expected
values updated to what a cluster actually renders.
Nothing changes at runtime — the templates are self-consistent whatever
the release is called; this only aligns the descriptions and the test
fixture with reality.
Out of scope here (from the issue, left for a maintainer decision):
- Confirming/removing the apparently-dead `cozystack.monitoring`
PackageSource in `packages/core/platform/sources/monitoring.yaml` (issue
item 3) — removing a source is a call I can't safely make.
- Re-checking Detail B of #3259 against the corrected topology (issue
item 4).
Verification, rebased on current main: `helm unittest -f
'tests/oidc_test.yaml' packages/system/monitoring` — 39/39 pass; the
whole monitoring chart passes as well. After setting the fixtures I
updated only the assertions helm-unittest reported as newly diverging.
Disclosure: prepared with AI assistance; I reviewed the change and
verified the release-name topology against the source files cited above.
### Screenshots
N/A — no UI changes.
### Downstream repositories
Walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file. The three changed paths are a `system/monitoring`
chart `_helpers.tpl` comment, the repo-internal `docs/oidc-grafana.md`,
and a chart test fixture. None match a listed trigger: no change under
`packages/apps/` or `packages/extra/`, no
`packages/core/platform/values.yaml`, and no
variant/bundle/component/release-asset/Talos/tooling change. So no
downstream repository is mechanically affected. (The website's own OIDC
page referenced in cozystack/website#597 is a separate artifact the
reporter is already engaged with, outside this repo's trigger map.)
- [x] No downstream repository is affected by this change
### Release note
```release-note
docs(monitoring): correct the release name (`monitoring-system`) and platform namespace (`tenant-root`) encoded in the Grafana OIDC helper comment, the OIDC guide, and the chart test fixture.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Clarified that both platform and tenant Grafana instances use the
`monitoring-system` release name. The platform instance runs in
`tenant-root`, while tenant instances run in their respective tenant
namespaces.
- Updated the OIDC integration guidance to reflect the platform
namespace and tenant release naming.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
4295930989 |
feat(dashboard): show categories brought by tapped repositories in the sidebars (#4741)
## What this PR does The marketplace and console sidebars render a fixed category list, `["IaaS", "PaaS", "NaaS"]`, and drop every other `spec.dashboard.category`. A repository tapped with `cozypkg` that brings its own category is reachable only at the bottom of the full application list, and its installed applications get no console section at all. Both sidebars now follow `groupByCategory`, the order the marketplace page already uses: IaaS, PaaS, NaaS first, then any other category alphabetically. `Administration` stays in the Admin portal, as before. Unknown categories get the generic grid icon. The note in `docs/agents/contributing.md` that described the hardcoded list is removed, since it no longer holds. Tests: two new cases in `sidebar-sections.test.tsx` (a tapped `Paleocomputing` category appears after the built-in ones in both sidebars, `Administration` stays out). Both fail on the previous code. The console suite (448 tests), `pnpm typecheck` and `eslint` on the changed files pass. ### Screenshots Same cluster and tenant, a repository tapped that brings the `Paleocomputing` category. Before, marketplace: the category is only reachable at the bottom of the full list.  After, marketplace: the category gets its own filter.  Before, console: no section for the tapped applications.  After, console: installed applications of that category get their own section.  ### Downstream repositories Walked the trigger map against the diff: it touches only the console sidebar code and a note in `docs/agents/contributing.md`. No package, values schema, CRD, installer value or `ApplicationDefinition` field changes. The website page on `ApplicationDefinition` does not restrict `spec.dashboard.category` values, and `external-apps-example` uses `PaaS`, which keeps working as before. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(dashboard): categories brought by tapped repositories now appear in the marketplace and console sidebars, after IaaS, PaaS and NaaS, instead of only in the full application list. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Marketplace and Console sidebars now show application categories beyond the built-in options, including custom categories. Categories appear in the order provided by the application grouping. * The Administration category remains excluded from both sidebars. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
be55f6eb2b |
test(ci): load the shared bats helper in the labeler contract test
Every unit bats file under hack/ must load test_helper so it runs with nounset, and hack/bats-strict-setup.bats fails the suite for a file that does not. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
shaurya703
|
79e95a1b30 |
test(ci): pin the label-writing workflows with contract tests
pr-labeler.yaml and stale.yaml decide which labels land on issues and PRs, and neither had a test behind it. A mistyped label name is not an error, it is a rule that silently stops matching. hack/label-workflows-contract.bats pins: - every label either workflow can write or match resolves against a name: in .github/labels.yml (typeToKind, scopeToArea, the three literal adds, stale/close labels, both exempt lists) - the typeToKind mappings are exactly the rows of the type table in docs/agents/contributing.md, in both directions, and a type the table lists under "(no kind label)" has no key in the workflow - the trigger list stays [opened, reopened, synchronize] and 'edited' does not come back in either YAML spelling - the labeler stays additive-only (addLabels only, no removeLabel or setLabels) with the area/uncategorized fallback - stale policy values: label choices, 12 issue and 4 PR exemptions, days-before-stale 60, days-before-close 14, operations-per-run 100 - stale runs from its daily cron and workflow_dispatch only JS // and YAML # comments are stripped before matching, so a mapping or a gate demoted to a comment cannot satisfy the contract. Negative checks use the grep -c / -eq 0 form of the sibling contract files: a negated pipeline is ignored by set -e, so it cannot fail a test under hack/cozytest.sh and fails under bats only as the last command. Every assertion was mutation-checked under both bats and hack/cozytest.sh. POSIX-only constructs; picked up automatically by BATS_UNIT_FILES. Closes #3589 Signed-off-by: shaurya703 <kesar.shaurya@gmail.com> |
||
|
youdie006
|
b7708fb285 |
docs(monitoring): correct the release name the OIDC comment, guide and test fixture encode
The naming block in _helpers.tpl, the guide's opening paragraph and the oidc test fixture all describe the per-tenant Grafana as the release `monitoring`. Both instances are released as `monitoring-system` by the monitoring application and differ only by namespace: the platform one in `tenant-root`, a per-tenant one in each tenant namespace. Assisted-by: LLM Signed-off-by: youdie006 <youdie006@users.noreply.github.com> |
||
|
Ray Low
|
904bc38fc3 |
fix(kubernetes): delete the Talos PKI and oidc-bootstrap Job on uninstall
cert-manager runs with enableCertificateOwnerRef: false, so the <release>-talos-ca and <release>-talos-tls-cert Secrets carry no ownerReference to their Certificates and outlive the release, keeping the CA private key of a cluster that no longer exists. A new cluster created under the same name then adopts that stale CA instead of issuing its own trust root. The <release>-oidc-bootstrap Job is a hook whose only delete policy is before-hook-creation. Hooks are not part of the release manifest, so uninstall never removes it. The pre-delete hook deletes the two Certificates before their Secrets, because deleting a Secret while its Certificate exists makes cert-manager re-issue it. It runs after the TenantControlPlane is gone, since the control plane's CSR signer mounts the CA. The Role grants each delete by resourceName only. Assisted-by: LLM Signed-off-by: Ray Low <crossrt@gmail.com> |
||
|
Yasar Rahman
|
9a88afe6ca |
feat(redis): run Sentinel on the same image as the Redis pods
The Sentinels were left on redis:6.2.6-alpine when the operator moved to the freshworks-oss fork, so that its new compiled-in default could not move them on upgrade. 6.2 is end of life, and every instance now runs a Sentinel one or two major versions behind the Redis it monitors. The rollback risk behind that pin, a newer engine rewriting dump.rdb in a format the older one cannot load, does not apply to Sentinel. It stores no RDB or AOF, only sentinel.conf on an emptyDir that the config-copy init container rewrites from the operator's ConfigMap on every start, so moving back to an older image starts from a clean config. Taking the image from the same version map as the data pods also means one image per node instead of two. The e2e comment that relied on busybox's timeout exit code in the Sentinel container is updated, since that container now carries coreutils. Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> Assisted-by: LLM |
||
|
Aleksei Sviridkin
|
52f542c4f0 |
docs(security): add an OpenSSF Security Insights manifest (#4757)
## What this PR does Adds `security-insights.yml` at the repository root, an [OpenSSF Security Insights](https://github.com/ossf/security-insights) v2.2.0 manifest. [CLOMonitor](https://clomonitor.io/projects/cncf/cozystack) reads two checks only from this file, `security_insights` and `dependencies_policy`, and both fail today although the facts already exist in prose. The manifest restates what SECURITY.md, MAINTAINERS.md, GOVERNANCE.md and the General Technical Review already say: the private reporting channel, the maintainers, the current Security Champion, the CodeQL / zizmor / Scorecard / Trivy controls, and the release distribution points. `dependency-management-policy` points at the Day 2 → Dependencies section of the GTR, which holds the dependency lifecycle policy. Expected effect after the next CLOMonitor scan: security section 72.73 → 86.36, project score +2.5. Who is named: - `project.administrators` lists the six people who hold admin on this repository (kvaps, lllamnyp, tym83, lexfrei, myasnikovdaniil, IvanHunters). The `thelinuxfoundation` service account is left out, as the comment above the list says. - `project.core-team` follows the active maintainers in MAINTAINERS.md. - `champions` names @tym83, the current Security Champion from SECURITY.md. The header comment asks for `last-reviewed` to be bumped on each rotation. Validation: the file passes `cue vet -d '#SecurityInsights'` against `spec/schema.cue` at v2.2.0 and the extra field checks in CLOMonitor's `security_insights/v2.rs`, and every file and anchor it links exists on main. ### Downstream repositories - [x] No downstream repository is affected by this change The trigger map has no entry for a root metadata file. A related but independent website change fixes CLOMonitor's `trademark_disclaimer` check: cozystack/website#736. The remaining CLOMonitor gaps are filed as #4758 (FOSSA), #4759 (signed releases and SBOM) and #4760 (Artifact Hub). ### Release note ```release-note docs(security): publish an OpenSSF Security Insights manifest (security-insights.yml) describing the project's vulnerability reporting, security tooling and dependency policy. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a standardized security information document covering project and repository details, administrators, security contacts, private vulnerability reporting, governance, and release information. * Documented security assessments and tools, including which tools run in CI and the tools’ ad hoc and release scan settings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
df2e1c33e1 |
feat(kubernetes-nodes): build the Proxmox worker template from the pool's Talos values (#4742)
## What this PR does
A `substrate: proxmox` pool clones its workers from a VM template the
chart never sees: capmox selects it by `proxmox.templateTags`, and the
pool's `talos.*` values reach only the installer written into the worker
machineconfig. Nothing in the repository says which image that template
has to carry, and the image an operator reaches for is the wrong one:
- The release asset `nocloud-amd64.raw.xz` is built from
`packages/core/talos/images/talos/profiles/nocloud.yaml`, the bare-metal
node image. Its `zfs` extension starts `ext-zfs-service`, which waits
for `/dev/zfs`. A VM never has it, so `startAllServices` never finishes,
and Talos fails the boot sequence and reboots the guest each time the
boot timeout runs out. On our v1.13.6 stand that was every ~70 minutes,
visible in the tenant only as `SandboxChanged` on every Pod (218 cilium
restarts in 12 days); from the Proxmox side the VM never restarts. The
same profile also follows the host nodes' Talos (v1.14.2 since
|
||
|
Marian Koreniuk
|
e21d63aafa |
docs(security): add an OpenSSF Security Insights manifest
CLOMonitor reads its security_insights and dependencies_policy checks only from a Security Insights manifest, so both fail today even though SECURITY.md and the General Technical Review already document the vulnerability process and the dependency lifecycle. The manifest restates those facts in the machine-readable v2 schema and points dependency-management-policy at the Day 2 Dependencies section of the GTR. Administrators are the six people holding admin on this repository, core-team follows the active maintainers in MAINTAINERS.md, and the champion is the Security Champion named in SECURITY.md. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Aleksei Sviridkin
|
1d813c9445 |
fix(backups): refuse concurrent restores and skip the CNPG purge only for this restore's Cluster (#4754)
## What this PR does Fixes #3322. An in-place Postgres (CNPG) restore decided whether to skip deleting the live `Cluster` by comparing its `creationTimestamp` with the RestoreJob's `StartedAt`. The timestamp says the Cluster is new, not which restore rendered it, so two cases let a restore skip its purge and report `Succeeded` on data from another restore: 1. Two RestoreJobs into the same application at once. The second sees the Cluster the first produced, created after its own `StartedAt`, and skips its purge. 2. The live Cluster is deleted while the restore waits for the WAL archive. The still-active HelmRelease re-renders it from the previous restore's values, again created after `StartedAt`. ### Refuse a restore into an application another restore holds The RestoreJob controller now refuses, for every driver, a RestoreJob that has not started yet when another unfinished RestoreJob in the namespace writes into the same application and goes first. The refused job fails with reason `ConcurrentRestore`, and its message names the job that goes first. It fails instead of waiting, so the application is never restored a second time later, after the first restore, by a job its author may no longer expect to run. A started restore goes before one that has not started. Between two that have not, the older goes first, by creation time and then by name; creation times have one-second precision, so two jobs created in the same second are ordered by name. A running restore is never failed by a newer one. The target application is resolved by `restoreJobTarget`, the same helper the BackupJob hold uses, so a Velero restore into another namespace does not collide with an in-place one. The gate lists RestoreJobs through the manager's API reader, not the informer cache. A job reconciled before the cache holds its peer's `StartedAt` would otherwise see that peer as not started and, on a same-second name tie, proceed alongside it. The controller runs a single worker, so a peer's `StartedAt` written by an earlier reconcile is already on the API server when the next job reads it; `SetupWithManager` now sets `MaxConcurrentReconciles: 1` explicitly, since the guarantee depends on it. ### Skip the CNPG purge only for this restore's Cluster The restore patch already sets `bootstrap.newServerName` to `restoredServerName(cluster, RestoreJob UID)`, a value unique to each RestoreJob. The postgres chart now annotates the `Cluster` with it as `postgres.cozystack.io/restored-server-name`. The annotation is rendered even when the barman plugin is not; the plugin's `serverName` also carries the value, but the plugin renders only with chart-level S3 coordinates. The driver skips the purge only when the annotation names this RestoreJob. A Cluster produced by a concurrent restore, or re-rendered from an earlier restore's values, carries another value and is purged. The mark lives on the rendered Cluster rather than in RestoreJob status. That keeps it valid in the case the skip exists for: the purge succeeded, but the `TargetPurged` status write failed. A UID recorded in status was rejected in the code for exactly that reason. A Cluster rendered by a chart that predates the annotation has none and keeps the creation-time check, so a restore in flight across an upgrade behaves as before. ### Tests - `TestRestoreAhead` covers order, a started job going first, the name tie-break, finished and terminating jobs, other applications, retargeted restores and a Velero restore into another namespace. - `TestReconcileRestore_ConcurrentRestoreIsRefused` checks that the newer job fails with `ConcurrentRestore` and the running one is left alone. - `TestReconcileRestore_ConcurrentRestoreReadsPastTheCache` gives the cache a stale, not-started peer and the API reader the started one, and requires the job to fail with `ConcurrentRestore`. - `TestCNPGClusterFreshlyRecovered` adds both cases from the issue: a Cluster marked by another restore and created after `StartedAt` must be purged. The timestamp-only cases are kept for the unmarked fallback. - `TestReconcileCNPGRestore_RepeatInPlacePurgesStaleRecoveryCluster` gains two reconcile-level cases. One purges a Cluster that another restore marked, even though it was created after `StartedAt`. The other runs a full purge, takes the mark from the `bootstrap.newServerName` the restore patch wrote, drops `TargetPurged` as a lost status write would, and requires a Cluster carrying that mark to survive, although it predates `StartedAt`. That pins the purge decision to the value the chart actually renders. - `packages/apps/postgres/tests/restore_mark_test.yaml` checks that the annotation is rendered with and without the barman plugin, and is absent on a Cluster that was never restored. Each new test was checked against the code it guards: disabling the gate call, listing through the cache, flipping the started-job precedence, ignoring the annotation, or deriving the expected mark from the app name instead of the cluster name each turns one red. `go test ./internal/backupcontroller/...` and `helm unittest packages/apps/postgres` (152 tests) pass. The restore flow was not run against a live cluster. `docs/operations/backup-classes.md` documents the one-restore-at-a-time rule and the annotation. ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff: Go changes in `internal/backupcontroller` (no CRD or API type change), one annotation added to the postgres chart's Cluster (no `values.yaml` or `values.schema.json` change), and a docs paragraph. No package is added or renamed, no platform values, ApplicationDefinition, RestoreJob CRD, metric, label or namespace changes, so no trigger is reached. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backups): a RestoreJob now fails with reason ConcurrentRestore when another unfinished RestoreJob is already restoring the same application, instead of running alongside it; create it again once the first restore has finished. An in-place Postgres restore now skips deleting the live Cluster only when that Cluster was rendered for this RestoreJob, identified by the new postgres.cozystack.io/restored-server-name annotation, so it no longer reports Succeeded on data from a concurrent or earlier restore. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Restores targeting the same application are now handled one at a time. A competing restore is refused rather than queued, with started restores taking priority. * PostgreSQL now calculates a default `shared_buffers` value when the memory limit is below 512 MiB, while respecting explicitly configured values. * **Bug Fixes** * Recovery clusters are more reliably distinguished from manual recoveries and clusters created by other restores, reducing the risk of purging the wrong cluster. * **Documentation** * Clarified restore ordering and how recovery-cluster cleanup is determined. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
05cfb63044 |
feat(keycloak): expose zone placement for Keycloak and the database proxy (#4606)
## What this PR does Neither the Keycloak StatefulSet nor the `keycloak-kms-proxy` Deployment exposes any placement, so the scheduler is free to put both replicas of each in one failure domain. For Keycloak this is worse than it looks: a StatefulSet does not replace a pod stuck on an unreachable node, so losing that domain takes every Keycloak replica with it for as long as the domain stays lost. This exposes placement for both, with empty defaults so nothing changes unless an installation opts in: - `affinity` and `topologySpreadConstraints` for the Keycloak StatefulSet; - `encryption.affinity` and `encryption.topologySpreadConstraints` for the proxy; - a `PodDisruptionBudget` (`maxUnavailable: 1`) for the proxy when `encryption.replicas > 1`, so a drain cannot take both replicas at once. The `values.yaml` example spreads one replica per zone with `whenUnsatisfiable: DoNotSchedule` and `nodeTaintsPolicy: Honor`. The taint policy matters for a zone loss: kube-scheduler already ignores terminating pods when it counts the spread (`countPodsMatchSelector` in `pkg/scheduler/framework/plugins/podtopologyspread/common.go`, "Bypass terminating Pod"), but with the default `nodeTaintsPolicy: Ignore` the lost zone's nodes still count as a domain holding zero pods, so a replacement pod could not be placed on the surviving zone without violating `maxSkew`. With `Honor`, nodes carrying the `node.kubernetes.io/unreachable` NoSchedule taint drop out of the count and the replacement schedules. `ScheduleAnyway` was tried first in a live deployment and did not spread the proxy replicas on rollout. Found during a site-isolation drill in which both Keycloak pods and both proxy replicas had drifted into one zone. Tests: new `tests/placement_test.yaml` (defaults render no placement; set values render as given; PDB only for more than one proxy replica); `helm unittest` passes 87/87; the default render is unchanged apart from a randomly generated secret. ### Screenshots Not applicable, no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff: it touches only `packages/system/keycloak` (templates, values, tests) — no app under `packages/apps` or `packages/extra`, no platform values, CRD, `hack/` tooling, commit convention or PR template. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(keycloak): expose affinity and topologySpreadConstraints for Keycloak and the keycloak-kms-proxy, and add a PodDisruptionBudget for the proxy when it runs more than one replica. Defaults are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional node affinity and topology spread settings for Keycloak, its KMS proxy, and the database to control pod placement. * Database-specific topology spread settings take precedence over cluster-wide settings; constraints without a selector are scoped to the database cluster. * Added a disruption budget for the KMS proxy when running multiple replicas, allowing up to one pod to be unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
f093273736 |
feat(redis): size the Sentinel pods separately from Redis (#4832)
## What this PR does
The Redis application renders `spec.sentinel.resources` from the same
`resources`/`resourcesPreset` pair as the Redis pods. Every instance
runs three Sentinels, so it reserves three extra copies of a Redis
replica's memory for processes that hold only their configuration and
the failover state. The dataset that drives the Redis sizing never
reaches a Sentinel.
On our Cozystack v1.6.4 staging cluster, an HA Redis on `t1.nano`
reserves 3 x 128Mi for its Sentinels. The VPA recommender
(recommendation mode only) gives an uncapped target of 16Mi for the
`sentinel` container. On a `u1.small` instance the same three Sentinels
would reserve 3 x 4Gi.
This PR adds an optional `sentinel` block with the same two keys the top
level has:
```yaml
resourcesPreset: u1.small
sentinel:
resources:
cpu: 100m
memory: 64Mi
# resourcesPreset: t1.nano
```
Resolution rules:
- `sentinel` empty (the default): the Sentinels keep the Redis sizing
(`resources` + `resourcesPreset`). Existing instances therefore render
exactly as before and an upgrade does not roll their Sentinels.
- `sentinel.resourcesPreset` set: the Sentinels use that preset.
Explicit top-level `resources` are not carried over, since they were
sized for the dataset.
- `sentinel.resources` set: it wins. Any key it leaves out is filled
from `sentinel.resourcesPreset`, which falls back to `resourcesPreset`.
- The Sentinel `WorkloadMonitor` label
`workloads.cozystack.io/resource-preset` now carries the preset the
Sentinels actually use. The Redis `WorkloadMonitor` is unchanged.
Changes:
- `values.yaml`: new `Sentinel parameters` section (`@typedef Sentinel`,
`@param {Sentinel} [sentinel]`, default `{}`), reusing the existing
`Resources` and `ResourcesPreset` types so the enum and quantity
validation is the same as the top-level pair.
- `templates/redisfailover.yaml`: computes the Sentinel preset and
resources once and uses them for `spec.sentinel.resources` and the
Sentinel `WorkloadMonitor` label.
- `README.md`: generated table rows plus a short hand-written `sentinel`
section next to `resources and resourcesPreset`.
- Generated: `values.schema.json`, `api/apps/v1alpha1/redis/types.go`
(new `Sentinel` struct, `ConfigSpec.Sentinel`),
`api/apps/v1alpha1/redis/zz_generated.deepcopy.go`,
`packages/system/redis-rd/cozyrds/redis.yaml`.
- `tests/sentinel_resources_test.yaml`: 10 helm-unittest cases covering
the rules above and an unknown preset being refused.
I chose not to change the default sizing. A smaller default (for example
a fixed small preset for Sentinel) would save more for everyone, but it
would also roll every instance's Sentinels and change what tenants are
billed for on upgrade. If maintainers prefer that, it is a one-line
follow-up on top of this change.
Testing done:
- `make generate` in `packages/apps/redis` with cozyvalues-gen v1.7.0
(the version `pre-commit.yml` installs); regenerated files committed.
- Deepcopy: `controller-gen object` v0.16.4 on `api/apps/v1alpha1` (the
last step of `hack/update-codegen.sh`), only the redis package changed;
`go build ./...` and `go vet ./redis/` pass in that module, `go mod
tidy` leaves no diff. I could not run the full root `make generate` on
my machine: code-generator v0.37.1 fails to parse the Go 1.27.1 standard
library (`math/rand/v2: method must have no type parameters`), which is
a host toolchain problem unrelated to this change. CI's `codegen-drift`
job should confirm.
- `make test` in `packages/apps/redis` (helm-unittest v1.1.2): 58
passed. With the template change reverted, 4 of the new cases fail (the
ones that set `sentinel.*`); the cases that pin today's behaviour pass
both ways.
- Rendered output with no `sentinel` block is identical to `main` (all
documents except the random-password Secret) for the defaults,
`resourcesPreset=u1.small`, and explicit `resources` (cpu 1, memory
2Gi).
Backwards compatibility: purely additive. The field is optional with an
empty default, existing instances render unchanged, and nothing rolls on
upgrade. Clients that do not know the field are unaffected.
### Screenshots
Not a UI change. The dashboard form gets a `sentinel` object from the
schema like other nested parameters (e.g. `tls`).
### Downstream repositories
- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
Trigger map walked against the diff: this adds a field to
`values.schema.json`, so `cozystack/terraform-provider-cozystack` needs
to model `sentinel.resources` / `sentinel.resourcesPreset` in its Redis
resource. I will open that issue once this PR has a number and link it
here; a draft is in `FOLLOWUP-terraform-provider-redis-sentinel.md`. The
website page for `redis` is regenerated from `README.md` by the release
bot (`redis` is in the website `APPS` list), so no manual follow-up
there. Nothing else in the map is touched.
### Release note
```release-note
feat(redis): new optional `sentinel.resources` and `sentinel.resourcesPreset` parameters size the three Sentinel pods independently of the Redis pods. When unset, Sentinels keep the Redis sizing as before, so existing instances are unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added independent CPU and memory resource settings and sizing presets
for Redis Sentinel pods.
* When Sentinel-specific settings aren’t provided, Sentinel pods inherit
Redis resource settings. Unspecified Sentinel resource values can use
the Sentinel preset, which defaults to the Redis preset.
* Added configuration documentation and validation for Sentinel resource
settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
bd8264fbda |
docs(api): clarify kind-wide Helm timeout overrides (#4493)
## What this PR does Clarifies that Helm install and upgrade timeout annotations are configured on an ApplicationDefinition and apply to every Application of that kind. It also documents that the upgrade-specific timeout takes precedence over the shared install timeout and the global upgrade default. Fixes #4382 ### Screenshots N/A — this change only updates comments and CLI help text. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified that install and upgrade timeout overrides are configured for an Application kind through its `ApplicationDefinition`, rather than per Application. * Specified that the install-timeout setting applies to both installs and upgrades, while the upgrade-timeout setting takes precedence for upgrades. * Updated descriptions of global timeout defaults and override precedence. Timeout behavior is unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Marian Koreniuk
|
6852f723b8 |
fix(kubernetes-nodes): refuse a newline in the Proxmox template tags
COZY_TEMPLATE_TAGS is comma-separated, but the normalisation also split on a newline, so `talos-v1.\n13` became two tags where the operator meant one. A newline there is now refused, like in every other value the program receives. The stub tests clean up their scratch directory after the last assertion, so a failing test leaves its qm, curl and ssh records behind for inspection. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Marian Koreniuk
|
a8635f4f68 |
fix(kubernetes-nodes): refuse multi-line values in the Proxmox template script
matches() held each value to its shape with grep -x, which matches line by line, so a value with a newline passed when any one of its lines did and carried the others into the program create runs as root. A value with a newline is now refused, for every field the program receives. The test file loads test_helper and spells its absence check without `!`, for the two Bats audits on main. The Talos version check accepts the pre-release suffix the chart's render accepts, and a failure after qm create names the qm destroy that frees the VMID for the next run. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Marian Koreniuk
|
bbd39c94a4 |
feat(kubernetes-nodes): build the Proxmox worker template from the pool's Talos values
A proxmox pool clones its workers from a VM template the chart never sees, and nothing said which Talos image that template has to carry. The one an operator reaches for, the platform release's nocloud-amd64.raw.xz, is the bare-metal node image: its zfs extension's service waits for a /dev/zfs no VM has, the boot sequence never completes, and Talos reboots every worker each time the boot timeout runs out, about every 70 minutes on v1.13. It also follows the host nodes' Talos version, not the pool's talos.version that the installer in the worker machineconfig names. The chart's default schematic has a trap of its own: its qemu-guest-agent service waits the same way for the agent's virtio port, which Proxmox adds only to a VM with the agent enabled. hack/proxmox-talos-template.sh builds the template from the chart's talos.schematicID and talos.version in the nocloud flavour with the agent enabled, refuses a tag set another template already carries, because capmox then finds two and clones neither, and prints the templateTags that select it. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Aleksei Sviridkin
|
08ee470f6a |
fix(postgres)!: refuse plaintext sessions on external instances (#4847)
## What this PR does CloudNativePG offers TLS on an external Postgres but does not require it. Its built-in last `pg_hba` rule is `host all all all`, which matches TLS and non-TLS connections alike, so a client outside the cluster that connects with `sslmode=disable` gets an unencrypted session. The chart's `tls.enabled` only decides whether the external hostname goes into the certificate, so it cannot prevent this. This is #4619. When `external: true`, the chart now renders `spec.postgresql.pg_hba`: ```yaml pg_hba: - hostssl all all all scram-sha-256 # md5 on PostgreSQL 13, matching CloudNativePG's default rule - host all all all reject ``` CloudNativePG puts user rules after its fixed rules (`local` peer, `streaming_replica` and pooler `cert`) and before its default `host all all all` rule, and PostgreSQL uses the first matching rule, so the two lines above make remote sessions TLS-only without touching replication or the metrics exporter. See the CloudNativePG [`pg_hba` section](https://cloudnative-pg.io/documentation/current/postgresql_conf/#the-pg_hba-section) and the [template](https://github.com/cloudnative-pg/cloudnative-pg/blob/v1.30.0/pkg/postgres/configuration.go) for v1.30.0, the operator version the platform ships. A new tri-state `tls.required` follows the same pattern as `tls.enabled`: when omitted it follows `external`, `false` opts out, `true` requires TLS on an in-cluster instance too. The issue is silent on a switch; I added one because the change is breaking and a tenant with a client that cannot do TLS needs a way back that does not mean patching the operator's resources. In-cluster clients: the pod network cannot be told apart from an external client (the external Service uses `externalTrafficPolicy: Local`, so the source IP is the client's), so the rules apply to every remote client. The chart's init job connects to `<release>-rw` with libpq's default `sslmode=prefer`, which already negotiates TLS, as do the backup and monitoring paths that run inside the pod over the Unix socket. Only clients that disable TLS explicitly are refused. In-cluster instances with `external: false` are unchanged. Changes: - `templates/_tls.tpl`: `postgres.tls.required` helper. - `templates/db.yaml`: renders `spec.postgresql.pg_hba` when required. - `values.yaml`: `tls.required`, with the generated `values.schema.json`, `README.md`, `api/apps/v1alpha1/postgresql/types.go` and `packages/system/postgres-rd/cozyrds/postgres.yaml`. The `Required` deepcopy block in `zz_generated.deepcopy.go` is controller-gen v0.16.4 output. - `README.md`: new "Refusing plaintext sessions" paragraph. - `tests/pg_hba_test.yaml`: 10 helm-unittest cases. Backwards compatibility: breaking for existing external instances whose clients connect without TLS. The rules apply on the next reconcile. They are refused until the client enables TLS or the instance sets `tls.required: false`. Testing done: - `make generate` in `packages/apps/postgres` with cozyvalues-gen v1.7.0; regenerated files committed. - `helm unittest .` in `packages/apps/postgres`: 206 passed (25 suites), 10 of them new. I did not run it against a live cluster, so the `pg_hba` ordering is taken from the operator source and docs rather than observed. ### Screenshots Not a UI change. The dashboard form gets a `tls.required` toggle from the schema. ### Downstream repositories I walked the trigger map against the diff and left every box empty so a human decides. The nearest hit is the website's managed-apps reference page for postgres, which a stable-tag bot regenerates from `README.md`; the Terraform provider's postgres schema may also want the new `tls.required` field. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(postgres)!: an external Postgres now refuses sessions that are not encrypted. The chart renders `hostssl` rules for all users and databases with scram-sha-256 (md5 on PostgreSQL 13, matching CloudNativePG's default rule) and rejects plain `host` connections, so a client using `sslmode=disable` is refused. The new `tls.required` value follows `external` by default; set it to `false` to keep accepting plaintext sessions. ``` Fixes #4619 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a `tls.required` setting for PostgreSQL instances. By default, TLS is required for external instances and optional for others. You can override this behavior to require encrypted connections for both external and in-cluster clients, or allow plaintext connections for external instances. Rendering fails if TLS is required while PostgreSQL SSL is disabled. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
f0e3e17493 |
feat(kubernetes-nodes): let CDI trust a self-hosted Talos image factory (#3797)
## What this PR does Adds `talos.imageFactoryCA` to the `kubernetes-nodes` chart, so CDI can verify a Talos Image Factory served with a certificate from an internal CA. Fixes #3796. ### The gap `talos.imageFactoryURL` is documented for self-hosted factories and mirrors in air-gapped, rate-limited or flaky-egress environments. Such an endpoint is normally served by the organisation's own CA, and CDI, which trusts only its system store, refuses the import with `x509: certificate signed by unknown authority`. The DataVolume never completes and no worker joins. `DataVolume.spec.source.http.certConfigMap` exists for exactly this, but the worker DataVolume in `templates/nodegroup.yaml` renders `url` alone and no value reaches that field. ### The change One optional value on the existing `talos` group of a `KubernetesNodes` pool: ```yaml talos: imageFactoryURL: "https://factory.example.com" imageFactoryCA: | -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- ``` When set on an HTTP import path (the default and `osImage.factory`), the chart renders a ConfigMap `<release>-talos-factory-ca` in the release namespace and points the DataVolume's `source.http.certConfigMap` at it. A pool on `osImage.builtin` clones a golden image, builds no HTTP source and gets no ConfigMap. Empty, the default, renders nothing: `helm template` with the field unset is byte-identical to `main`, so existing pools keep their KubevirtMachineTemplate name. Setting the CA changes that name, so the pool rolls once. CDI appends the bundle to the system pool rather than replacing it (`x509.SystemCertPool()` followed by `AppendCertsFromPEM` in `pkg/importer/http-datasource.go`), so a publicly trusted endpoint keeps verifying. The PEM is taken inline rather than as a ConfigMap name because `certConfigMap` must live in the DataVolume's namespace, and the dashboard offers no way to create that object by hand. ### Fail-closed guards The value lands verbatim in a ConfigMap that anything able to read the namespace can read, so it must be complete PEM certificate blocks and nothing else, and must never carry private key material. That includes the realistic accident of `cat ca.pem key.pem` pasted into the field. The rules now live in one shared helper, `cozy-lib.tls.validateCACert`, which `cozy-lib.tls.caCertSecret` also calls instead of keeping its own copy. The regexes are unchanged; the failure message names the caller's field. ### Testing - `packages/apps/kubernetes-nodes/tests/talos_image_factory_ca_test.yaml`, 13 cases: unset renders nothing, set renders the ConfigMap and a matching `certConfigMap`, whitespace-only counts as unset, multi-certificate chains, the builtin path, and each guard on the input that trips it. - `packages/tests/cozy-lib-tests/tests/tls_validatecacert_test.yaml`, 6 cases for the shared helper, plus the existing `caCertSecret` tests updated for the new message. - `make generate` in the package: regenerated `values.schema.json`, `README.md`, `api/apps/v1alpha1/kubernetesnodes/types.go` and the `kubernetes-nodes-rd` definition are committed. Not yet exercised end to end against a live internal factory. ### Not in this PR `packages/apps/vm-disk/templates/dv.yaml` renders `source.http.url` with the same gap. Left for a follow-up if this shape is accepted. ### Screenshots Not a UI change. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: cozystack/terraform-provider-cozystack#31 (the field belongs to the `kubernetes_nodes` model) - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff: - **terraform-provider-cozystack**: ticked. The diff adds a field to `packages/apps/kubernetes-nodes/values.schema.json`, which the trigger map names directly. Filed as an issue there. - **website**: not ticked. No package is added, renamed or removed, and the reference page is regenerated from the package `README.md`, which is regenerated and committed here. - Every other repository: no path in the trigger map is touched. ### Release note ```release-note feat(kubernetes-nodes): add `talos.imageFactoryCA` so CDI can verify a self-hosted Talos Image Factory served with a certificate from an internal CA. Previously the worker image import failed with `x509: certificate signed by unknown authority`. Set the CA in PEM form and the chart wires it into the worker DataVolume; leave it empty and nothing changes. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an optional setting for providing PEM certificate authorities trusted when importing worker OS images over HTTPS. This supports the default image source and factory images, allowing imports from endpoints that use certificates not already trusted by the system. * Certificate authorities are provided as PEM certificate blocks; when configured, they are used for HTTPS image imports. The setting defaults to empty and has no effect on pools using built-in OS images. * **Bug Fixes** * Invalid certificate input, including private keys, is rejected. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
dde75d6cf5 |
feat(keycloak): expose zone placement for the keycloak-db instances
The Keycloak StatefulSet and the database proxy can already be spread across failure domains, but the CloudNativePG Cluster behind them could only follow the cluster-wide topology constraint, with no way to pick zones for this database alone. With both instances in one zone, losing that zone loses the database and every login with it. Add database.affinity (CloudNativePG shape) and database.topologySpreadConstraints, with empty defaults so nothing changes unless an installation opts in. The cnpg.io/cluster label selector is added to constraints that carry none, and the constraints replace the cluster-wide ones for this database when set. Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Aleksei Sviridkin
|
faf8a250ba |
fix: typo in monitoring-agents values.yaml (#4799)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Fixes a typo in monitoring-agents values.yaml. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Corrected the namespace metadata field used for event streams. Event metadata is now labeled consistently, helping downstream tools identify the namespace associated with each event. This fixes an incorrect field name in the event output configuration without changing other event stream behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
6a75883013 |
fix(backups): keep the CNPG purge for unmarked Clusters this restore did not render
A recovery Cluster without the restored-server-name annotation fell back to the creation-time comparison. On the current chart the annotation is absent when bootstrap.newServerName is empty, i.e. a manually configured recovery: re-rendered during the WAL-archive wait it is created after StartedAt, the purge was skipped and the restore could report success on the manual recovery source. Use the creation-time fallback only when the application's bootstrap.newServerName already carries this restore's server name, which keeps it for Clusters rendered by a chart that predates the annotation. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
35ca217839 |
fix(backups): name the restore that goes first in ConcurrentRestore
restoreAhead returned the alphabetically smallest of the jobs ahead, so with more than one ahead the ConcurrentRestore message could name a job that is not the one holding the application. Return the first job in restoreGoesFirst order, and pin the creation-time tie-break with cases where creation order and name order disagree. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
e4e991e837 |
fix(backups): skip the CNPG restore purge only for this restore's Cluster
The in-place CNPG restore skips deleting the live Cluster when it carries bootstrap.recovery and was created after the RestoreJob started, taking that as "this restore's own purge and re-render produced it". The timestamp does not say which restore rendered the Cluster. A Cluster deleted while the restore waits for the WAL archive is re-rendered by the still-active HelmRelease from the previous restore's values and is created after StartedAt too; the restore then skipped its purge and reported Succeeded on the previous restore's data. The restore patch already sets bootstrap.newServerName to restoredServerName(cluster, RestoreJob UID). The chart now annotates the Cluster with it as postgres.cozystack.io/restored-server-name, whether or not it renders the barman plugin that also carries the value, and the purge is skipped only when the annotation names this RestoreJob. The mark lives on the rendered Cluster rather than in RestoreJob status, so it still covers the failed status write the skip exists for. A Cluster rendered by a chart that predates the annotation keeps the creation-time check. Fixes #3322 Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
3daf942dce |
fix(backups): refuse a restore into an application another restore holds
Nothing stopped two RestoreJobs from rewriting one application at once. Each restores the target from its own Backup, so the application ends up on whichever finishes last, and the CNPG driver's purge decision cannot tell which of the two produced the live Cluster: the second job could skip its purge and report Succeeded on the first job's data. A RestoreJob that has not started now fails with reason ConcurrentRestore when another unfinished RestoreJob in the namespace writes into the same application and goes first. A started restore goes before one that has not; between two that have not, the older one goes first, by creation time and then name, so a running restore is never failed by a newer one. The refused job fails instead of waiting, so the application is not restored a second time later, after the first restore, by a job its author may no longer expect to run. The gate lists RestoreJobs through the manager's API reader rather than the informer cache. Creation times have one-second precision, so two jobs created in the same second are ordered by name, and a job reconciled before the cache held its peer's StartedAt would see that peer as not started and proceed alongside it. The controller runs one worker, so a peer's StartedAt written by an earlier reconcile is on the API server by the time the next job reads it; SetupWithManager now states that. The target is resolved as the BackupJob hold resolves it, so a Velero restore into another namespace does not collide with one in place. Refs #3322 Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Aleksei Sviridkin
|
22e0fdbc63 |
test(tests): stop unit tests from reaching a cluster (#4804)
## What this PR does One unit test was reading from whatever cluster the caller's kubeconfig selected. The knob test in `hack/run-kubernetes-node-join_test.bats` runs `cozy_report_node_join_failure` in a fresh `bash -c` and stubs only `kubectl`, as a shell function. `cozy_diag_read` runs every read through `timeout`, and `timeout` execs the real `kubectl` from PATH, so the stub never ran. With a recording `kubectl` first on PATH, one run of that file logged 170 calls such as `kubectl -n tenant-test describe virtualmachineinstances.kubevirt.io`. All of them were reads, but a unit suite should not touch a cluster at all. The PR has three parts: 1. Both `bash -c` blocks in that file now stub `timeout`, as the post-source test already did. They also put recording `kubectl` and `talosctl` executables first on the subshell's PATH and fail if either one is executed. The declined-path block made no calls in my runs, because a zero budget declines every read. It gets the same guard because it has the same shape. 2. The unit and controller targets now run with no cluster to reach. The root `Makefile` sets `KUBECONFIG=/dev/null` and an empty `KUBERNETES_SERVICE_HOST` for `unit-tests`, each of its prerequisites, and `test-controllers`, so the CI command and each target run on its own are covered. The second variable is needed because client-go falls back to in-cluster credentials when the kubeconfig is empty and that variable plus a service-account token are present. The new `hack/unit-lane-no-cluster.bats` runs every prerequisite of `unit-tests` as its own goal with a sentinel kubeconfig and service host in the environment, and fails if any target receives them. 3. One unit file was missing the strict-mode helper. `hack/bats-strict-setup.bats` requires a `load test_helper` line in every unit file, and `hack/fork-new-images.bats` is the only one without it, so the bats lane is red on main and would stay red here. #4805 carries the same one-line fix on its own; if it lands first, this commit drops out on rebase. I ran the whole lane (`make unit-tests test-controllers`) with recording fakes for 20 cluster clients first on PATH. It made no `kubectl` call. `helm` was called 444 times, all `template` or `unittest`, none with `--validate`. `flux` was called once, for `push artifact --help`. Deleting either export line from the Makefile, or removing one target from the list, turns the new test red. Removing the `timeout` stubs turns the knob test red, and the failure lists the reads that escaped. Closes #4793 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note test(tests): unit and controller test targets run with no kubeconfig and no in-cluster service host, so a unit test can no longer reach a cluster. ``` |
||
|
Aleksei Sviridkin
|
bcde187858 |
fix(dashboard): small Console fixes from the #3828 triage (#3837)
## What this PR does Six small Console fixes from the #3828 triage. Each one is small and most of the code sits in `apps/console/src`, so they go together in one PR. `Fixes #3105`: sizes between 1Ki and 1Mi printed as a raw byte count. `humanizeBytes` had branches for Ti, Gi and Mi and then fell through to bytes. It now has a Ki branch with the same `toFixed(0)` as the Mi branch. The `1023B` case still holds, and the test covers 1Ki and 512Ki. `Fixes #3106`: the tenant picker showed on admin pages where the tenant changes nothing. It is now hidden on the cluster-scoped admin pages only: Capacity, Backup classes, External IPs and Modules. The rest of `/admin` mounts the same tenant-scoped pages as `/console`, and those take their namespace from the tenant context, so the picker stays there. `/admin/tenants` keeps it too, because its rows come from the picker-filtered tenant list. `Fixes #3107`: both capacity drill-downs showed one generic error, so a permission failure looked like a broken page. Both now use the `error instanceof K8sApiError && error.status === 403` check that `ClusterStorageSection` already uses, with a test for the forbidden case and one for a generic error on each page. `Fixes #3102`: an immutable leaf was not restored when the YAML edit dropped its parent object. `overlayPath` stopped as soon as the submitted body had nothing at a path segment. It now creates `{}` for the missing parent, but only when the target there is undefined or null and the original really has the leaf to restore. So a scalar or an explicit null the user put there survives. A wildcard with more path under it does not rebuild a deleted parent, since that would put back entries the user removed. The test uses foundationdb's `storage.storageClass`, which is a shipped path of this shape. `Fixes #3135`: a blocked submit did not scroll to the field that blocked it. Plain `focusOnFirstError` crashes here, because RJSF's built-in handler reads `form.elements` and this form is `tagName="div"`. So the form passes a small handler that finds the field by its generated id and scrolls it into view. The ids come from one shared definition, and the widgets that had none now carry it: the `DynamicOptionsWidget` select, which every `x-cozystack-options` field uses, and the `SourceField` controls. The nested form that `AdditionalPropertiesField` mounts per map entry continues the outer id namespace instead of restarting at `root`, so fields like `nodeGroups.*.instanceType` resolve. A map key with a dot in it, such as a mariadb user `john.doe`, resolves by trying both readings of the dotted error path. `Fixes #3108`: `cilium`, `coredns` and `verticalPodAutoscaler` have only `valuesOverride` and no `enabled`, so they rendered as plain groups next to the toggleable addons and looked like a switch that failed to appear. Setting `<addon>.enabled: true` in the YAML editor is accepted, because the schema sets no `additionalProperties`, and the API stores the field and echoes it back. Nothing reads it. The three addons are not the same case. `cilium` and `coredns` are always installed. `verticalPodAutoscaler` has no switch of its own and is installed and removed together with `addons.monitoringAgents.enabled`. The three are now marked in `values.yaml` with a new `x-cozystack-no-enable-switch` schema keyword, and the form renders them with a note that there is no enable switch and an `enabled` field in YAML does nothing. The per-addon reason goes into the schema description, which the form renders and which regenerates into the README, the Go types and the ApplicationDefinition. The generated files are regenerated with cozyvalues-gen v1.7.0, the version CI pins, and the pre-commit check passes. The console typecheck and tests pass in CI. ### Screenshots ##### Ki formatting <img width="1182" height="184" alt="image" src="https://github.com/user-attachments/assets/62662265-4f77-455d-b90e-85e417684bb9" /> ##### Admin without tenant picker <img width="1440" height="245" alt="image" src="https://github.com/user-attachments/assets/99aff4f3-fda9-4c4c-8cf8-0c1eb01fb973" /> ##### Capacity 403 error <img width="1232" height="270" alt="image" src="https://github.com/user-attachments/assets/f4e4b328-ee5e-4516-b78f-9b5084b6bbc8" /> <img width="1232" height="944" alt="image" src="https://github.com/user-attachments/assets/142e438e-f538-436e-aae8-dfb79ec747e8" /> ##### Validation focus <img width="1166" height="135" alt="image" src="https://github.com/user-attachments/assets/89abb73b-a624-406f-9736-3bf38bb8a923" /> ##### Addon lifecycle <img width="1166" height="591" alt="image" src="https://github.com/user-attachments/assets/14a95891-84d1-4073-a009-bc0eab94f114" /> ### Downstream repositories I walked the trigger map in `docs/agents/contributing.md` against the diff, file by file. This PR changes field descriptions, adds the `x-cozystack-no-enable-switch` schema keyword on three addon objects, and changes the console. No field is added, removed or renamed, and no default, version enum, `kind`, `plural`, `release.prefix`, output Secret or Service changes, so none of the terraform provider triggers fire. No package is added or removed, and nothing changes in `packages/core/platform/values.yaml`, the variants, `hack/` or the node prerequisites. cozyvalues-gen itself is unchanged, since the keyword goes through its generic `@x-*` passthrough. The kubernetes reference page on the website regenerates from `README.md` on a stable tag, which is automated rather than a follow-up. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(dashboard): Ki-range sizes now render as Ki instead of raw bytes, the tenant picker is hidden on cluster-scoped admin pages, capacity drill-downs distinguish a permission error from a broken page, an immutable field whose parent object is absent is now applied, a blocked submit scrolls to the field that blocked it including fields inside maps and the source and dropdown widgets, and the cluster addons with no enable switch now say so instead of showing a toggle that never appears ``` |
||
|
Aleksei Sviridkin
|
1b0a5a2829 |
fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2 (#4834)
## What this PR does Bumps the `velero-plugin-for-aws` init container in `packages/system/velero/values.yaml` from `v1.14.1` to `v1.14.4`, and updates the pin in `tests/velero_test.yaml`. `v1.14.1` sets `x-amz-tagging` on every `PutObject`, even when the BackupStorageLocation configures no tags, in which case the header is empty. Cloudflare R2 rejects that with `501 NotImplemented: Header 'x-amz-tagging' with value '' not implemented`. Velero then cannot upload backup metadata, so every Velero backup to an R2-backed location fails. That includes Cozystack's own `cozy-default` BackupStorageLocation when the platform backup bucket is on R2. Upstream fixed this in v1.14.3: velero-plugin-for-aws#309 (a backport of #299, "Only set PutObject Tagging when tags are configured"). `v1.14.4` is the current patch release of the 1.14 line. Its only change on top of v1.14.3 is CVE dependency bumps (#336). v1.14.3 itself also carries #310 (SSE-C presigned URLs), #313 (no amz-checksum encapsulation when `checksumAlgorithm=""`) and CVE/Go bumps. The plugin's compatibility table lists `v1.14.x` for Velero `v1.18.x`, which is what the chart ships (`velero:v1.18.1`), so this stays inside the supported pairing. Testing done: - `make test` in `packages/system/velero` (helm-unittest v1.1.2): 7 passed, with the existing init-container pin assertion moved to `v1.14.4`. - `helm template` renders `velero/velero-plugin-for-aws:v1.14.4` as `initContainers[0]` of the `velero` Deployment. - On a Cozystack v1.6.4 staging cluster with the backup bucket on Cloudflare R2: before the bump, Velero backups failed on the empty `x-amz-tagging` header. After pinning the same plugin image through a Package values override (`docker.io/velero/velero-plugin-for-aws:v1.14.4@sha256:c85a7fa803569e3128c33e4f5766f84ab156c8d171c9f1dd2715581d45ce98fa`), all of our volume backup and restore drills pass. That covers CSI snapshots with the Kopia data mover to R2, and incremental backups confirmed. Backwards compatibility: same plugin minor line and the same Velero version. The init container image changes, so the `velero` Deployment rolls once on upgrade. Users who never set `tagging` on a location now get no tagging header at all; users who do set it get the same header as before. The reference stays in the existing shape (tag only, no digest), like the current `v1.14.1` line. If you'd rather pin third-party init containers by digest, the v1.14.4 multi-arch index digest is `sha256:c85a7fa803569e3128c33e4f5766f84ab156c8d171c9f1dd2715581d45ce98fa`. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Trigger map walked against the diff: a third-party image tag in a system package values file and its test; no app schema, platform values, names or tooling. Fixes #4852 ### Release note ```release-note fix(velero): velero-plugin-for-aws is bumped from v1.14.1 to v1.14.4, which no longer sends an empty x-amz-tagging header, so Velero backups work against S3-compatible stores that reject it, such as Cloudflare R2. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Updates** * Updated the AWS backup plugin used by Velero to version 1.14.4. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
cd1345e661 |
fix(redis): re-send the Sentinel auth-pass when the operator resets a Sentinel (#4833)
Fixes #4851 ## What this PR does Adds a downstream patch to the redis-operator image (`packages/system/redis-operator/images/redis-operator/patches/unauthenticated-sentinel.diff`) so that a Sentinel which lost its `auth-pass` is repaired by the operator instead of being reset forever. ### The failure After a node power cycle on a Cozystack v1.6.4 cluster (3 nodes, Talos, HA Redis with `authEnabled: true`), the Sentinel pod on that node came back monitoring the correct master address, but with no `auth-pass`. The master answered it with `NOAUTH`, it reported the master as `s_down,disconnected`, and `SENTINEL CKQUORUM mymaster` on it returned `NOQUORUM 1 usable Sentinels`. The operator logged `mismatch number of sentinels in memory. resetting` on every resync and sent `SENTINEL RESET *`. That does not touch `auth-pass`, so the Sentinel never recovered on its own: the instance keeps running with two working Sentinels out of three until someone deletes that pod. One more failure then leaves Sentinel without a majority. The mechanism, which I reproduced locally with Redis 8.6.2 (script and output in #4851): 1. The pod restarts in place (same pod, same IP), so the `sentinel-config-copy` init container writes the operator's initial `sentinel.conf` again: `sentinel monitor mymaster 127.0.0.1 6379 2` and no `auth-pass`. 2. The other Sentinels still know that address and keep sending their hello messages to it directly. A failover has raised the config epoch, so the restarted Sentinel adopts the current master address from a peer (`+config-update-from sentinel ... +switch-master mymaster 127.0.0.1 ... <current master>`) before the operator looks at it. 3. `CheckAndHeal` then finds the right address in `CheckSentinelMonitor` and skips `NewSentinelMonitor`, the only call that sends `SENTINEL SET <master> auth-pass`. 4. `INFO sentinel` reports `status=sdown`, so `CheckSentinelNumberInMemory` fails and `RestoreSentinel` sends `SENTINEL RESET *`. The reset keeps the empty `auth-pass`, and the cycle repeats. The same code is on `main`: freshworks-oss v3.3.5 sets `auth-pass` only in `MonitorRedisWithPort`, and none of the three patches Cozystack carries touch this path. ### The change `RestoreSentinel` sends `SENTINEL SET <master> auth-pass <password>` right after `SENTINEL RESET *` when the RedisFailover has auth enabled. It uses the same password `NewSentinelMonitor` uses (`k8s.GetRedisPassword`), the same TLS config, and the same master name (`rf.MasterName()`, so `disableMyMaster` is respected). - It only runs on the unhealthy path. `RestoreSentinel` is called only when a Sentinel's view is already wrong: its sentinel or replica count does not match, or its status is not `ok`. A healthy Sentinel is never touched. - It does not disturb a working Sentinel. `SENTINEL SET auth-pass` drops the Sentinel's links to the monitored instances, but the `RESET` just before it has already done that. - Without auth, `GetRedisPassword` returns an empty string and nothing is sent, so the behaviour is exactly as before. - Bootstrap mode goes through the same function and gets the same password `NewSentinelMonitorWithPort` already sends there. Patch contents: a `SetSentinelAuthPass` method on `service/redis.Client`, its mockery-style mock, the two-step `RestoreSentinel`, and `TestRestoreSentinel` (5 cases: auth off, auth on, reset error, unreadable Secret, `SET` error). The file name sorts after `tls.diff` on purpose: the patch is written on top of it (same functions, new `tlsConfig` parameters), and the Dockerfile applies `patches/*.diff` in glob order. If you prefer numbered patch names, I can rename it. Like `tls.diff`, this belongs upstream too: the same fix is open as freshworks-oss/redis-operator#90. This carried patch can be dropped once a release containing it is vendored here. ### Why not a quorum-checking liveness or readiness probe The RedisFailover CRD does allow `spec.sentinel.customLivenessProbe` / `customReadinessProbe`, so a `SENTINEL CKQUORUM` probe could be set from the chart. I looked at that first and decided against it: - A liveness restart does not repair this. It restarts the container, not the pod. `sentinel.conf` lives on an `emptyDir` that survives a container restart, and only the init container rewrites it, on pod start. The restarted Sentinel reads the same file without `auth-pass` and fails again, so the probe would restart it in a loop. - `CKQUORUM` does not reliably detect it. In the local reproduction the broken Sentinel still answered `OK 3 usable Sentinels`, because its peers reach it directly. On the cluster it answered `NOQUORUM`. What does show the problem every time is `status=sdown` in `INFO sentinel`, which the operator already reads. - It hurts in real outages. With two of three nodes down, a quorum-based liveness probe would restart the last Sentinel, and a quorum-based readiness probe would empty the `rfs-` Service, at the moment clients most need it. The operator already detects this state and already runs a repair for it. The repair just did not include the password. ### Testing done - Applied the four patches to a fresh v3.3.5 tarball in Dockerfile order (`git apply patches/*.diff`). Then ran the Dockerfile's test step: `go test .` and `go vet -tags integration .` in every directory the patches touch, all 11 pass. Also ran `GOOS=linux GOARCH=amd64 go build ./cmd/redisoperator`. I could not run `make image` because Docker was not available on my machine. - `TestRestoreSentinel` fails on 3 of its 5 cases without the `heal.go` change and passes with it. The existing `operator/redisfailover/...` and `service/redis` tests pass unchanged. - End-to-end against real Sentinels (Redis 8.6.2, local): master + replica with `requirepass`, three Sentinels configured the way the operator configures them, a `SENTINEL FAILOVER`, then one Sentinel restarted on the same port with the operator's initial config. Result: that Sentinel switched to the new master from a peer's hello and sat in `status=sdown`, `flags s_down,master,disconnected`. `SENTINEL RESET *` alone left it in `sdown`. The patched `service/redis` client (`ResetSentinel` then `SetSentinelAuthPass`) brought it to `status=ok,...,slaves=1,sentinels=3`. - The mock was written in the format mockery v2.45.0 generates; v2.45.0 itself does not build with Go 1.26+ (`golang.org/x/tools` v0.17.0), so I could not regenerate it mechanically. Backwards compatibility: no API or chart change. The operator image has to be rebuilt to pick up the patch, like the existing patches (the image digest in `packages/system/redis-operator/values.yaml` is stamped by the build). Instances without auth are unaffected. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Trigger map walked against the diff: one new file under `packages/system/redis-operator/images/redis-operator/patches/`; no values, schema, CRD, name or tooling change. ### Release note ```release-note fix(redis): the redis-operator now re-sends the Redis password to a Sentinel it resets, so a Sentinel that restarted in place (for example after a node reboot) and lost its auth-pass recovers on the next check instead of staying down until its pod is deleted. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Sentinel recovery now restores the Redis authentication password after a successful reset when a password is configured. * Recovery stops if the reset fails and reports errors encountered while retrieving or applying the password. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
gireassen
|
525814d399 |
fix: typo in monitoring-agents values.yaml
Signed-off-by: gireassen <80508012+gireassen@users.noreply.github.com> |
||
|
frolov.so
|
9b35fa0b48 |
docs(api): clarify kind-wide Helm timeout overrides
Timeout annotations are read from an ApplicationDefinition and therefore affect every Application of that kind. Describing them as per-Application hides their actual scope and the upgrade override precedence. Fixes cozystack/cozystack#4382 Assisted-by: LLM Signed-off-by: frolov.so <opsmon.sh@yandex.com> |
||
|
Fedor Batonogov
|
68cd1028bd |
feat(kubernetes-nodes): let CDI trust a self-hosted Talos image factory
talos.imageFactoryURL is documented for self-hosted factories and mirrors in air-gapped, rate-limited or flaky-egress environments. Such an endpoint is normally served with a certificate from an internal CA, and CDI — which trusts only its system store — refuses the import with "x509: certificate signed by unknown authority". The DataVolume never completes, no worker joins, and the release sits in install until it times out. DataVolume.spec.source.http.certConfigMap exists for exactly this, but the chart rendered source.http.url alone and no value reached the field. Add an optional talos.imageFactoryCA taking the CA in PEM form. When set, the chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references it from the worker DataVolume on the HTTP import path; a pool cloning a golden image via osImage.builtin builds no source.http and renders neither. When empty, nothing is rendered and the output is byte-for-byte what it was. CDI appends the bundle to the system pool rather than replacing it, so a release pulling from a publicly-trusted endpoint keeps verifying either way. The value is emitted verbatim into a ConfigMap readable by anything that can read the namespace, so the render fails closed on private key material and on anything that is not complete PEM certificate blocks. Those guards live in a new validate-only cozy-lib.tls.validateCACert helper that takes the caller's field name for its messages, and cozy-lib.tls.caCertSecret is refactored onto it, so the check exists in exactly one place. Assisted-by: LLM Signed-off-by: Fedor Batonogov <fedor@batonogov.ru> |
||
|
Yasar Rahman
|
8580a9d46d |
feat(redis): size the Sentinel pods separately from Redis
Every Sentinel pod gets the Redis pods' resources, so the three Sentinels of an instance reserve three times a Redis replica's memory while holding only their configuration and the failover state. On a t1.nano instance that is 3 x 128Mi while the VPA recommender's uncapped target for a Sentinel container is 16Mi, and the waste grows with every larger preset, because the dataset that drives the Redis sizing never reaches a Sentinel. sentinel.resources and sentinel.resourcesPreset follow the convention of the top-level pair, and sentinel.resourcesPreset falls back to resourcesPreset. Explicit Redis resources are not carried over once the sentinel block sets either key, since they were sized for the dataset. An empty sentinel.resources map counts as unset, as an empty top-level resources does. An empty sentinel block keeps today's sizing, so existing instances render unchanged and an upgrade does not roll their Sentinels. The Sentinel WorkloadMonitor reports the preset the Sentinels actually use. Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> Assisted-by: LLM |
||
|
Yasar Rahman
|
f6f879cf38 |
fix(postgres)!: refuse plaintext sessions on external instances
CloudNativePG offers TLS but its built-in last pg_hba rule, host all all all, also accepts plaintext, so a client outside the cluster that connects with sslmode=disable gets an unencrypted session. tls.enabled only decides whether the external hostname goes into the certificate and cannot prevent this. When external is true the chart now renders spec.postgresql.pg_hba with hostssl all all all scram-sha-256 followed by host all all all reject. CloudNativePG places user rules after its fixed rules and before its default rule, and the first match wins, so only TLS sessions are accepted. The pod network cannot be told apart from an external client, so in-cluster clients are held to the same rule; libpq's default sslmode=prefer already negotiates TLS. The new tri-state tls.required follows external when omitted. Setting it to false keeps accepting plaintext on an external instance and true requires TLS on an in-cluster one. The hostssl rule uses the same password method as CloudNativePG's own default rule (md5 before PostgreSQL 14, scram-sha-256 from 14), so existing md5 verifiers keep working on PostgreSQL 13. Rendering fails when postgresql.parameters.ssl is off while TLS is required, since hostssl would never match and every remote session would be refused. BREAKING CHANGE: external clients that connect without TLS are refused after the upgrade. Set tls.required to false to keep accepting them. Assisted-by: LLM Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> |
||
|
Aleksei Sviridkin
|
f4bde9e50d |
fix(backups): delete the original PVC of an in-place VM restore without keepOriginalPVC (#4672)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
An in-place VMInstance restore with `keepOriginalPVC: false` reports
`Succeeded` without restoring the disks, and leaves every disk it
touched unable to come back. `prepareForRestore` deletes each DataVolume
with Orphan propagation and leaves the PVC in place, on the assumption,
stated in the `deleteDataVolume` comment, that Velero overwrites it.
Velero does not overwrite a PVC. Its CSI restore action returns early
for a claim that already exists, so no DataDownload is ever created, and
with `existingResourcePolicy: update` the merge patch it attempts
instead is rejected because a bound claim's spec is immutable. All that
reaches the claim is the labels-only fallback patch, which also drops
the `cdi.kubevirt.io/allowClaimAdoption` annotation the resource
modifier added.
The claim left behind then blocks the disk for good. Orphan propagation
has removed its owner reference to the DataVolume, and it lacks the
`cdi.kubevirt.io/storage.populatedFor` annotation that
kubevirt-velero-plugin writes on the backed-up copy, so when the
HelmRelease resumes, the CDI webhook refuses the DataVolume it creates
with `Destination PVC <ns>/<name> already exists`, on every reconcile.
Deleting the disk through the API leaves the PVC behind, and a new disk
of the same name fails the same way.
Observed on a v1.6.3 cluster, for a VMInstance restore with
`keepOriginalPVC: false` whose disk was attached to the VM. The Velero
log for the restore, trimmed:
```text
level=info msg="Retrieved per-restore resource modifiers from configmap cozy-velero/restore-modifiers-..."
level=warning msg="PVC already exists and ExistingVolumeDataPolicy is not in-place restore. Skip restore this PVC." Action=PVCCSIRestoreItemAction PVC=tenant-datalab/vm-disk-e2edisk
level=info msg="Applying resource modifier patch on tenant-datalab/vm-disk-e2edisk"
level=warning msg="patch attempt failed for /v1, Kind=PersistentVolumeClaim tenant-datalab/vm-disk-e2edisk: PersistentVolumeClaim \"vm-disk-e2edisk\" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests and volumeAttributesClassName for bound claims"
level=info msg="backup/restore labels successfully updated for PersistentVolumeClaim tenant-datalab/vm-disk-e2edisk"
```
`velero restore describe --details` lists the PVC as `(failed)` and the
Restore is `Completed` with two warnings. Afterwards the claim had its
original creation timestamp and UID, no owner reference, no
`allowClaimAdoption`, no `populatedFor`, and the disk's HelmRelease was
`Ready=False` with the webhook error above. That cluster runs a Velero
build from `main`, whose wording is quoted above; the released v1.17.1
and v1.18.3 return early at the same point with `PVC already exists.
Skip restore this PVC.` (`pkg/restore/actions/csi/pvc_action.go`).
The fix deletes the PVC when `keepOriginalPVC` is false, only once its
DataVolume is confirmed gone so that CDI cannot recreate it, and holds
the Velero Restore until the claim has actually disappeared. A claim
still mounted stays `Terminating` behind `kubernetes.io/pvc-protection`,
and creating the Restore then would hit the same existing-claim path.
Velero then creates the PVC from the backup, the data mover fills it,
and the recreated DataVolume adopts it through `populatedFor`. A failed
deletion now fails the RestoreJob instead of letting it report
`Succeeded` over the old data. Before
|
