mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-23 03:27:50 +09:00
main
7628
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
215e582f03 |
fix(hack): keep the backup stack in the vminstance selective install (#5027)
## What this PR does `vminstance` chainsaw suite has two velero round-trips (`vmdisk-2-backup-roundtrip` and `vminstance-2-backup-roundtrip`), both wait for `cozy-default` BSL that `backupstrategy-controller` package renders together with platform Bucket. `hack/select-install.sh` keeps this package only for database and broker suites ([#4022](https://github.com/cozystack/cozystack/pull/4022)), so PR with selection of exactly `vminstance` gets platform without it: no Bucket, no BucketClaim, no BSL, and both round-trips fail on BSL wait after about 190 seconds, before any step of example scripts runs. Seen on the [E2E run of #4955](https://github.com/cozystack/cozystack/actions/runs/39225333408), selected apps there are `vminstance` only and the backup packages are in disabled list. Any edit of `examples/backups/vmi` or `examples/backups/vmdisk` selects `vminstance` and nothing else, so PRs that change those scripts can't run them. Fix is one more entry in `suite_runtime_sources`, the table that already keeps the stack for database suites. Install closure of `vminstance` grows by `backup-controller`, `backupstrategy-controller`, `bucket-application`, `velero`, `metrics-server` and `monitoring-agents`. Bats test for backup suites now includes `vminstance` and fails without the script change. `hack/select-install_test.bats` and `hack/select-e2e_test.bats` pass (110 tests), `hack/select-install.sh --validate` is clean. Own E2E run of this PR doesn't cover the fix, because `hack/*.sh` escalates to full suite and database suites install the backup stack anyway. Real check is re-run of E2E for [#4955](https://github.com/cozystack/cozystack/pull/4955) after merge, it selects `vminstance` alone. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff, it is one selector table entry and its bats test, no file under `hack/` moved or renamed and no make target changed. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(hack): E2E install for `vminstance`-only selection keeps the backup stack, so VMInstance and VMDisk backup round-trips run instead of timing out on BSL wait. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit - **Bug Fixes** - Virtual machine instances now correctly include the backup strategy component required for backup-related functionality. - Installation checks now consistently validate this requirement, including when backup features are disabled. This improves setup reliability and helps prevent incomplete or misconfigured virtual machine deployments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
d78897f842 |
fix(cluster-api): resync capi-operator every minute (#4939)
## What this PR does After an upgrade restarts capi-operator, a Cluster API provider can keep `Ready=False` for about 20 minutes while its controller is already running, and `capi-providers-core` times out waiting for the CoreProvider. The bundled operator v0.19.0 drops create events of provider Deployments it finds running after a restart (kubernetes-sigs/cluster-api-operator#1031), so it looks at them again only when one changes or on informer resync, and the first resync after a restart comes two sync periods later, about 20 minutes with the default 10m. This sets the operator `--sync-period` to 1m. On a test cluster a provider left `Ready=False` across an operator restart recovered 2m02s after the operator took its lease, the operator with the default period had not recovered after more than 4 minutes. A resync is served from the operator cache and writes nothing while providers are in sync: no provider object was updated over five resyncs, about 15 log lines a minute, 2m CPU and 26Mi memory. The vendored chart has no value for this flag, so a small package patch (reapplied by `make update`) adds one, and a test holds the rendered argument. The upstream fix is in cluster-api-operator v0.27.0, bumping to it is a bigger change and is left for a later release. Mitigates #4926. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note fix(cluster-api): resync capi-operator every minute so a provider left not ready across an operator restart recovers in about two minutes instead of about twenty ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * The Cluster API operator now refreshes its cached state every minute. After an operator restart, provider readiness status can recover sooner instead of remaining stale for a longer period. * Providers that are already in sync are not written back to the API during these cache refreshes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d24b92fe1d |
docs(agents): define where a new document belongs (#3777)
## What this PR does `docs/` had collected component guides that belong on the website, standalone oidc, gpu and vm networking documents written inside the feature PRs that added the code and never routed anywhere a user looks. Nothing said where documentation should go, and in particular nothing mentioned that the website appends a whole package README onto a published page, so a contributor with package scoped prose had no reason to use that route. This replaces the four line "Documentation is organized as follows" list in `docs/agents/overview.md` with a destination table and the test that separates `docs/` from the website: whether a non-documentation file has to link to the document by path. It also says that a merged document records current behaviour and its limitations, not the development process around it, which is the "What is NOT in Phase N" and "this PR" residue that motivated the whole thing. One such residue dropped while here, `docs/operations/backup-classes.md` described a pin as one "this PR removes". Moving the documents themselves is a separate PR. ### Downstream repositories - [x] No downstream repository is affected by this change Walked the trigger map against the diff: this touches `docs/agents/overview.md` and `docs/operations/backup-classes.md` only. No package added, renamed or removed, no `values.yaml` or `values.schema.json` field, no `packages/core/platform/values.yaml` key, no bundle or variant, no release asset name, no `ApplicationDefinition` semantics, no Talos pin, and no change to `cozyvalues-gen`, `cozypkg` or a package Makefile. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Documentation** - Added guidance for choosing where new documentation belongs, including package READMEs, the website, repository docs, API design documents, and community resources. - Clarified package README publishing and sync conditions, when repository documentation is warranted, and where to document feature limitations and development-process notes. - Clarified that documents referencing unmerged pull requests or unreleased dependencies need a tracking issue. - Updated the CloudNativePG upgrade note to explain that the CRD version skew came from an image tag pin outrunning the vendored chart CRDs; the pin has since been removed. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
9a782f697a |
fix(hack): keep the backup stack in the vminstance selective install
The vminstance Chainsaw suite carries the VMInstance and VMDisk Velero round-trips. Both build the cozy-default BackupStorageLocation and the platform bucket from the backupstrategy-controller package, but the selective install retained that package only for the database and broker suites. A selection of vminstance alone therefore installed a platform with no Bucket, no BucketClaim and no BSL, and both round-trips timed out on the BSL wait before running a step of the example scripts. Every pull request whose selection is exactly vminstance hits this, and that includes each edit to examples/backups/vmi and examples/backups/vmdisk, which select that suite on purpose. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
3cd261277e |
fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit (#4941)
## What this PR does Fixes a size regression from #4929. Helm stores each release revision in one Secret and the apiserver refuses a Secret over 1 MiB. With all default Strategy CRs rendered the `backupstrategy-controller` release crosses that limit, so the forced upgrade that creates them fails with `Secret "sh.helm.release.v1.backupstrategy-controller.vN" is invalid: data: Too long: may not be more than 1048576 bytes`, the CRDs stay and the default strategies never appear. The release was already within about 1 KB of the limit before #4929, which added tests to the chart. The strategy CRDs are 2.2 MB of YAML and Helm stores them twice per revision, as chart files and in the rendered manifest. Almost all of it is the upstream `PodTemplateSpec` schema inlined by the Altinity, Job and FoundationDB strategies. Kafka, RabbitMQ and Redis already mark their `Template corev1.PodTemplateSpec` as `Schemaless` and `PreserveUnknownFields`, this PR does the same for `AltinitySpec.Template`, `JobSpec.Template` and `FoundationDBTemplate.BackupDeploymentPodTemplateSpec`. Go types are unchanged and every description is kept. The CRDs go from 2.2 MB to 317 KB and the largest release revision from over the limit to about 250 KB, 24% of it. The size rationale now lives once in the package doc, the field docs of the six schemaless templates share one short line. What changes for users: the apiserver no longer validates these three fields and `kubectl explain` no longer lists children under them (`kubectl explain pod.spec` still documents them), same as for Kafka, RabbitMQ and Redis today. Unknown keys are stored instead of pruned, the typed decode in the drivers drops them either way. Labels and annotations under `template.metadata` were rejected by the strict v1.6 schema and are now accepted and kept. A Go template in a non-string slot never worked, it was rejected on apply, now the apiserver accepts it and the typed decode in the controller is what rejects it, as for the other three kinds (I did not run that case live). The drivers read these fields through the non-strict client and `internal/template` rewrites only string leaves, nothing under `internal/backupcontroller` relies on pruning. Go tests for `internal/backupcontroller`, `internal/template` and `api` pass. `hack/strategy-crd-schemaless.bats` fails on any strategy CRD that inlines a pod schema and prints the property path and the two markers to add. `hack/helm-release-size.bats` estimates the stored release of every chart under `packages/` the way Helm encodes it (release JSON, gzip, base64) and fails above 90% of the limit, plus the worst case of this chart. The estimate runs 1 to 2 percent above real Secrets. A chart has to render offline to be measured with its manifest: the 36 charts that do not are listed in `NO_OFFLINE_RENDER` and measured on files and templates only, any other render failure fails the test, and so does a listed chart that starts rendering. Checked on a throwaway kind cluster: v1.6.4 upgrades to this branch and the forced follow-up creates all default strategies, a fresh install works, strategy objects created under the old schema stay valid, a Job strategy with templated args and pod template labels is accepted. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note fix(backups): the backupstrategy-controller release no longer fails with "Secret ... Too long" on the upgrade that creates the default backup strategies, the Altinity, Job and FoundationDB strategies store their pod templates schemaless like Kafka, RabbitMQ and Redis ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Backup strategy templates for Altinity, FoundationDB, Jobs, Kafka, RabbitMQ, and Redis now preserve fields not defined in the API schema. The API server no longer validates these templates against the pod schema; validation occurs when the driver creates the workload. This lets strategies include pod-template fields without requiring those fields to be listed in the strategy schema. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
c6fcb4e7c6 |
fix(ci): dispatch frozen-line builds with GITHUB_TOKEN (#4888)
## What this PR does The first RC freeze hit HTTP 403 when dispatching `build-release.yaml` because the CI app has no Actions write permission ([run](https://github.com/cozystack/cozystack/actions/runs/38229949424)). I switched this dispatch to `GITHUB_TOKEN` with job-scoped `actions: write`. Tag and branch pushes still use the app token. Verified with `make unit-tests`, zizmor 1.26.1 and the commit trailer check. The regression test fails on the old workflow and passes with the fix, including warning and manual recovery on dispatch failure. Live dispatch with `GITHUB_TOKEN` has not been tested. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(ci): use GITHUB_TOKEN to start the initial packages build when freezing a release line ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated release automation to use appropriate permissions and authentication for CI triggers. * Improved warnings and recovery guidance when automated package builds cannot be started. * **Tests** * Added automated checks covering release dispatch permissions, branch targeting, successful execution, and non-blocking failure handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
2e4783600b |
docs(release): point changelog check B at the Talos installer profile (#4959)
## What this PR does Check B loops over three values files and the first one, `packages/core/talos/values.yaml`, is an empty file, so nothing is ever printed for Talos. Talos version is `version` in `packages/core/talos/images/talos/profiles/installer.yaml` (package Makefile reads it as `TALOS_VERSION`), this PR points the loop there. I ran old and new loop on real tags. On `v1.5.4..v1.6.0` the Talos bump from `v1.13.0` to `v1.13.6` was not shown before and is shown now, on `v1.6.1..v1.6.2` and `v1.6.4..v1.6.5` output is the same. Found it while reviewing #4952, which fixes the Dockerfile pathspec right below the loop. The two PRs merge cleanly in any order. I left `head -40` alone, it also hides the new `talos.version` block of `packages/apps/kubernetes/values.yaml` on `v1.5.4..v1.6.0`, but that is a separate decision. ### Screenshots N/A: documentation-only change, no UI impact. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note docs(release): point changelog check B at the Talos installer profile, a Talos version bump was not shown by it before ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated changelog verification to check the installer configuration; checks for installer and Kubernetes values remain unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
a6cb84673e |
docs(release): check changelog claims against the code (#4936)
## What this PR does Changelog generator sees only `docs/agents/changelog.md`, two inline prompts and `AGENTS.md`. Failure modes and pre-publish checks live in `docs/release.md`, which it never reads. The prompt has detailed rules for authors and PR numbers and none for checking a claim against the code. I reviewed the generated v1.7.0 draft against the code and found 22 statements wrong or missing, while all authors and PR references were right. Several were in breaking changes: a talm fix credited to a talm release that doesn't contain it, worker pools said to be adopted without churn while Talos workers are rolled once because of the hash-named TalosConfigTemplate (#3523), migrations said to run on a variant that has them disabled. Three changes: - `docs/agents/changelog.md`: new section 8b about verifying claims against the code. Identifiers, migration behaviour and variant scope come from the tree at the upper-bound tag, required actions come from release notes and newly added upgrade sections, not only from `!` commits. Section 9 now asks to run the `docs/release.md` checks and `hack/validate-changelog.sh` on the written file before exit. - `.github/workflows/changelog-generate.yaml`: refresh `docs/release.md` and `hack/validate-changelog.sh` from the dispatch ref, next to the existing `changelog.md` refresh. Job checks out the rc tag detached, so without this generator runs older copies from that tag. Permissions, tokens and tool flags are unchanged. - `hack/validate-changelog.sh`: reject an entry cut off with an ellipsis right before its attribution, and any link to `cozystack.io/docs/next/` (live site answers 404). Reason names the offending lines in one line, because workflows quote it into a single annotation. How I checked it: - Old validator accepts the original v1.7.0 draft, new one rejects it on three truncated entries (lines 335, 393, 518) and, after those are repaired, on the `/docs/next/` link (line 25). All 29 shipped changelogs from v1.3.0 to v1.6.4 still pass. - New contract test plants each defect into a shipped changelog, checks the reason names the line, and checks that near misses real changelogs carry still pass (`content/en/docs/next/` in prose, ellipsis inside a sentence, three dots of a compare link). It goes red on the old validator and on widened patterns. - Overlay test pins both new copies and the cone-mode sparse checkout that brings `docs/release.md` in. It goes red if either copy is removed, cone mode is turned off or `docs/agents` is dropped from sparse paths. - I ran every command the new prompt rules prescribe on v1.6.0..v1.7.0-rc.1, each one finds an error the draft shipped. - `make bats-unit-tests bats-posix-compat-tests` pass (2424 and 1120), zizmor and actionlint are clean on the workflow. Not included: a script that gives the generator component versions and migration headers, and a networked check for references, authors, link liveness and companion-release containment. Both are separate changes. Things I noticed and left alone: - validator summary in the Phase 2 section of `docs/release.md` doesn't list the two new rejections - `docs/agents/changelog.md` runs `docs/changelogs/hack/check-optional-repos.sh`, but the script lives at `hack/check-optional-repos.sh` and still resolves repo root from the old location - a changelog the validator rejects is still not uploaded as an artifact, so rejection costs a regeneration or a hand-commit, that is why section 9 has the generator run the validator first - current v1.7.0 draft still has the three truncated entries, a promotion that picks it up will discard it and generate a new one unless the entries are rewritten - three unrelated bats tests (in `check-gpu-recording-rules.bats` and `promote-rewrite-tags_test.bats`) fail under en_US.UTF-8 collation on main too, they pass under C.UTF-8 that runners use, I ran the suite that way ### Screenshots Not applicable, there is no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: I walked the trigger map in `docs/agents/contributing.md` against each changed file: - `docs/agents/changelog.md` is the changelog prompt, not the commit convention or the PR template that ccp and community restate. Its list of release-window repositories is unchanged. - `.github/workflows/changelog-generate.yaml` is not `tags.yaml`, the release-prep workflow ccp relies on. - `hack/validate-changelog.sh` is changed in place, not moved or renamed, and it is not `hack/package.mk` or `hack/update-crd.sh`. - `hack/release-changelog-contract.bats` is a test. No package, values key, CRD, release asset, make target or node prerequisite changes. Code search across the organization finds no reference to any of these files outside this repository. ### Release note ```release-note docs(release): check changelog claims against the code. The changelog generator now verifies operator-facing statements against the code and runs the release checks on its own output, and the changelog validator rejects entries cut off with an ellipsis and links to cozystack.io/docs/next/. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Release guidance now emphasizes verifying operator-facing claims against the release version and carrying required actions into upgrade instructions. * Documentation links in changelogs should point to versioned documentation. * **Chores** * Changelog validation now flags truncated entries and links to unversioned “next” documentation. * Release guidance and validation checks are refreshed automatically during changelog generation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
f69958b15d |
docs: move the operator guides out of docs/ to where readers find them (#3780)
## What this PR does
Five documents under `docs/` were operator facing guides written
alongside the feature PRs that added the code, and left in the source
tree where no user looks. They are on the website now, so this deletes
them here and repoints every reference.
```
docs/oidc-tenant.md -> kubernetes/oidc-authentication
docs/oidc-grafana.md -> operations/services/monitoring/oidc-authentication
docs/gpu-vgpu.md -> virtualization/vgpu
+ operations/troubleshooting/gpu-operator-host-driver
docs/vm-external-vlan.md -> networking/vm-external-vlan
docs/hubble-observability.md -> networking/hubble
```
`docs/vm-external-vlan.md` needed splitting, not moving. Its "Staging
the reference CNI plugins" section is a contract,
`hack/cni-plugins-staging-contract.bats` greps six literal sentences out
of it and asserts them against the multus Dockerfile, the daemonset
manifest and the platform values. That half is package documentation so
it moves verbatim into a new `packages/system/multus/README.md` and the
suite reads it there, the operator guide goes to the website. The bats
messages name the README now, so a future failure points at the file
that has to change.
References repointed in the `@param` blocks, template comments and test
comments that named the guides. The generated files come from `make
generate` with cozyvalues-gen v1.7.0, the version pinned in
`.github/workflows/pre-commit.yml`. Where a reference points at text
that exists only on `next/` of the website (the aggregated API servers
section of the tenant OIDC guide), the comment names the page and the
section instead of a `v1.6` URL, which would not contain it.
Two changelog links in `docs/changelogs/v1.6.0.md` repointed as well.
The wording is untouched, only the location of the documents it names
has changed.
One bit of process framing dropped on the way past, per #3777:
`bundles/system.yaml` said GrafanaAdmin promotion was "out of scope for
Phase 1" when it is simply not implemented.
Tests: 10 cni-plugins contract, 47 install-wait, 346 kubernetes chart,
22 multus chart, 301 monitoring chart, rd-presets-check.
### Downstream repositories
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: cozystack/website#648 (the four guides),
cozystack/website#754 (edits made to the originals in `docs/` after the
import), cozystack/website#647 (grafana guide made reachable in v1.6),
cozystack/website#596 (tenant kubectl guide, rebased)
Walked the rest of the trigger map against the diff. No provider
follow-up: the kubernetes `values.schema.json` diff is two `description`
strings and the monitoring one is one, with no field added, removed or
renamed, no enum, no default, and no `kind`, `plural` or
`release.prefix` change. `packages/core/platform/values.yaml` changed in
a comment only and no key, so the hand written platform-package table on
the site is unaffected.
Please merge after cozystack/website#754 and cozystack/website#647. The
`@param` URLs only resolve once those pages deploy, and
cozystack/website#754 carries the edits made to the guides since the
import, which deleting them here would lose.
### Release note
```release-note
docs: operator guides for tenant Kubernetes OIDC, Grafana OIDC, NVIDIA vGPU, VM external VLAN and Hubble now live on cozystack.io instead of the docs/ directory of this repository, and in-repo references point there. The CNI plugin staging contract that shared the VLAN guide moved to packages/system/multus/README.md.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated configuration and test references to link to the published
Cozystack documentation.
* Added Multus guidance covering staged CNI plugins, configuration,
troubleshooting, and rollout behavior.
* Removed outdated local guides for GPU/vGPU, Hubble observability, OIDC
authentication, and external VM VLANs.
* Clarified that server-level Grafana administrator promotion is not
currently supported.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
myasnikovdaniil
|
1f92bcd20e |
fix(fluxcd): close the flux-plunger review follow-ups for the hook, drain and Warnings (#4914)
## What this PR does Follow-up to the flux-plunger uninstall fence: the points the last review round raised and left for after the merge, plus four found by a later independent review. None changes the fence's design; each closes a gap in what it reports, how long it waits, or what its documentation says. **Pre-delete hook.** - A release being deleted that only the fence still holds is no longer unfenced in the same step that lifts its suspension. helm-controller runs the uninstall of a release being deleted once it is no longer suspended, but removing the fence at once deleted the object before helm-controller could reconcile it. The hook now keeps that fence while helm-controller handles the generation the lifted suspension wrote, until shortly before its retry window closes, and names a release it still had to let go of as before. A release whose state cannot be read keeps its fence for the wait the same way, rather than being let go of on one failed read. - A release being deleted while still suspended, by anyone but flux-plunger, was reported as cleanly released, although helm-controller skips the uninstall of a suspended release. It is now named as deleted without an uninstall and counted with the others. - Each release gets its own share of the retry window, capped at the window's end. One release that could never be read used to take the whole window and leave every release listed after it a single attempt. The share is a cap, not a floor of attempts: a floor would let enough failing releases run the Job past its deadline, which kills it before its report is written. - The hook stops, waits for and lists again only for the flux-plunger of its own release. Another one (a second installation under another release name, changed labels, a binary outside the cluster) is not stopped by the hook, keeps the fences it sets and gives them up itself; the README now says so instead of the hook re-listing unconditionally, which would not close that case and would fail the uninstall for as long as a second installation is recovering anything. - helm-controller's finalizer, which the hook reads to tell whether a release being deleted is still held, is tied to the constant the controller uses, as the fence already was. Two comments are corrected: the retry window counts from the script's start, not the Job's, and kubectl scale's internal conflict retry never runs here. **Shutdown drain.** - A drain deadline of zero or less is refused at startup, and the chart's lowest accepted grace period is held to the same constants as the drain subtraction, so raising the manager's stop bound cannot make the chart render a deadline that is already spent. The timings the README and the values comment state are held to those constants too. - Fences left behind are reported as an error naming each release, not an Info line; a drain that never got a listing answered says so instead of reporting a count of -1; a flux-plunger that never led no longer claims it released every fence. - The drain's Events fold repeats into one counted Event per release and reason, as the manager's recorder does, instead of one object per pass. The fold is a patch, the verb the chart grants on events, and every write is bounded by the drain's deadline. It does not add a per-release Event for fences left at the end: by then the drain's deadline is spent, and writing one would reach past it. - Tests now hold the outcome line, the retry of a release whose fence cannot be given up yet, and the pause between passes, none of which any suite noticed when removed. **Controller.** The `SuspensionNotOwned` and `TerminatingWhileSuspended` Warnings were keyed on the processed-version annotation, which stays on every release flux-plunger ever recovered, so a later restore, migration or operator suspension of a healthy release drew a Warning telling the reader to resume it, and a migration's deliberate suspend-then-delete was called an accident. Both now key on the state a leftover actually freezes the release in, `Ready=False` with `has no deployed releases`, since helm-controller rewrites no condition of a suspended release; the README's candidate query follows. Two comments that claimed more than holds are qualified. **Chart and README.** A `shutdownGracePeriodSeconds` of a million or more now gets the range message rather than being refused as a fraction. The README gives the recovery steps in the order the controller runs them (the annotation is written before the Secret is deleted), and names the window in which a backup restore started during a recovery is unsuspended under it, since the etcd, CNPG and Velero restore steps write `spec.suspend` only when it differs. ### Testing Each change was written test first: a test that builds the binary and runs it with a deadline of zero, helm-unittest for the chart, unit tests with a captured logger and the fake client for the drain's outcome, Events and passes (the Event fold under a client that refuses the update verb, as the chart's role does), table tests for both Warnings, and the bats suite for the hook, whose fake kubectl now models helm-controller handling a release some reads after its suspension is lifted and a state read that fails. Every fix was then reverted on its own, and the suite went red each time. The envtest suites pass unchanged. The README's candidate query was run against HelmReleases read from a live cluster, with a stuck, a healthy, an owned and an unsuspended variant, and lists only the stuck one. ### Screenshots Not applicable: no UI change. ### Downstream repositories The trigger map was walked against the diff: `internal/controller/fluxplunger`, `cmd/flux-plunger`, the `packages/system/flux-plunger` chart (templates, tests, README) and its bats test under `hack/`. The website's FluxPlunger section in `guides/concepts.md` draws the recovery with the Secret deleted before the annotation is written, which is the order before the fence change; it needs the same correction as the chart README. That change is held as a draft until v1.7.0 is released and the website's `next/` is re-initialized: the fence change is not part of release-1.7, so the corrected order would be wrong for v1.7 docs, which are copied from `next/`. No other repository is reached by this diff. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/pull/752 (draft, merge after v1.7.0) - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(fluxcd): flux-plunger's pre-delete hook now lets helm-controller run the uninstall of a release being deleted before it removes the release's fence, gives each release its own share of its retries, and reports releases deleted under another manager's suspension; the shutdown drain reports the fences it leaves as errors and refuses a deadline that leaves no time ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Shutdown drain timeouts must be positive, and invalid values now stop startup with a clear error. * Uninstall cleanup waits for Helm to handle releases before removing their fences, while bounding retries and reporting releases that remain incomplete. * Recovery and suspension warnings now reflect the release’s actual failure state, and recovery can retry stale storage cleanup against the same revision. * **Documentation** * Clarified shutdown grace-period requirements, uninstall behavior, and guidance for backup restores and existing suspensions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
d1d4e92acb |
feat(release): sign the release assets and attach a source SBOM (#4861)
## What this PR does Implements the release-asset part of #4759: a release cut from a tree that has this change gets an SBOM of its source and one keyless signature that covers all of its assets. These are the two CLOMonitor checks still failing for this repository, `sbom` and `signed_releases`, after #4757 and #4899 brought the project score to 96.69. `hack/upload-assets.sh` now collects its asset list into an array and hands it to the new `hack/sign-release-assets.sh` before the first upload. The script writes three files into `_out/assets`, and they are uploaded with the rest: | File | What it is | |---|---| | `cozystack-source-sbom.spdx.json` | SPDX 2.3 SBOM made with syft, over `git archive HEAD` (the committed release tree, without `_out/`, `.release-tooling` or runner leftovers) and the unpacked `cozypkg-*.tar.gz` binaries. It does **not** describe the container images that `packages/` pins by digest (their SBOMs are #2937, A2 in #3278) or the Talos ISO, disk, kernel and initramfs images. It is named by what it covers, so `cozystack-sbom` stays free for an SBOM of the release as a whole. CLOMonitor's `sbom` check matches `sbom` in an asset name. | | `cozystack-checksums.txt` | SHA-256 of every uploaded asset and of the SBOM. | | `cozystack-checksums.txt.sigstore.json` | Sigstore bundle of `cosign sign-blob` over the checksums, keyless, signed with the job's GitHub OIDC identity. | Before it returns, the script runs the same `cosign verify-blob` that `docs/release.md` gives users, with `--certificate-identity-regexp '^https://github\.com/cozystack/cozystack/\.github/workflows/(tags\.yaml@refs/tags/v|promote-rc\.yaml@refs/heads/main$)'`: `tags.yaml` on a `v*` tag push, `promote-rc.yaml` dispatched from `main`. A signature that check would reject therefore fails the job before anything is uploaded. It also refuses its own three outputs as inputs: `promote-rc.yaml` downloads every rc asset, and a stable release has to get freshly computed checksums and a fresh signature, since promotion restamps several assets. Workflows: `tags.yaml::prepare-release` and `promote-rc.yaml::promote`, the two jobs that call `make upload_assets`, get `id-token: write` and two setup steps: - syft v1.54.1 from its release tarball, checked against a committed SHA-256 (`c069905b…`, the release's checksums entry; the checksums file verifies against anchore/syft's keyless release signature). Not `anchore/sbom-action/download-syft`: pinned by commit, it still runs syft's `install.sh` fetched from a tag at run time, in a job that holds the app token, the GHCR login and `id-token: write`. v1.54.1 is past GHSA-mw2c-m758-9v5q (fixed in v1.52.0). `hack/ci-tool-installs-contract.bats` now fails on `anchore/sbom-action`, on a syft install script, on a syft download without `sha256sum -c` and on version or checksum drift between the two workflows. - `sigstore/cosign-installer` v4.1.2, pinned by SHA, with `cosign-release: v3.1.3`: the version `docs/release.md` asks users for, so the job checks its own signature the way they will. The action verifies that binary against cosign's keyless and KMS bundles with its digest-pinned bootstrap cosign. `hack/ci-tool-installs-contract.bats` holds the action commit and the release equal in both workflows and fails on a step without `cosign-release`. `docs/release.md` gets a "Release signatures and SBOM" section with the user-side verification: cosign v3.1.3 or later (v2.6.5 on the v2 line), the two commands joined with `&&`, what the SBOM covers and what the certificate identity proves. It now also says, at each place `promote-rc.yaml` is run, that it is dispatched from `main` only. ### Decisions for the release owners - **Fail closed.** If Fulcio or Rekor is down, the release job fails and nothing is uploaded. I chose that so that no release goes out unsigned. If you would rather publish unsigned than block, the change is a `|| echo ::warning::` around the call in `upload-assets.sh`. - **Promotion runs the rc tree's script.** `promote-rc.yaml` calls `make upload_assets` from the rc checkout, not from `.release-tooling`. The first signed stable release is therefore the promotion of the first rc tagged after this merges. `release-1.6` gets signatures only if this is backported. CLOMonitor reads the latest release, so the score moves only when that release carries the files. - **Certificate identity pins the refs** (decided in review): `tags.yaml@refs/tags/v…` and `promote-rc.yaml@refs/heads/main`. A `promote-rc.yaml` dispatched from another branch fails the job's own check at the upload step, after the staging branch, the `promotion-*` candidate and the empty draft release exist; nothing in `parse` refuses a non-`main` dispatch up front. Until the `v*` tag rulesets exist, a hand-pushed `v*` tag whose tree has an edited `tags.yaml` still signs, which `docs/release.md` says. - **Follow-ups after the first signed release** (#4759 stays open for them): - `project.documentation.signature-verification` in `security-insights.yml` (#4757); - the SBOM row in `docs/security/self-assessment.md`, which says "Not published at this time"; the image caveat above stays in it. I left both alone so that they do not describe releases that don't exist yet. ### How it was checked - Bats 1.14.0 (the version CI installs), with gh, syft and cosign stubbed: - `hack/sign-release-assets.bats`, 8 tests: what the checksums cover; that syft sees the committed tree and the cozypkg binaries but not untracked files; that the verify identity accepts `tags.yaml` on a `v*` tag and `promote-rc.yaml` on `main`, and rejects a fork, another workflow, `tags.yaml.evil`, `tags.yaml` on a branch, and `promote-rc.yaml` on `release-1.6`, on `main-evil` and on a tag; that `docs/release.md` shows the same regexp as the script; that a failed verification fails the script; that it refuses a missing asset, assets split over two directories (with a same-named file in both, so the test fails if the directory check is removed) and its own outputs as input. - `hack/upload-assets.bats`, 4 tests, new: the exact upload list and order (the assets, then the SBOM, the checksums and the bundle); that the checksums name every upload but the last two; zero `gh` calls when verification fails or an asset is missing. The old `upload-assets.sh` fails all four, a version that signs after uploading fails the last two. - `hack/ci-tool-installs-contract.bats`, 8 tests. Putting `download-syft` back, dropping the `sha256sum -c` line, letting `SYFT_VERSION` drift between the workflows, dropping `cosign-release` from one step, letting it drift, unpinning the installer or giving one step another installer commit each fails it. - Putting the old any-ref regexp back in the script fails the identity test and the docs tie; putting it back in the docs alone fails the tie. - The meta-suites `bats-strict-setup`, `bats-no-negated-assert`, `bats-no-exit-trap`, `bats-runner-coverage` and `bats-runner-contract` pass: 90 tests across these eight files. - The pinned regexp gives the same accept/reject results under `grep -E` and Go's `regexp` (what cosign uses). cosign-installer's install script at the pinned commit, run as written with `cosign-release: v3.1.3`, verifies the binary's keyless and KMS bundles and installs v3.1.3 (its SHA-256 matches `cosign_checksums.txt`). `--yes`, `--bundle`, `--certificate-identity-regexp` and `--certificate-oidc-issuer` are listed, not deprecated, in cosign v3.1.3's help. - The `Set up syft` step's script, run as written with `RUNNER_TEMP`/`GITHUB_PATH` set: the checksum passes and `syft version` reports 1.54.1. A real syft v1.54.1 run through `hack/sign-release-assets.sh` (cosign stubbed) over this tree and the v1.6.5 `cozypkg-linux-amd64.tar.gz` took 10 s and wrote an SPDX 2.3 document with 920 packages: 440 golang, 373 npm, 103 github, 4 without a purl; 60 come from the cozypkg binary. - cosign behaviour behind the docs, with a key-signed legacy bundle whose `cert` holds the bare public key, against the documented identity regexp (tlog ignored, since an attacker can upload their own entry): v2.6.4 and v3.1.2 print "Verified OK", v2.6.5 and v3.1.3 reject it. `--new-bundle-format` makes v2.6.4 reject it but not v3.1.2, so the docs give the minimum version and not the flag. A real keyless protobuf bundle (syft's) verifies on v2.6.5 and v3.1.3. - zizmor 1.26.1 `--offline` reports no findings. actionlint 1.7.12 (with `.github/actionlint.yaml`) reports the same 13 pre-existing info-level shellcheck notes in `tags.yaml`/`promote-rc.yaml` as before, nothing new. shellcheck passes on both scripts. `hack/check-commit-trailers.sh` passes. - Not checked: a real keyless signature. It needs the OIDC token of a job in this repository, so the first rc after the merge is the end-to-end test. Cutting an `-alpha.N` first would exercise it without touching an rc. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: I checked the trigger map file by file. The only entry `hack/upload-assets.sh` can trigger is renaming a release asset, because the website install pages hardcode asset names. This PR adds three assets and renames none of the released ones, `cozypkg-checksums.txt` included (the SBOM was renamed to `cozystack-source-sbom.spdx.json` in review, before any release carried it). The workflow and docs changes match no entry. ### Release note ```release-note feat(release): releases carry cozystack-source-sbom.spdx.json, an SPDX SBOM of the source tree and the cozypkg binaries (not of the container or Talos images), cozystack-checksums.txt with the SHA-256 of each asset, and cozystack-checksums.txt.sigstore.json, a keyless Sigstore signature over the checksums made by the release workflow; docs/release.md shows how to verify a download with cosign v3.1.3 or later. ``` Refs #4759 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Releases now include an SPDX software bill of materials, SHA-256 checksums, and a keyless Sigstore signature bundle. * Release assets are verified against the expected workflow identity before upload; uploads stop if verification fails. * **Documentation** * Added guidance on release signatures, SBOM coverage, asset verification, and which branch dispatches release workflows. * **Tests** * Added coverage for generated release files, upload behavior, and signing and verification failures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Marian Koreniuk
|
bc24b588b2 |
feat(release): sign the release assets and attach a source SBOM
No release carried a signature or an SBOM, so users could verify none of the ISO, disk images, kernel, initramfs, manifests and cozypkg binaries they download, and the OpenSSF Scorecard Signed-Releases check and CLOMonitor's signed_releases and sbom checks failed. hack/upload-assets.sh now hands its asset list to the new hack/sign-release-assets.sh before the first upload. That script writes cozystack-source-sbom.spdx.json with syft over the committed tree and the cozypkg binaries, cozystack-checksums.txt with the SHA-256 of every asset and the SBOM, and cozystack-checksums.txt.sigstore.json with a keyless cosign signature over the checksums. The SBOM does not describe the container images or the Talos boot images, which is what its name says; image SBOMs are #2937. The script verifies the bundle against the identities of tags.yaml on a v* tag and promote-rc.yaml on main before returning, so a bad signature fails the job with nothing uploaded, and hack/upload-assets.bats pins the upload list and that order. The two jobs that call make upload_assets get id-token: write, cosign v3.1.3 from the SHA-pinned cosign-installer, and syft from its release tarball checked against a committed SHA-256, because anchore/sbom-action/download-syft runs an install script fetched at run time; hack/ci-tool-installs-contract.bats now covers both. docs/release.md describes the files, what the SBOM covers, the minimum cosign version to verify a download with, what the certificate identity proves, and that promote-rc.yaml is dispatched from main only. Refs #4759 Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Myasnikov Daniil
|
cc7f534f5e |
docs(release): point changelog check B at the Talos installer profile
packages/core/talos/values.yaml is an empty file, so the first path of the check B loop never produces a diff and a Talos bump goes unnoticed by the release verification. The Talos version is the version field of packages/core/talos/images/talos/profiles/installer.yaml, which the package Makefile reads as TALOS_VERSION. Between v1.5.4 and v1.6.0 it moved from v1.13.0 to v1.13.6 and check B printed nothing for it. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
7ad7d39349 |
test(hack): keep every chart's stored Helm release within budget
A chart can grow past the 1 MiB Helm release Secret without any test noticing: it renders, lints and installs, and only the upgrade whose manifest is large enough fails, on a live cluster, with a release that stays failed. backupstrategy-controller sat under 1 KB from the limit before it crossed it. Estimate the stored size the way Helm encodes it (release JSON with the chart's templates and files, gzip, base64) and fail any chart under packages/ above 90 percent of the limit, plus the largest revision of backupstrategy-controller with every default Strategy rendered. The estimate counts files a .helmignore names, because the production path stores them. A size measured without its manifest is a lower bound that passes for the wrong reason, so every chart must render offline except an explicit list of charts that need cluster state, and the worst-case render must succeed and contain the default strategies before it is measured. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
513b5088a8 |
test(backups): fail on a strategy CRD that inlines a pod schema
The release-size budget only reports that the backupstrategy-controller release grew too large, not which field did it, and three strategies once inlined the upstream pod schema before anyone noticed. Check the generated strategy CRDs for an inlined PodSpec and name the property path, with the markers that fix it. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
2f6c6a27b8 |
fix(backups): store the Altinity, Job and FoundationDB pod templates schemaless
backupstrategy-controller renders its strategy CRDs from templates, so Helm stores each of them twice in every release revision, and a revision is one Secret capped at 1 MiB. The Altinity, Job and FoundationDB CRDs each inline the full upstream PodTemplateSpec schema, about 600 KB apiece; once the default Strategy CRs render as well, the stored release no longer fits and the forced upgrade that creates them fails with "Secret ... is invalid: data: Too long". The Kafka, RabbitMQ and Redis strategies already carry their pod template schemaless for this reason. Give the three older fields the same markers. The Go types stay typed, the drivers decode the template as before, and the apiserver still validates the pod when the driver creates the workload; every description is kept, and the largest revision drops to about a quarter of the limit. The rationale moves to the package doc, so the six field docs only tell the reader what schemaless means for them. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
aed7fa896e |
docs(release): add changelog for v1.6.5 (#4940)
This PR adds the changelog for release `v1.6.5`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.6.5.md`.
|
||
|
Myasnikov Daniil
|
4814ccfed9 |
docs(release): note the PostgreSQL restart on upgrade in the v1.6.5 changelog
Clusters with a memory limit below 512Mi and no explicit shared_buffers get the new default on upgrade, which needs a PostgreSQL restart. The changelog described the fix but not the restart. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-by: LLM |
||
|
cozystack-ci[bot]
|
13f612d55e |
docs: add changelog for v1.6.5
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com> |
||
|
Myasnikov Daniil
|
c2dab12a2e |
fix(cluster-api): resync capi-operator every minute
A restarted capi-operator v0.19.0 drops the create events for the provider Deployments it finds already running (kubernetes-sigs/cluster-api-operator#1031). A provider whose Ready condition went False while the operator was down therefore keeps it until that Deployment changes or the informer resyncs, and the first resync after a restart lands two periods later because the handler registers after the informer has listed. With the 10m default that is about 20 minutes, during which capi-providers-core times out on the CoreProvider and everything depending on it waits; every further operator restart starts the clock again. With a one-minute period the same repro recovered 2m02s after the operator took its lease, against no recovery within 4m26s with the default on the same cluster. A resync replays the cache and, with providers in sync, ends in "No changes detected" and an unchanged condition, so it writes nothing to the API server; the operator logged about 15 lines a minute and used 2m CPU and 26Mi of memory. The upstream chart has no value for the flag, so a package patch that make update reapplies adds one, and a bats test holds the rendered argument. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
512ffa7ccd |
ci(release): reject truncated entries and docs/next links
Two defects in the v1.7.0 draft were mechanical and passed the validator anyway. Three entries end in an ellipsis right before their attribution: the PR text was pasted up to a length limit, and the tail it lost is where PR authors put what the operator has to do, so the entry is wrong rather than short. And Feature Highlights linked cozystack.io/docs/next/, which the live site answers with 404. Neither appears in any shipped changelog, so rejecting them costs no good output. The link check is anchored on the host because v1.3.0 names the website's content/en/docs/next/ tree in prose, and the ellipsis check only fires before an attribution because v1.4.0 has one inside a sentence and every compare link has three dots. The offending line numbers go into the one-line reason, since the workflows quote the reason into an annotation that ends at the first newline. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
e2ca1b7bc5 |
ci(release): overlay the release checks the changelog agent runs
The changelog job checks out the rc tag detached, so every file the agent reads by its working-tree path is the rc tag's copy. Only the agent instructions were refreshed from the dispatch ref. The instructions now tell the agent to run the checks in docs/release.md and hack/validate-changelog.sh on its own output before it exits, and the rc tag's copies of those predate any check added since, or for an old enough tag do not exist at all. Refresh both alongside the instructions. docs/release.md is under neither sparse-checkout path and reaches the tooling checkout only because cone mode also carries the files of every parent directory of docs/agents. The contract test pins both copies and that dependency, so a switch to non-cone patterns fails here rather than as a hard failure of the refresh step at promote time. The AI step keeps its read-only token and its tool flags. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
6395ca1759 |
docs(release): check changelog claims against the code
The generator writes each entry from the opening of the PR text, one PR at a time, and nothing in its instructions asks it to check a statement against the tree it ships. The v1.7.0 draft had every author and PR number right and still carried 22 wrong or missing statements, several in the breaking-changes section, where a wrong instruction breaks an upgrade: a companion-repository fix credited to a tag that does not contain it, worker pools said to survive a migration that never pins their template, migrations said to run on a variant that disables them. The checks that would have caught them live in docs/release.md, which the model is never pointed at, and the detailed rules it does read are about bookkeeping. Add the semantic half: read the whole release note, describe the final state at the upper-bound tag, compare with the previous release, take identifiers, migration behaviour and variant scope from the code, collect required actions beyond `!` commits, pin companion changes to a tag, and run the pre-publish checks and the validator on the written file before exiting. The pre-publish checks name the file after $NEW, which in the promotion configuration is the rc tag, so they are told to read the stable file name; otherwise the reference and link checks read a file that does not exist and report nothing. A minor release's versioned docs are not live until it publishes, so those links are checked against the website tree rather than expected to answer 200. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
8bd6d91dbb |
fix(backups): gate every default Strategy CR on its CRD (#4929)
## What this PR does Upgrade from v1.6.x to v1.7 leaves `backupstrategy-controller` HelmRelease in `UpgradeFailed`: `no matches for kind "Kafka" in version "strategy.backups.cozystack.io/v1alpha1"`, same for Rabbitmq and Redis, and none of the new strategy CRDs get created. CRDs of this chart are ordinary templates, and Helm maps every object of a release through the RESTMapper before applying any of them. Default Strategy CRs for kafka, rabbitmq and redis were gated only on the bucket name, so they render next to their not yet served CRDs, Build() fails, nothing is applied and every retry renders the same manifest. Same happens on a first install with external S3 (`provisionBucket: false`) for all older default strategies, bucket name comes from values there. MongoDB default strategy already has the `crdEstablished` gate, this PR adds it to all other default strategies. On an installed cluster nothing changes. Otherwise first revision creates the CRDs and skips the CRs, then DefaultObjectsGate forces the follow-up upgrade that renders them. CRDs stay templates so later CRD changes still reach the cluster on upgrade. Short explanation is added to `docs/operations/backup-classes.md`. To keep this from coming back there is `hack/backupstrategy-crd-gate.bats`: it renders the chart with the bucket gate open and no CRD served and fails on any Strategy CR from any template, so a new `strategy-*-default.yaml` without the gate goes red without anyone updating a list. It also checks that every CR gates on the CRD this chart ships for its kind. helm-unittest covers both halves of the gate for all 11 templates. Checked on a throwaway kind cluster: - v1.6.4 installed, then upgraded: current main fails with the error above, this branch passes with all 11 CRDs Established and the follow-up upgrade creates the rest - modified CRD is applied on a later upgrade - fresh install converges, and fresh install with external S3 converges here while main fails with "no matches for kind" for eight kinds Fixes #4925. Needs backport to release-1.7, both commits apply there cleanly. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note fix(backups): backupstrategy-controller release no longer gets stuck in UpgradeFailed on upgrade from v1.6 and no longer fails a fresh install with external S3, strategy CRDs are created first and default strategies follow on the next reconcile ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Default backup strategies are now created only after their required resource types are available and the backup bucket is resolved. This helps prevent installation and upgrade issues, including when using externally managed S3 storage. * **Documentation** * Clarified when default backup strategies are created during installation and upgrades. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
f47bab7280 |
test(backups): fail CI on any default Strategy CR rendered without its CRD gate
The CRD gate has now been missed twice: MongoDB shipped without it, and Kafka, RabbitMQ and Redis followed after it existed. helm-unittest only checks the templates a case lists, so a new strategy-*-default.yaml that forgets the gate stays green until an upgrade wedges. Render the chart with the bucket gate open and no CRD served, and fail on any strategy.backups.cozystack.io object from any template. Also check that every rendered Strategy CR gates on the CRD this chart ships for its kind, because a gate naming the wrong CRD passes the first check and keeps the CR out of every live render. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
7204992452 |
fix(backups): gate every default Strategy CR on its CRD
The chart ships its strategy CRDs as ordinary templates, and Helm maps every document of a release through the RESTMapper before it applies any. A default Strategy CR gated only on the bucket name therefore fails Build() with "no matches for kind" whenever the bucket name resolves before the CRD is served, the release applies nothing, the CRDs are never created, and every retry renders the same manifest. That happens on the upgrade that first ships a kind: the Kafka, Rabbitmq and Redis CRs leave a v1.6 cluster's HelmRelease stuck in UpgradeFailed on the way to v1.7. It also happens on the first render of an external-S3 install (provisionBucket=false), where the bucket name comes from values and every older default CR renders next to its absent CRD. MongoDB already gates on crdEstablished for this reason. Give every default Strategy the same gate. On an installed cluster the CRDs are established and the render is unchanged; otherwise the first revision creates the CRDs and skips the CRs, and DefaultObjectsGate, which reads the cozy-default BackupClass routes, forces the follow-up upgrade that renders them once the kinds are served. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
11b19635e0 |
fix(monitoring): send alerts to every Alertmanager replica (#4889)
## What this PR does vmalert sends every alert to every Alertmanager replica instead of to the shared Service. The chart gave vmalert a single notifier, `http://vmalertmanager-alertmanager.<namespace>.svc:9093`. That Service is headless and resolves to the three Alertmanager pods, but each alert reached one replica at a time and moved to another one whenever vmalert reconnected. The Alertmanager README asks for the opposite. It says "Do not load balance traffic between Prometheus and its Alertmanagers" and to point the sender at every instance, because the cluster deduplicates notifications only when each replica receives each alert. What this looked like on three clusters running 1.6.4, read through the Alertmanager API of each pod. - On all three, every active alert sat on one replica and the other two had none. The counts were 0, 0 and 10 on the first, 0, 0 and 23 on the second, and 1, 0 and 9 on the third. - On the third one, which also has an email receiver, the active alert count of each replica changed 163 to 180 times in 24 hours. The three replicas sent 63, 67 and 62 emails in that window. Over the same week only three alert names with a severity routed to email fired there. An alert that leaves a replica expires there and goes out as resolved, and the replica that receives it next sends it as a new firing notification. So a single long-running alert keeps producing firing and resolved notifications, and a receiver that rejects concurrent duplicates sees them as conflicts. The change. - `vm/vmalert.yaml` renders one notifier per Alertmanager pod, `http://vmalertmanager-alertmanager-<i>.vmalertmanager-alertmanager.<namespace>.svc:9093`. That is the address the operator builds for StatefulSet pods (`VMAlertmanager.asPodFQDN`), and the replicas already reach each other through the same per-pod names in their `--cluster.peer` arguments. - The replica count comes from a new `monitoring.alertmanager.replicas` helper, which now also sets `spec.replicaCount` of the VMAlertmanager. The notifier list cannot drift from the number of pods. Why not the operator's notifier selector. `VMAlertNotifierSpec.selector` expands matched VMAlertmanagers into the same per-pod addresses, but only when the VMAlert is reconciled. The VMAlert controller does not watch VMAlertmanager objects, so a VMAlert reconciled before its VMAlertmanager exists, which can happen on a fresh install, would get no notifier until something else triggers a reconcile. A static list renders the same addresses without depending on that order. Upgrade impact. The VMAlertmanager is unchanged, it renders `replicaCount: 3` as before. The VMAlert gets new notifier arguments, so the operator restarts the vmalert pod once. Testing. - `helm unittest` passes for the monitoring chart, 24 suites and 304 tests. The new suite `tests/vmalert_notifiers_test.yaml` checks the three notifier addresses, the replica count of the VMAlertmanager and the release namespace in the addresses. - Each of these breakages turns at least one case of the new suite red. The notifier back on the Service address, the helper set to 2, a hardcoded namespace in the address, and a hardcoded `replicaCount` that no longer follows the helper. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Walked the trigger map in `docs/agents/contributing.md` against the diff. It changes two templates and the helpers of `packages/system/monitoring` and adds a test file. No values, schema, README, package name, make target or `ApplicationDefinition` changes, so no entry applies. ### Release note ```release-note fix(monitoring): vmalert now sends alerts to every Alertmanager replica instead of the shared Service, so the Alertmanager cluster deduplicates them and a long-running alert no longer produces repeated firing and resolved notifications. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * vmalert now configures a notifier URL for each of the three Alertmanager replicas, rather than using a single shared URL. * Alertmanager’s replica count is managed consistently with the notifier configuration. * **Tests** * Added checks for the replica count, all three notifier URLs, and the configured namespace. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
f3486f4e3a |
docs: move the operator guides out of docs/ to where readers find them
Five guides in docs/ were written next to the feature code that needed them and left in the source tree, where no operator looks. They are on the website, so a second copy here only drifts: edits kept landing in docs/ after the website copy was made. docs/vm-external-vlan.md is split rather than moved. Its section on staging the reference CNI plugins is a contract that hack/cni-plugins-staging-contract.bats asserts against the multus Dockerfile, the daemonset manifest and the platform values, so that half is package documentation and goes to packages/system/multus/README.md, which the suite now reads. The operator half goes to the website. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
acf02ef5cf |
docs(agents): define where a new document belongs
The `docs/` tree had accumulated component guides that belong on the website: standalone OIDC, GPU/vGPU and VM-networking documents written as part of a feature PR and never routed anywhere. Nothing said where documentation should go, so `docs/` was the default, and nothing said that the website imports a package README only for the packages on its Makefile lists, so prose written for any other package reached no reader. The destination table turns on one test: whether a non-documentation file has to link to the document by path. A merged document records current behaviour and its limitations rather than the development process around it, because a claim about a phase or an unmerged change turns false once the work ships and nothing re-checks it. backup-classes.md still described the `image.tag` pin as one that an in-flight change removes. The pin has since been removed, so the note now says that. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
e05c56524e |
fix(backups): prepare standalone VMDisk restore targets and add VM backup/restore e2e round-trips (#3427)
## What this PR does Closes the uncovered VM backup path from #3236. `hack/e2e-chainsaw/backup/` held only a disabled Chainsaw port of the old `backup.bats`, written against the retired API (`kind: VirtualMachine`, `BackupJob.storageRef`/`strategyRef`, `velero-strategy-default`), so it guarded nothing. This removes it, fixes the in-place restore of a standalone VMDisk in the backup controller, and adds two real round-trips in the `vminstance` suite that drive the `examples/backups/{vmi,vmdisk}` scripts, so the documented demo and the automated test stay in step. ### Controller: standalone VMDisk in-place restore `collectUnderlyingResources` in `internal/backupcontroller/velerostrategy_controller.go` returned nothing for any kind but VMInstance, so `prepareForRestore` did nothing for a standalone VMDisk: no HelmRelease suspend, no DataVolume delete, no `keepOriginalPVC` rename. An in-place VMDisk restore then raced Flux and CDI over the PVC name the data mover was repopulating. A standalone VMDisk now reports its own disk as its underlying resource, so the restore preparation already used for VMInstance disks suspends the `vm-disk-<name>` HelmRelease, deletes its DataVolume and renames the live PVC to `<name>-orig-<hash>` before Velero recreates it. Unit tests cover both the VMDisk and the VMInstance branch. A VMInstance is halted before its disks are touched, but a VMDisk target is not, and every VMInstance disk is a VMDisk. Renaming the PVC of a disk that a running VM mounts would leave the original `Terminating` behind `pvc-protection`; Velero's PVC restore action skips a PVC that still exists, so the RestoreJob would report `Succeeded` having moved nothing, and the VM would lose its disk on the next restart. An in-place VMDisk restore therefore fails before changing anything when a Pod that has not terminated mounts the disk's PVC. The message names the Pod and tells the user to stop the VM or restore the VMInstance instead. `TestPrepareForRestore_VMDiskInUse` covers a running mounting Pod (refused, PVC not renamed, HelmRelease not suspended), a finished Pod, a Pod on another disk, and a VMInstance target whose own launcher Pod must not refuse it. A VMDisk Backup taken before the disk was recorded as its underlying resource has none, so the disk is derived from the application name (`vm-disk-<name>`): such a restore is prepared and checked for a mounting Pod too. The per-disk `orLabelSelectors` stay a VMInstance concern: a VMDisk strategy already selects its disk, and Velero rejects a Backup or Restore carrying both `labelSelector` and `orLabelSelectors`, so appending them for a VMDisk would break a strategy written with `labelSelector`. When an in-place restore fails after preparation, nothing is rolled back. Resuming the disk's HelmRelease while its PVC is renamed and its DataVolume deleted would have CDI import a fresh disk under the original name, which looks healthy and hides that the data sits in the `-orig-` PVC. The failure message and a Warning event instead name the HelmReleases preparation suspended and the `-orig-` PVCs holding the original data, read back from the cluster so that a failure before a step ran does not name what it would have left, and say that a HelmRelease Velero already restored is resumed, so the disk has to be checked; `examples/backups/vmdisk/README.md` describes the recovery. Tests cover both selector paths, the pre-upgrade Backup, a failed Pod list (the restore requeues and suspends nothing), and all three failures after preparation: a `PartiallyFailed` Velero Restore, several Restores owning the job, and a Restore that cannot be created. Because preparation takes the target apart before Velero validates the Restore, the RestoreJob is refused before preparation, with nothing changed, when the Velero Backup is gone (expired or deleted) or not `Completed`. This is the same single check #4672 added, now run on every pass behind the preparation failure path, so a Backup that expires after preparation started names the leftovers too. A Velero Restore that ends in `FailedValidation` fails the RestoreJob like `Failed`, with Velero's validation errors and the leftovers in the message, instead of polling forever. A HelmRelease suspend or a VM halt that does not happen now stops preparation (requeue) rather than letting the rename and DataVolume delete run under a live release or VM, and a VMInstance's disks are renamed only once no Pod mounts them. A VMDisk held only by a Pod already being deleted is waited for rather than refused, for up to 30 minutes after the RestoreJob started, so a Pod stuck terminating on a lost node fails the job instead of keeping it `Running`. Each of these is pinned by a test that turns red when the guard is removed. Between renaming a kept disk's PVC and deleting its DataVolume, CDI can recreate an empty PVC under the original name for the still-present DataVolume. Velero then skips that PVC as already present ("PVC already exists. Skip restore this PVC."), the RestoreJob reports `Succeeded`, and the VM boots a blank disk. The round-trip's new DataDownload check caught this on a VMInstance restore, where the uid, label and `-orig-` checks had all passed. Once the DataVolume is gone (the wait #4672 added), preparation now deletes a PVC found under the original name (read uncached) before the Velero Restore is created; `TestPrepareForRestore_ClearsPVCRecreatedAfterRename` reproduces the recreation inside the DataVolume delete. Preparation spans several reconciles, so the RestoreJob records a `PreparationStarted` condition before it first changes the target; if recording it fails, preparation retries without changing anything, so a later failure cannot report leftovers that were never made. A refusal on a later pass (the Velero Backup expired, a VM started on the disk) then names what preparation left behind instead of saying nothing was changed, and a preparation still waiting 30 minutes after it started fails the same way instead of polling forever. The vmdisk script starts its consumer Pod only on a PVC that carries the restore label, so it cannot pin a PVC CDI recreated in the rename window. Now that a VMDisk target is prepared, `keepOriginalPVC: false` reaches it as well. Its DataVolume is deleted with orphan propagation, and the bound PVC is deleted and waited for before the Restore (the path #4672 added for VMInstance disks). Left in place, Velero would skip the PVC and the CDI webhook would refuse the DataVolume the resumed HelmRelease creates, leaving the disk's HelmRelease stuck. `TestPrepareForRestore_VMDiskWithoutKeepDeletesOriginalPVC` pins it. ### `vminstance-2-backup-roundtrip` (un-gated in CI) A tiny cirros guest (`u1.nano`, 2Gi, in-place restore only) runs the current API end to end: Velero strategy, BackupClass, a booted VMInstance, an ad-hoc `BackupJob` and an in-place `RestoreJob`, each waited to `Succeeded` and failing fast on `Failed`. The VM must come back `Running`, and its disk PVC must have been created by the restore: a new `metadata.uid`, Velero's `velero.io/restore-name` label, and the `vm-disk-<name>-orig-<hash>` PVC that the rename leaves behind. The label alone does not prove the data moved, because with `existingResourcePolicy: update` Velero also stamps it on a PVC it only patched, and the patch path never reaches the data mover. An in-guest signal is not available on this platform: the KubeVirt CR disables serial-console logging cluster-wide and cirros ships no qemu-guest-agent. ### `vmdisk-2-backup-roundtrip` (un-gated in CI) The disk-only companion: import a 5Gi Ubuntu disk, back it up, restore it in place relying on the controller fix above, and require the same three signals (new uid, restore label, `-orig-` PVC), a `Completed` DataDownload for the restore that wrote bytes, and a Ready HelmRelease. On a `WaitForFirstConsumer` StorageClass, such as the `local` class the CI lane uses, the Velero data mover waits until a scheduled Pod has selected a node for the restored PVC (`WaitPVCConsumed` in Velero's generic restore exposer). A VMInstance gets that from its virt-launcher Pod; a detached disk has none, so its `RestoreJob` stays `Running` until the 30m data-mover prepare timeout fails it. The script therefore starts a restricted-profile `pause` Pod on the new PVC once Velero has created it, standing in for the VM that would attach the disk, and removes it after the restore. `examples/backups/vmdisk/README.md` documents this for operators restoring a detached disk. ### Harness details - The disk `storageClass` is an override (default `replicated`). The Chainsaw steps pass the lane's `COZY_E2E_STORAGE_CLASS` (`local` on the DRBD-off PR gate). - Each harness stands up a throwaway in-cluster BSL (`cozy-e2e-vmi`/`cozy-e2e-vmdisk`) at `seaweedfs-s3.tenant-root:8333` with `insecureSkipTLSVerify` and `checksumAlgorithm: ""` (the #3236 fixes), reusing the platform bucket and `cozy-backups-creds` under a distinct prefix, because the platform `cozy-default` BSL derives the external S3 ingress endpoint, unroutable in the CI cluster. - The script waits are bounded so that their worst case stays under the 40m op timeout (38m for the VMDisk step), and a hung wait trips the script's own named timeout rather than a bare `signal: killed`. - On failure, `catch` also describes the Velero `Restore`, `DataDownload` and `DataUpload` objects in `cozy-velero`. - `skipDelete` plus a `finally` that resumes the suspended HelmReleases, reclaims the `-orig-` PVC and its LINSTOR volume (the PV is flipped back to `Delete` first), and drains the Velero backups on the throwaway BSL, branching on kubectl's exit status, before deleting it. When the drain is not confirmed, the BSL is kept and the step fails, so the Velero Backups are not orphaned behind a green step. - HelmRelease waits use the shared `wait_hr_ready`, which fails fast on `Stalled`. After the VMDisk restore the script first waits for `observedGeneration` to reach the restored spec's `generation`, because the `Ready=True` from before the suspend would otherwise satisfy the wait at once. The `vmi` example is CI-drivable: `run-all.sh`/`cleanup.sh` are non-interactive, the VM is waited to boot before the backup, `volumeSnapshotLocations` is dropped from the demo strategies (`snapshotMoveData` routes data through the BSL), and `BACKUP_STORAGE_LOCATION` defaults to `cozy-default`. `select-e2e.sh` maps `examples/backups/{vmi,vmdisk}` edits to the `vminstance` suite, except `07-restore-to-copy.sh`, which the suite skips (`SKIP_RESTORE_TO_COPY=1`) and so stays docs-only. A `packages/system/velero` change selects the `vminstance` suite as well, since these are the only Tests that run a Velero backup and restore. ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff (backup controller restore preparation for VMDisk, shell scripts under `examples/backups/`, Chainsaw tests, `hack/select-e2e.sh` routing, a deleted stale suite, one doc line): no `packages/apps|extra` add/rename, no `values.schema.json`/enum/default changes, no CRD/ApplicationDefinition/label/annotation/namespace/metric changes. The controller change alters what a VMDisk `RestoreJob` does in the cluster, not any API surface a downstream repository consumes. `cozystack/examples` holds Terraform for bare nodes, unrelated to the in-repo `examples/` tree. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(backups): an in-place RestoreJob for a standalone VMDisk now suspends the disk's HelmRelease, deletes its DataVolume and (with keepOriginalPVC) renames the original PVC before Velero restores it, instead of racing Flux and CDI over the same PVC; without keepOriginalPVC the original PVC is deleted so Velero recreates it from the backup. Such a restore fails before changing anything when a running Pod, such as the launcher of a VM that uses the disk, mounts the disk's PVC: stop the VM or restore the VMInstance instead. If such a restore fails after the disk was prepared, its message and a Warning event name the HelmRelease preparation suspended and the PVC holding the original data. On a WaitForFirstConsumer StorageClass the data mover still starts only once a Pod consumes the restored PVC, for example when the disk is attached to a VMInstance. An in-place restore with keepOriginalPVC no longer reports Succeeded over an empty PVC that CDI recreated under the disk's name while the original was being kept aside. Adds Chainsaw backup/restore round-trips for VMInstance and standalone VMDisk that run a real Velero backup and restore in CI. ``` |
||
|
Aleksei Sviridkin
|
588d8292e7 |
fix(seaweedfs): build SeaweedFS in-tree with a bounded S3 read-ahead (#4819)
## What this PR does Large objects served through a Bucket's S3 endpoint stop tens of megabytes into the download (cozystack/cozystack#3782). The cause is not TLS: the SeaweedFS S3 gateway is OOM-killed mid-response. Its read-ahead for a sequential GET is meant to stay four chunks in front of the reader, but SeaweedFS counts only newly started chunk downloads toward that window and re-runs the prefetch after every 256 KiB read, so the gateway pulls up to about a gigabyte of the object into memory and exceeds the 512Mi limit the S3 deployment gets from its resources preset. The client sees `unexpected eof` over HTTP/1.1 and `INTERNAL_ERROR` over HTTP/2, which is what the CDI importer reported. Upgrading does not help yet: 4.31, 4.45 and the latest release 4.48 all carry the defect. The defect is reported upstream in seaweedfs/seaweedfs#11672, and the fix with its regression test is merged there in seaweedfs/seaweedfs#11674, after 4.48. Until a SeaweedFS release carrying it is vendored, this PR builds SeaweedFS in-tree the way #3343 builds cilium-operator: `images/seaweedfs` downloads the 4.45 source archive and checks it against a pinned checksum, applies `patches/reader-cache-prefetch-window.diff`, runs the tests of the packages the patch touches (including the regression test the patch adds) once, compiles `weed` with the Go toolchain upstream's own builder uses for the tag, and lays it over the upstream `chrislusf/seaweedfs:4.45` image pinned by digest, so everything else in the image stays the upstream build. `weed version` reports `cozystack-patched` in the commit field. All SeaweedFS components run the in-tree image. `make update` re-pins the Dockerfile from the vendored chart: the version, the source checksum, the upstream image digest and the builder toolchain. It fails when the patch no longer applies, which is how the first release carrying the upstream fix will show that the patch is ready to drop. Renovate is disabled for that Dockerfile, since it would move those pins one at a time, and whitespace checks are off for the patch. Tests: - `tests/image_test.yaml` pins every component (master, volume, filer, S3) to the in-tree image with a digest. - `make test-image-version` fails when the Dockerfile's version drifts from the vendored chart's appVersion. - `make update-image`, which `make update` runs, was checked to be idempotent on the current chart, to restore deliberately corrupted pins, and to fail with a clear message when the patch context no longer matches. - `hack/build-matrix_test.bats` now expects seaweedfs to be a build unit next to objectstorage-controller. - The Go regression test inside the patch fails on unpatched 4.45 (read-ahead reaches 60 chunks past the reader) and passes with the patch; the image build runs it. Verified on a test cluster against an 840 MiB object, with every SeaweedFS component running the image this package builds and the S3 gateway at the production limits (512Mi, 1 CPU). Unpatched 4.31, 4.45 and 4.48 gateways were OOM-killed on every full GET. The patched gateway served nine full GETs (six sequential over HTTP/1.1 and HTTP/2, three in parallel) with matching checksums and no restarts; it read from the volume servers about as many bytes as it sent, its live heap after a GET was about 40 MiB, and three parallel GETs peaked at about 140 MiB. The image this PR pins reports `4.45 cozystack-patched` and was rerun through the same checks. Drop the patch, and the in-tree build with it, once a SeaweedFS release carrying the upstream fix is vendored. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(seaweedfs): stop the S3 gateway from being OOM-killed while serving large objects, which cut downloads (including CDI imports from a Bucket URL) off tens of megabytes in. SeaweedFS is now built in-tree with a patch that bounds the gateway's read-ahead. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * SeaweedFS now runs a Cozystack-built image based on version 4.45, pinned to a specific image digest. * **Bug Fixes** * Sequential small reads now keep read-ahead within the configured prefetch window, preventing it from extending unexpectedly. * **Build & Maintenance** * SeaweedFS image updates are aligned with the chart version and verify the source archive, builder image, and patches before building. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
58b4343e5a |
ci(hack): route examples/backups/{vmi,vmdisk} edits to the vminstance suite
The two backup round-trips run the examples/backups/{vmi,vmdisk} scripts as their
harness, but select-e2e.sh mapped an examples/backups/<dir> edit to a suite only
when <dir> was itself a suite name. The dirs are vmi/vmdisk while the suite is
vminstance, so a harness-only edit selected no suite and the round-trip that runs
it never ran. Translate both dirs to vminstance, matching the vm-disk-application
entry in src_to_suites, and pin it with a bats case. The suite runs the vmi demo
with SKIP_RESTORE_TO_COPY=1, so 07-restore-to-copy.sh stays docs-only: selecting
the suite for it would pass without executing it. Both round-trips write to
the in-cluster seaweedfs-s3 endpoint, so a seaweedfs change selects the
vminstance suite as well. Also correct the controller-
owned-teardown note in docs/agents/e2e-testing.md (only Backup/RestoreJob
finalizers reach Velero; a BackupJob has none).
The two round-trips are also the only Tests that run a Velero backup and
restore, so the velero PackageSource maps to the vminstance suite; it
otherwise maps to a velero suite that does not exist, and a Velero change
ran neither round-trip.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrey Kolkov
|
9cc3a9e459 |
test(vmi): add VMInstance and standalone VMDisk backup/restore e2e round-trips
The disabled hack/e2e-chainsaw/backup/ suite was a Chainsaw port of the retired
backup.bats, written against the old backup API, so the VMInstance and standalone
VMDisk Velero paths had no e2e coverage. Replace it with two round-trips in the
vminstance suite that drive the validated examples/backups/{vmi,vmdisk} scripts,
so the documented demo and the automated test stay in step.
Both run un-gated in CI:
- vminstance-2-backup-roundtrip boots a tiny cirros VM (u1.nano, 2Gi), backs it
up via BackupClass + BackupJob, restores in place, and asserts the VM returns
Running off a disk PVC the restore created: a new uid, Velero's restore-name
label and the -orig- PVC keepOriginalPVC leaves behind. The label alone is not
enough, because existingResourcePolicy: update also stamps it on a PVC Velero
only patched, which moves no data. A guest-side boot proof is unavailable: the
KubeVirt CR disables serial-console logging and cirros ships no guest-agent.
- vmdisk-2-backup-roundtrip imports a 5Gi disk, backs it up, and restores it in
place with the same three assertions. The controller prepares the target
(suspend HelmRelease, delete DataVolume, rename PVC). On a WaitForFirstConsumer
class the Velero data mover waits for a Pod to select a node for the restored
PVC, which a detached disk never gets, so the script starts a pause Pod on the
new PVC, standing in for the VM that would attach it.
The disk storageClass is an override (default replicated; the CI steps pass the
lane's COZY_E2E_STORAGE_CLASS, 'local' on the DRBD-off gate) and the script waits
are bounded so their worst case stays under the 40m op timeout, sized to keep the
whole parallel:1 suite inside the 215m job cap. On failure the catch also dumps
the Velero Restore, DataDownload and DataUpload objects. Each harness stands up a
throwaway in-cluster BSL (the platform cozy-default BSL derives the external S3
endpoint, unroutable in kind); its finally resumes the suspended HelmReleases,
reclaims the keepOriginalPVC orig PVC and its LINSTOR volume, and drains the
Velero backups before deleting the BSL. The example scripts are non-interactive
so the harness can drive them.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrey Kolkov
|
9af1c0f902 |
fix(backups): prepare standalone VMDisk restore targets before the data mover runs
collectUnderlyingResources returned nothing for a non-VMInstance kind, so prepareForRestore did nothing for a standalone VMDisk: no HelmRelease suspend and no DataVolume delete. An in-place VMDisk restore then raced Flux/CDI — the restored HelmRelease reinstalled and CDI recreated the DataVolume over the same PVC name the data mover was repopulating, wedging the RestoreJob. Report a standalone VMDisk's own disk as its underlying resource, so the existing (VMInstance-proven) restore preparation suspends the vm-disk-<name> HelmRelease, deletes its DataVolume and — with keepOriginalPVC — renames the live PVC before the data mover recreates and repopulates it. No VM, so no IP/MAC is collected. A VMInstance is halted before its disks are touched; a VMDisk target is not, and every VMInstance disk is a VMDisk. Renaming the PVC of a disk a running VM mounts leaves the original Terminating behind pvc-protection, Velero skips a PVC that still exists, and the RestoreJob would report Succeeded having moved nothing while the VM loses its disk on the next restart. An in-place VMDisk restore now fails before changing anything when a Pod that has not terminated mounts the disk's PVC, naming the Pod and telling the user to stop the VM or restore the VMInstance instead. A VMDisk Backup taken before the disk was recorded as its underlying resource has none, so its disk is now derived from the application name and such a restore is prepared and checked for a mounting Pod as well. The per-disk orLabelSelectors stay a VMInstance concern. A VMDisk strategy already selects its disk, and Velero rejects a Backup or Restore that carries both labelSelector and orLabelSelectors, so appending them for a VMDisk would break a strategy written with labelSelector. When an in-place restore fails after preparation, nothing is rolled back: resuming the disk's HelmRelease while its PVC is renamed and its DataVolume deleted would have CDI import a fresh disk under the original name, which looks healthy and hides that the data sits in the -orig- PVC. The failure message and a Warning event now name the suspended HelmReleases and the -orig- PVCs so the user can recover them, read back from the cluster, so a failure before a step ran does not name what it would have left. Preparation takes the target apart before Velero validates the Restore, so it is refused up front when the Velero Backup is gone or not Completed. That check now runs on every pass behind the same failure path as the rest of preparation, and a FailedValidation Restore fails the RestoreJob like a Failed one instead of leaving it Running. A HelmRelease suspend or a VM halt that did not happen now stops preparation instead of renaming and deleting under a running release or VM, and a VMInstance's disks are renamed only once its launcher Pod has let go of them. A VMDisk held only by a Pod already being deleted is waited for rather than refused, for up to 30 minutes after the RestoreJob started. Between renaming a kept disk's PVC and deleting its DataVolume, CDI can recreate an empty PVC under the original name. Velero then skips the PVC as already present, the RestoreJob reports Succeeded and the VM boots a blank disk. Once the DataVolume is gone, preparation now deletes a PVC found under the original name, read uncached, before the Velero Restore is created. Preparation spans several reconciles, so the RestoreJob records a PreparationStarted condition before it first changes the target, and retries without changing anything when recording it fails. A refusal on a later pass (the Velero Backup expired, a VM started on the disk) then names what preparation left behind instead of claiming that nothing changed, and a preparation still waiting 30 minutes after it started fails the same way instead of polling forever. Now that a VMDisk target is prepared, keepOriginalPVC=false reaches it too: its DataVolume is deleted with orphan propagation and the bound PVC is deleted before the Restore, as for a VMInstance disk. Left in place, Velero would skip it and CDI would refuse the DataVolume the resumed HelmRelease creates. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Alexey Artamonov
|
e8635f6e4b |
build(seaweedfs): stamp the re-pinned in-tree image
The builder, the compile flags and the version stamp changed, so the image built from them has a new digest, and values.yaml has to name it for the components to run that build. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
ecdc043e63 |
build(seaweedfs): keep Renovate and git whitespace off the in-tree build
Renovate would bump the builder, the base image or their digests one at a time into a combination upstream never shipped, and the next `make update` would silently put them back, so it leaves that Dockerfile to the update recipe. The patches are unified diffs of Go sources, whose context lines start with a marker before tabs and blank lines, which git's whitespace checks would otherwise flag or rewrite. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
5592a3f65d |
build(seaweedfs): re-pin the in-tree build from make update
The in-tree build pinned its source, base image and toolchain by hand, and nothing moved them when `make update` vendored a new chart, so a bump would build one release's weed into another release's image, or keep patching a release that already carries the fix. Pin the source archive by checksum and the upstream image by digest as build arguments, compile with the toolchain upstream's own builder uses for the tag, and let `make update` re-pin all of them from the vendored chart and fail when the patches no longer apply. The tests of the patched packages now run once rather than once per target platform, and `weed version` names the build. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
d2c6b85cac |
fix(seaweedfs): run the in-tree SeaweedFS image in every component
With the image keys unset the vendored chart renders chrislusf/seaweedfs at its appVersion, so the patched build from images/seaweedfs would be built and never run, and the S3 gateway would keep being OOM-killed on large GETs. Point every component at the in-tree image, pinned by digest like every first-party reference, and hold that in a unit test so a values change cannot quietly fall back to the upstream binary. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
1913104bd3 |
build(seaweedfs): build SeaweedFS in-tree with a bounded read-ahead
The S3 gateway is OOM-killed while serving a large object. SeaweedFS starts read-ahead downloads after every read of a sequential stream and counts only the downloads it newly starts toward its prefetch window, so a GET reading 256 KiB at a time out of 4 MiB chunks pulls up to about a gigabyte of the object into memory, past the 512Mi limit of the S3 deployment. No SeaweedFS release carries a fix yet, so there is no release to move to instead. Build weed from the 4.45 sources with a patch that keeps the window at the next prefetchCount chunks, and lay it over the upstream image of the same tag so the rest of the image stays upstream's own. The image build runs the tests of the packages the patch touches, including the regression test the patch adds, because go build never compiles them. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Aleksei Sviridkin
|
c0449eeabb |
feat(backups): back up Valkey with the Redis strategy (#4898)
## What this PR does
Adds backup and restore for the managed Valkey app.
Valkey runs on the same spotahome RedisFailover operator as Redis
(`spec.engine: Valkey`), with the same sentinel topology, replication
protocol and RDB format. So instead of adding a new strategy kind, the
existing Redis strategy now serves both apps:
- **Driver.** `applicationRef.kind` may be `Redis` or `Valkey`. The
template context gets `.Release.Prefix`, which the driver derives from
the app's kind (`redis-` or `valkey-`). The chart uses that prefix for
every name it builds: the sentinel Service `rfs-<prefix><app>`, the auth
Secret `<prefix><app>-auth` and the CA Secret. In cleanup mode the
prefix comes from the Backup's `applicationRef`, so it works after the
app is gone. A kind with no known prefix fails the render instead of
silently producing an empty prefix. Error messages name the real kind.
- **Upgrade window.** The shipped template reads `{{ .Release.Prefix |
default "redis-" }}`. The strategy CR and the controller ship in one
release, and until the new controller takes the lease, the previous one
(which has no `Prefix` and serves only Redis) renders the new CR; a bare
reference would print `<no value>` into the Secret and sentinel names.
- **Same-kind restore only.** A restore whose target kind differs from
the backup's source kind fails before any Job is created. Replaying a
Redis dump into Valkey, or the reverse, is not something the strategy
can promise across engine releases.
- **Chart.** `cozy-default-redis` builds names from `.Release.Prefix`.
`cozy-default` routes `apps.cozystack.io/Valkey` to
`cozy-default-redis`; the DefaultObjectsGate already dedupes repeated
strategy refs. No new image: the `redis:8` base loads the RDB v11 that
both shipped Valkey versions write. The Valkey chart has no TLS option,
so the optional CA mount stays empty and the Job connects in plaintext.
- **CRD.** Only the `.Release.Prefix` description in the Redis strategy
CRD changes (regenerated with controller-gen v0.16.4; every other
strategy CRD regenerates byte-identical).
- **Docs.** `docs/operations/backup-classes.md` has the Valkey route and
a paragraph on how the shared strategy behaves.
- **E2E.** The Redis backup demo takes `APP_KIND=Valkey`, which switches
the app kind and the name prefix. Its BackupClass routes Valkey too. The
valkey suite gains `valkey-backup-roundtrip`, which seeds a marker,
backs it up, restores it in place and into a copy. In
`hack/select-e2e.sh`, a seaweedfs change now selects the valkey suite
(enforced by `select-e2e_test.bats`), an edit to the shared
`examples/backups/redis/` harness selects both `redis` and `valkey`
(pinned by a new test), and `hack/select-install.sh` keeps the backup
stack in the reduced install for the valkey suite (pinned in
`select-install_test.bats`).
### Validation
- `go test ./internal/backupcontroller/...` passes. New tests: a Valkey
BackupJob renders `--prefix=valkey-`; a Redis backup restored into a
Valkey target fails without creating a Job; the ref validator accepts
Valkey; a cleanup for a kind with no prefix is released without a Job.
The existing Redis render test now also asserts `redis-`.
- The `default "redis-"` fallback, run through the controller's own
template engine with no `Prefix` in the context, renders
`rfs-redis-cache`; the bare reference renders `rfs-<no value>cache`.
- `helm unittest` on backupstrategy-controller: 26/26, with
`cozy-default` at eleven routes. `helm lint` is clean.
- `hack/redis-backup-strategy.bats` executes the shipped script and adds
a Valkey case: it must query `rfs-valkey-<app>` and never `rfs-redis-`.
The fixture now also fails if any `{{ }}` placeholder survives
substitution. All 62 tests in `hack/select-e2e_test.bats` pass.
- Engine compatibility, checked locally with the strategy image
`redis:8-alpine@sha256:ba6e…` against `valkey/valkey` 8.1.8 and 7.2.13
using the shipped script's exact steps: `redis-cli --rdb` from a
password-protected Valkey master, load into the throwaway
`redis-server`, emit the RESP base AOF, `FLUSHALL`, then `--pipe` into a
fresh Valkey. On both versions every check held: the db0/db1/db3
boundaries, a stream with its consumer group, a key's TTL and the marker
all survived, and FLUSHALL wiped the stale target key.
- I have not run this on a live cluster. The new chainsaw round-trip is
what exercises it end to end.
Out of scope, and already true before this PR: the website's backup
pages in `docs/next` don't list Redis, RabbitMQ or Kafka yet, so none of
the triggers in the downstream map fire here.
### Screenshots
N/A — no UI changes.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
feat(backups): Valkey apps can now be backed up and restored. The platform `cozy-default` BackupClass routes `apps.cozystack.io/Valkey` to the existing `cozy-default-redis` strategy, which dumps an RDB snapshot to the system bucket and restores it in place or into another Valkey app. A Valkey backup restores only into a Valkey app.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for backing up and restoring Valkey applications using
the existing Redis backup strategy.
* Updated the backup walkthrough to support both Redis and Valkey, with
resource names selected for the chosen application type.
* **Bug Fixes**
* Prevented restores when the backup and target application types
differ.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Alexey Artamonov
|
7011510456 |
fix(fluxcd): keep the hook waiting on a release it cannot read
The pre-delete hook read a release's state once, so a single failed read was taken as the release being handled: its fence came off before helm-controller's uninstall, and the report counted it as released cleanly. A state read is now retried within the release's share of the window, a release whose state still cannot be read keeps its fence for the wait like one helm-controller has not handled, and the end of the wait reads each again before naming what the release was left to. A release still suspended after the sweep is named as suspended but not by flux-plunger, which also holds when no manager is recorded for the field at all. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
568e6f5e18 |
docs(fluxcd): correct flux-plunger's recovery order and name its limits
The README gave the recovery steps in the order flux-plunger used before the processed-version annotation moved ahead of the Secret delete. An operator who finds the annotation on a release whose revision Secret is still there would read that state as impossible, when it is what a failed delete leaves and the next pass retries. Two boundaries were left unsaid. The pre-delete hook stops, waits for and lists again only for the flux-plunger of its own release; another one, under a second release name or with changed labels or out of cluster, keeps the fences it sets and gives them up itself. And a backup restore that starts while flux-plunger holds its own suspension does not take the field over, since the restore steps write it only when it differs, so the end of the recovery unsuspends the release under the restore. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
699555c7cb |
fix(fluxcd): give each release its own share of the hook's retries
The pre-delete hook's retry budget was one window spent in listing order, so a release that could never be read or unfenced, listed first, took all of it and every release after it got a single attempt. The same transient failure then released a fence or not depending on where its release sorted. Each release now gets its own share of the window, capped at the window's end, and the steps that cover every release keep the whole of it. Two comments said more than holds. The retry window is measured from the script's start, not the Job's, so scheduling and the image pull come out of its one-minute reserve rather than being covered by it. And the five-minute conflict retry inside kubectl scale never runs here, since without a precondition it sends a merge patch with no resourceVersion to conflict on. helm-controller's finalizer, which the hook reads to tell whether a release being deleted is still held, is now tied to the constant the controller uses, as the fence already was. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
af50c4a6b4 |
fix(fluxcd): let helm-controller uninstall before the hook unfences
The pre-delete hook lifted flux-plunger's suspension of a release being deleted and removed its fence in the same step. helm-controller runs the uninstall of a release being deleted once it is no longer suspended, but when its own finalizer is already gone the fence is all that keeps the object, so removing the fence at once deleted the release before helm-controller could reconcile it, without an uninstall. The hook now keeps such a fence while helm-controller handles the generation the lifted suspension wrote, until shortly before its retry window closes, and names a release it still had to let go of as before. A release being deleted under a suspension another manager also holds was reported as cleanly released, although helm-controller skips the uninstall of a suspended release; it is now named as deleted without one and counted with the others. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
7b84919db3 |
fix(fluxcd): warn about a foreign suspension only on a stuck release
flux-plunger raised its SuspensionNotOwned and TerminatingWhileSuspended Warnings on a suspension it does not own when the release carried its processed-version annotation. That annotation is never removed, so every later suspension of such a release, by a migration, a backup restore or an operator, drew a Warning: one telling the reader to resume it, which mid-restore is the one thing nobody wants, and one calling a migration's deliberate suspend-then-delete an accident. Both also missed the leftover they are for when the first recovery stopped before writing the annotation. A suspension an earlier flux-plunger left mid-recovery freezes the release in the failure recovery started from, since helm-controller rewrites no condition of a suspended release. Both Warnings now key on that state instead, and the README's candidate query with them. Two comments are qualified where they claimed more than holds: a stop that overran the manager's grace period can still let a reconcile set a fence after the drain's last listing, and a process stopped before controller-runtime closes Elected holds the Lease yet does not drain. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
caeaa6b5e5 |
fix(fluxcd): make flux-plunger's shutdown drain report what it left
The drain's only report was an Info line, so an alert on errors never saw fences left behind; it named no release, and printed held: -1 when no listing had answered. A flux-plunger that never led logged that it had released every fence next to the line saying it drained nothing. Fences left behind are now an error naming each release, an unanswered listing says so, and a process that never led says only that. The drain writes its Events itself, since the manager's recorder stops with the manager, but it created one Event object per pass for a release it revisits every second, and each write had a deadline of its own outside the drain's. Repeats now fold into one counted Event, as the manager's recorder folds them, and every write is bounded by the drain. Tests now hold the outcome line, the retry of a release whose fence cannot be given up yet, and the pause between passes, none of which any suite noticed when removed. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
8bcb709ce8 |
fix(fluxcd): refuse a flux-plunger drain deadline that leaves no time
A drain deadline of zero or less is already spent when the drain starts, so flux-plunger would exit holding every uninstall fence without listing one. The chart could produce one: its 40-second floor was a literal, so raising the manager's stop bound or the margin before SIGKILL, which the drain subtraction is already tied to, would leave the floor rendering a non-positive deadline. The binary now refuses such a flag at startup, before it reaches for a cluster, and the chart's floor and the timings the README and the values comment state are held to the same constants. A whole number of a million seconds or more printed in exponent form and was refused as a fraction; the range is now checked first, so it gets the range message. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Aleksei Sviridkin
|
7f454d810c |
chore: link the FOSSA license scan for CLOMonitor (#4899)
## What this PR does Adds a root `.clomonitor.yml` that points CLOMonitor's `license_scanning` check at the FOSSA project for this repository. This is step 2 of #4758. ```yaml licenseScanning: url: https://app.fossa.com/projects/git%2Bgithub.com%2Fcozystack%2Fcozystack ``` - **Why a metadata file rather than a badge.** The check passes on a FOSSA or Snyk link in the README or on a URL in `.clomonitor.yml` ([check docs](https://github.com/cncf/clomonitor/blob/main/docs/checks.md#license-scanning)). #4222 trimmed the README badge row, so this uses the file. - **A correction to #4758.** The key is `licenseScanning`, in camelCase. CLOMonitor deserialises the file with serde `rename_all = "camelCase"` ([`metadata.rs`](https://github.com/cncf/clomonitor/blob/main/clomonitor-core/src/linter/metadata.rs), [template](https://github.com/cncf/clomonitor/blob/main/docs/metadata/.clomonitor.yml)). The snake_case `license_scanning` I proposed in the issue would have been ignored silently. - **Precondition from #4758 is met.** The FOSSA project is now analysed at a current `main` revision (`last_analyzed_revision` = `3a463609e`, 2026-10-12, moved past `bdc3525c9`). It has 0 unresolved licensing issues, and its update hook is the GitHub integration. Checked: the file parses to exactly that one key, the URL matches CLOMonitor's FOSSA regexp, and the project page is public (HTTP 200). Expected effect: `license_scanning` passes on the next CLOMonitor scan, worth about +4 to the project score. This does not close #4758. Per @tym83, licensing findings from the fresh scans stay to be triaged there. ### Downstream repositories <!-- Walked the trigger map in docs/agents/contributing.md against the diff: one new root metadata file read only by CLOMonitor; it touches no package, values file, CRD, asset name, hack/ script or namespace. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added repository metadata describing license scanning and linking to its results. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
43b02bda30 |
test(e2e): run the Redis backup round-trip against Valkey
The Redis backup demo now takes APP_KIND=Valkey, which switches the app kind and the chart prefix the helpers derive sentinel and Secret names from. The demo BackupClass routes Valkey to the same demo strategy. The valkey suite gains valkey-backup-roundtrip, which drives that demo end to end: seed a marker, back it up, restore it in place and into a copy. It writes to tenant-root's seaweedfs through seaweedfs-s3, so a seaweedfs change now selects the valkey suite too, and since the round-trip runs examples/backups/redis as its harness, an edit there selects both the redis and the valkey suites. The reduced E2E install keeps the backup stack for the valkey suite, as it does for the other backup round-trip suites. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
Andrey Kolkov
|
0c2f131192 |
feat(backups): back up Valkey with the Redis strategy
Valkey runs on the same spotahome RedisFailover operator as Redis and speaks the same replication protocol and RDB format, so it needs no strategy kind of its own. The only difference the backup Job sees is the chart's resource-name prefix: valkey-<app> instead of redis-<app> for the sentinel Service and the auth Secret. The Redis driver now accepts applicationRef kind Valkey as well as Redis and hands the template .Release.Prefix, derived from the app's kind, which cozy-default-redis uses for every name it builds. A kind with no known prefix fails the render instead of producing an empty one. cozy-default routes apps.cozystack.io/Valkey to cozy-default-redis. A restore must target an app of the same kind as the backup's source: replaying a Redis dump into Valkey, or the reverse, is not something the strategy can promise across engine releases. The shipped template reads the prefix as `.Release.Prefix | default "redis-"`: during an upgrade the previous controller, which has no Prefix in its render context and serves only Redis, renders the new CR, and a bare reference would print "<no value>" into the names. The redis:8 image already loads RDB v11, which both shipped Valkey versions (7.2, 8.1) write. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |