mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 12:13:55 +09:00
50959ccca20fa212c836cc2ccccf7583f1bc279a
8066
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
50959ccca2 |
chore(backport): discard temp-file cleanup errors explicitly
A temporary file or directory that cannot be removed, or closed after its write already failed, leaves nothing to act on: the run's result is the same, and the primary error is the one returned. The test that unsets an environment variable now fails if it cannot. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
7a481c4765 |
chore(workloadmonitor): discard best-effort close errors explicitly
The Prometheus response body is closed after it has been read, and the test stubs' writes fail on the client side of the test, so neither error changes a result. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
87b794a992 |
chore: handle errors dropped in cleanup paths and tests
The drain tests now assert that RunThenDrain returns the run's nil error, the idle-tail test fails on any error other than the stream ending or being cut, and a test fixture file fails the test when it cannot be closed. Removing a temporary directory and closing a response body that has been read are best-effort: a failure leaves nothing for the caller to act on and does not change any result, so those errors are discarded explicitly. The same holds for the stub servers' response writes, whose failure surfaces on the client side of the test. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
bd998d9653 |
fix(kube-ovn): fail the plunger when its status server cannot start
The status stream was started in a bare goroutine whose error was dropped, and the runnable's registration error was discarded too, so a port that could not be bound left the plunger running with no status endpoint and nothing in its logs. The server now runs inside its runnable: a listen error fails the manager, as a metrics server bind failure does, and stopping the manager is not reported as an error. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
41ad82842a |
fix(sse): drop a client as soon as a write to it fails
Write errors on the event stream were ignored, so a client whose connection broke kept its handler, pump and registration until the request context ended, and every event was written into the dead connection meanwhile. A failed write now ends the stream and unregisters the client, and the handler also returns once its pump ends, so Shutdown no longer leaves streams open. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
20ce6c691e |
fix(sse): close a client's channel once when the server shuts down
Shutdown closed every open client's channel, and the client's handler closed the same channel again when its request ended, so any stream still open at shutdown panicked with a close of a closed channel. Stopping a client is now idempotent. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
f99e511eea |
fix(backport): fail a run whose output could not be written
Every report, check and verify result goes to stdout, and every write error there was dropped, so a run whose output never reached its reader still exited 0. The terminal writer now keeps the first stdout error, and the run fails with it unless it already failed. Only stdout is tracked; a write error on stderr is still discarded. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
05818aa3b2 |
fix(cli): fail cozypkg and check-readiness on unwritable output
For the commands this change touches (cozypkg list, the validation report of validate, push and tap, tap's summary, and check-readiness), a write error the process observes is a failure: the command exits non-zero and says whether the action it reports on already happened. Progress lines printed before the decisive step stay best-effort. A closed pipe ends the process with SIGPIPE, as for other Unix tools. `cozypkg list` and the validation report dropped write errors, and the list deferred its tabwriter flush, so a closed or full stdout still reported success with nothing printed. The list now writes to the command's writer and returns the flush error, and the report returns its write error. When validation refuses a push or a tap, or fails a validate run, the refusal stays the reported reason and the report's write error is joined to it. A push whose report cannot be written stops before flux runs and says "nothing was pushed"; a tap whose summary cannot be written fails with "tap applied, but writing its summary failed". check-readiness fails --help, --version and its readiness result the same way, in one-shot, --wait and --watch mode, with exit 1 as for a result that is not ready, which its help now lists among the exit statuses along with SIGPIPE for a closed pipe; the watch loop stops at the first frame it cannot write, and a timed-out wait still ends with its verdict. push writes its "Pushing" line before it runs flux and stops with "nothing was pushed" when that line cannot be written, with validation skipped or after the report. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
2cda7d692e |
test(backport): drop an unused fixture helper
Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
04fa87ccd1 |
style(backups): clear staticcheck findings in overlapping controllers
Drop a field-manager constant nothing uses in the MariaDB driver, switch on the MongoDB operator's state as a tagged switch, and reword the error strings of the MongoDB driver and the workload monitor so they do not start with a capital. The errors keep their meaning and nothing matches their text. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
238e85577e |
refactor(lineage): resolve a HelmRelease's application in a helper
The walk used a loop that always ended in its first pass to get "break to the fallback" control flow. A function returning whether it resolved the application says the same without the loop. A test pins that each failure to resolve still falls back to the HelmRelease's own label and that a resolved one ends the walk. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
4c3a4b42cc |
style: clear unused code and style findings reported by staticcheck
Remove helpers and constants nothing references, escape format characters in test literals, and take the simplifications staticcheck suggests where they read the same. Error strings no longer start with a capital: they keep their meaning, with the kind or component name moved out of first position. Callers match errors by type, not text; the one test that matches text, for Kafka, keeps the phrase it looks for. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
0f169540a1 |
feat(release): withhold partial backports and park those that build on code their line lacks (#5269)
## What this PR does Stacked on #5264, that one merges first. Clean cherry-pick doesn't mean the original landed. On release-1.6 the batch carried a fix whose two deletions hit files the line never had, so only its test arrived. Other originals conflicted on every run because they change code that never reaches the line. Batch now tells these cases apart instead of picking half a fix or retrying a conflict nobody can resolve. - Every file an original modifies or deletes is checked at its counterpart on the line, renames followed on both sides. If part of the original has no effect there (change to a file the line lacks, deletion of a file it never had, deleted file the unit keeps), the original is parked as partial and its release note stays out. A file that already matches what the original makes counts as there (same blob, binary too, or clean `git merge-file`). This runs in the picker, `fold`, `verify` and over units already on the open batch, so such a unit fails `check --cut` whatever its label until `run --defer` takes it out. - Pick that differs from its original keeps its release note under "Check these notes" with a `git range-diff` command to compare, record and report mark it `changed`. - Run names what an original builds on that the line lacks: the commit on `main` that added a file the fix modifies or deletes, and blame of removed lines whose text is still missing on the line. Generated files and generated keys of ApplicationDefinitions are skipped. Lines a fix only adds are not checked, a dependency that lives only there still needs `--drop`. - If a prerequisite never reaches the line (`feat`, direct push, excluded or itself parked) the original is parked and not tried. If it is still to come, the original waits for it and is retried later. Prerequisites the line or open batch already carry, or this run picked, count as there. `run`, `check` and record-only commands decide in the same order. On scheduled run a workflow original whose only missing dependency is in tests or docs waits for a maintainer instead of parking. `backport/include` lifts neither park, wait nor partial. - `notify` announces each park once on the open batch and marks it in the record, nothing is announced while no batch is open. Record moves to version 3 and report to format 2, format 1 reports still read. I ran the new tool next to the previous one without `--push` on today's history. On release-1.7 nothing changes category and `check` prints the same. On release-1.6 #5071 is parked on the two features it builds on, #5088 already on the open batch now fails the cut as a partial unit, and #5142 on scheduled run still waits for a maintainer with a note that only its tests depend on what 1.6 lacks. Rule 6 holds twelve originals on 1.6 right now, run over them directly flags the seven that change code 1.6 never got and none of the other five. Detection adds under 2 s per line. After this merges, CI on the open 1.6 batch goes red at its next push since `verify` refuses #5088 there. `run --line release-1.6 --defer 5088 --reason ... --push` takes it out. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(release): backport batch holds back a pick that applies only part of its original and fails the cut on such unit already on a batch, sets apart the note of a changed pick, parks an original that builds on code its line never gets, waits for one whose prerequisite is still to come, and announces each park once. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Backport batches now identify partial picks and missing prerequisites, and can park affected originals or leave them waiting. * Batch reports and notifications provide details about parked, partial, changed, and waiting picks. * Release notes for picks that differ from their originals are set apart for review. * **Bug Fixes** * Verification, folding, and batch checks reject picks that omit effects from the original; renamed paths are supported. * **Documentation** * Updated backport guidance to explain parking, prerequisites, partial picks, and release-note checks. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
f061739ce2 |
ci(e2e): raise the E2E job limit from 215 to 265 minutes (#5288)
## What this PR does Full chainsaw run doesn't fit in 215 minutes anymore. Last four nightly runs were cancelled at the limit, and PRs that select full suite end same way while every suite that got to run passed (#5283 for example). This raises limit for all four jobs that run the suite (pull-requests, e2e-fork, e2e-tag, nightly) from 215 to 265, so they stay on one figure. Comments that state the cap follow it. Making e2e itself shorter is separate work. Part of #5261, `Collect report` ceiling in fork workflow from there is not covered here. ### Screenshots ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note ci(e2e): E2E job limit raised from 215 to 265 minutes. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Chores** * Increased the time limit for end-to-end test runs from 215 to 265 minutes across applicable workflows. This allows longer test runs to complete before the overall job is stopped. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
e978ed7f2f |
fix(postgres): stop backup jobs from fighting over the WAL archive destination (#5221)
## What this PR does Part of #5202. A second PR will restore the last good destination when a job that changed it fails. The ObjectStore a Postgres Cluster archives its WAL to is shared by every backup of the application, yet every running BackupJob server-side applied its own strategy's destination onto it every five seconds with forced ownership. Two BackupJobs of one application with different BackupClasses flipped the destination back and forth until the healthy backup failed too, and a webhook or apiserver blip during that apply failed the BackupJob at once. - A BackupJob writes the destination and the Cluster plugin only before its `cnpg.io/Backup` exists. Afterwards it only reads the live Cluster, and a failed read is returned for a controller retry instead of failing the running backup. - Before writing, a BackupJob waits, with a `Ready=False` / `WaitingForOtherBackup` condition naming the other job, while another running BackupJob of the application has a `cnpg.io/Backup` in flight. The wait is not capped, for the same reason as in #5211. The queue reads the Cluster's own Backups from the apiserver; with leader election one reconcile runs at a time, so only a leader handover can still let two jobs through. An atomic per-application claim is #4559. - Before writing, a BackupJob also checks the apiserver for a restore of its application and holds while one runs, including a job released from the queue: a write landing during the restore would put the restored Cluster back on the source's WAL prefix. - Write errors that cannot succeed on retry (invalid, bad request, forbidden) still fail the job at once. Any other is retried with a `ClusterPluginApplyRetrying` condition for the backup deadline, counted from the first failure since the job last waited, and recorded in a new `DestinationApplied` condition. - The pass that creates the `cnpg.io/Backup` records on it the destination, endpoint, credentials Secret and serverName it wrote (`backups.cozystack.io/cnpg-written-destination`), and the artifact and its size probe are built from that record, so a strategy edited, or a Cluster re-rendered, while the backup runs does not redirect the artifact. A Backup without the record falls back to the strategy as before. - The plugin rollout deadline and the instance check count from the write that attached the plugin, not from the job's start. The apply that attaches it also stamps the time on the Cluster (`backups.cozystack.io/barman-plugin-attached-at`), so a lost BackupJob status write after the attach does not fall back to an older time, and a job that starts while another job's recent attach is still rolling waits for that rollout too. A stamp a few minutes ahead of the controller's clock is waited out; one further ahead is treated as corrupt. Tests drive the reconciler through a running backup that must not rewrite the destination, the queue and its release, a restore starting while a job waits, each class of write error, a recovered write, a failed Cluster read while the Backup runs, lost status writes around the attach, another job's rollout, clock drift on the attach stamp, the recorded destination and serverName, and the rollout deadline. Each guard was checked by mutation. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(postgres): Postgres BackupJobs of the same application no longer overwrite each other's WAL archive destination: a job writes it only before its backup starts and waits while another backup or a restore of the application is in flight. A transient error while writing it is retried instead of failing the job at once, and the backup artifact names the destination and serverName the backup was actually written to. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Reliability** * CNPG backups wait while another backup or restore is using the shared destination, and retry temporary destination-write failures while observing the backup deadline. * Backup artifacts and size checks use the destination recorded when the backup was created, even if destination settings later change. * Backup processing handles plugin rollout timing and destination-hold release more consistently. * **Restore** * Restore operations account for shared WAL archive destination conflicts and guard against incompatible backup selections. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2cf80c02a4 |
fix(clickhouse): keep failed backup attempts out of the bucket (#5229)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does A ClickHouse BackupJob attempt that fails during upload leaves its local backup on the data volume, and with the upstream defaults the clickhouse-backup sidecar never prunes local backups and resumes every unfinished upload when it starts. On the next sidecar restart the failed attempt is uploaded to the bucket as an archive no Backup refers to, and while that resumed upload runs the sidecar answers new operations with 423, which burns the strategy Job's retries in seconds. The strategy itself only waited for the sidecar API to respond and then started its operation regardless. The sidecar now runs with `API_COMPLETE_RESUMABLE_AFTER_RESTART=false`, so a failed attempt is never published, and `BACKUPS_TO_KEEP_LOCAL=-1`, so at most the newest local backup is kept and it is removed once its upload succeeds. Backups already left on existing volumes are cleaned up by the first backup after the upgrade. The Altinity strategy now waits until the sidecar has no operation in progress before it starts a backup or a restore, and fails the attempt with a clear message instead of proceeding when the sidecar stays busy or unreachable. Waiting for an idle sidecar changes what a retried strategy Pod does. The Job retries a failed Pod up to twice, and a Pod can die after its operation was accepted (a dropped poll under errexit, an eviction, a node drain) while the sidecar keeps running it. Before, the retry hit 423 and failed; with the idle gate it would wait for the running operation to finish and start it again. For a restore that is destructive, because `restore_remote` drops every table it restores before recreating it, so the second run discards whatever was written after the first one finished. A retried restore therefore takes over a `restore_remote` of the same archive that is still running and waits for its result instead of starting another. Once an operation is accepted, polls tolerate up to six failures in a row before the attempt fails, and the sidecar losing the operation from its in-memory action log, or ending it cancelled, fails the Pod instead of leaving it polling. A retried backup works from the archive name the controller records for its BackupJob. That name is now `<application>-<YYYYMMDDHHMMSS>-<first 12 hex digits of the BackupJob UID>`, so two runs of one application started in the same second, as two Plans on one schedule are, never share an archive. It is exactly as long as the previous `clickhouse-<application>-<YYYYMMDDTHHMMSSZ>`, so an `s3PathOverride` prefix that fit before still keeps every object key within the 1024-byte limit. A retry takes over a `create_remote --resume` of its archive that is still running, counts an upload of its archive whose latest run ended in `success` as its own result instead of uploading the archive again, and otherwise resumes the upload. Restoring a Backup that records no archive still picks an archive by timestamp, and that path accepts only `clickhouse-<application>-<timestamp>` names, so it never picks a per-run archive. One limit remains. A finished `restore_remote` row is not taken over: the sidecar never trims its action log, so a `success` row of the same archive may be left by an earlier RestoreJob, and counting it as this one would report a restore that never ran. A retry that arrives after the restore it replaces had already finished, but before that Pod saw the result, therefore still runs the restore a second time. Upgrading changes the ClickHouseInstallation pod template, so ClickHouse pods with `backup.enabled` are rolled once. A backup Job started before the upgrade by a controller that named archives the previous way is not re-rendered and keeps writing that name, so the controller records the previous name when the new one is not on the Job. The local copy of the latest failed attempt stays on the volume until the next backup. Objects partially written to the bucket by an upload that fails midway are not removed by clickhouse-backup and are left for a follow-up. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(clickhouse): failed ClickHouse backup attempts are no longer uploaded to the bucket when the backup sidecar restarts, local backups no longer accumulate on the data volume, the backup strategy waits for the sidecar to be idle instead of failing with HTTP 423, a retried ClickHouse restore takes over the restore that is still running instead of starting it a second time, and a retried ClickHouse backup takes over or reuses its own upload instead of starting a new one. Every ClickHouse backup run now writes an archive of its own, even when two runs start in the same second, under a name no longer than before. ClickHouse pods with backups enabled are restarted once on upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Backup and restore operations wait for the backup service to become idle and join a matching operation already in progress rather than starting a duplicate. * Restore operations use the archive the Backup records and fail when it is not in remote storage; only a Backup that records no archive falls back to the newest eligible archive by timestamp. * New backups use distinct archive names, including for runs started in the same second. * Local backup copies are discarded after upload, and completed uploads are not resumed after a service restart. * **Bug Fixes** * Operations now fail when the service cannot be queried, an operation disappears from its activity log, or it ends with an error or cancellation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
bc268ce759 |
fix(monitoring-agents): let the release uninstall when the VictoriaMetrics operator is gone first (#5152)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Disabling the Monitoring Agents addon of a tenant Kubernetes cluster leaves `<cluster>-monitoring-agents` (release `cozy-monitoring-agents`, installed into the tenant cluster) in `UninstallFailed` for good. Seen in production on 1.6.5 and still reproducible from `main`. `monitoring-agents`, `victoria-metrics-operator` and `vertical-pod-autoscaler-crds` in `packages/apps/kubernetes/templates/helmreleases/` are all gated on `addons.monitoringAgents.enabled`, so they disappear together and Flux uninstalls them in parallel: `dependsOn` is not honoured on deletion. Two things then break the uninstall of `monitoring-agents`: 1. The VictoriaMetrics operator is often gone first. The operator's `apps.victoriametrics.com/finalizer` stays on the `VMAgent`, nobody removes it, and the deletion hangs. The manual workaround was to strip the finalizer, after which GC removes the VMAgent's Deployment, Service, Secrets and ServiceAccount through their owner references. 2. The VPA CRD was deleted with its own release while the stored manifest still contains `VerticalPodAutoscaler vpa-vmagent`, so Helm cannot build the manifest to uninstall. That root cause is fixed by #3586 (`helm.sh/resource-policy: keep` on the tenant addon CRDs), which this PR does not redo. This PR: - adds a `pre-delete` hook (`templates/vm-cleanup-hook.yaml`) that deletes every VictoriaMetrics object the release owns, waits up to 60s for a live operator to finalize them properly, and clears their finalizers by hand when it has not. Objects are selected through Helm's `meta.helm.sh/release-name` / `release-namespace` annotations rather than by name, so the 34 `VMRule`s rendered from `alerts/` and the `VMServiceScrape` in `cozy-victoria-metrics-operator` are covered along with the VMAgent. The kinds handled are every VictoriaMetrics kind the chart renders: `VMAgent`, `VMRule`, `VMServiceScrape`, `VMPodScrape`, `VMNodeScrape`. Operator v0.68.4 only finalizes the VMAgent; the other kinds are covered in case another operator version finalizes them, and for them the patch is a no-op. A kind whose CRD is already gone is skipped, and the script always exits 0, since a failing pre-delete hook would itself block the uninstall. The Job is bounded by `activeDeadlineSeconds: 180` and `backoffLimit: 0`; its kubectl calls carry no `--request-timeout`, which would make kubectl skip the in-cluster credentials (`hack/check-hook-request-timeout.bats`) and reuses the digest-pinned `docker.io/clastix/kubectl:v1.32` image the etcd-operator hook uses, via a new `kubectlImage` value. The shape follows `packages/system/forklift/templates/forklift-cr-cleanup-hook.yaml`. - guards `templates/vpa.yaml` on `.Capabilities.APIVersions.Has "autoscaling.k8s.io/v1/VerticalPodAutoscaler"`, so a cluster that does not serve the CRD never records a VPA in the release manifest. Cheap defence in depth next to #3586. Points for review: - The hook's RBAC is a ClusterRole/ClusterRoleBinding, limited to `get, list, delete, patch` on the five `operator.victoriametrics.com` resources, rather than Roles. The objects span two namespaces, and a Role in `cozy-victoria-metrics-operator` could not be created if that namespace were already gone or terminating, which would fail the hook and the uninstall with it. All hook objects carry `hook-succeeded,hook-failed,before-hook-creation` except the Job, which keeps a failed pod for its logs (`ttlSecondsAfterFinished: 600`), as in the forklift hook. - When the operator is already gone, clearing the finalizer skips whatever the operator does on finalize, notably the cluster-scoped RBAC it creates for the VMAgent, which has no owner reference. That residue is the same the manual workaround left, and it is no worse than today's stuck release; removing it would mean hard-coding the operator's naming scheme for those objects, which I preferred not to do. - When the operator is alive, the hook only brings forward deletions Helm was about to make, so the operator's normal finalization still runs. Tests: `tests/uninstall_test.yaml` asserts that the hook is a `pre-delete` Job in the release namespace with a bounded runtime, that its RBAC is hook-scoped and grants exactly the verbs needed on every VictoriaMetrics kind the chart renders, that the binding targets the hook's ServiceAccount, that the script selects by release ownership, deletes, clears finalizers and exits 0, and that the VPA renders only when the capability is present (using `capabilities.apiVersions`). With the hook removed and `vpa.yaml` reverted, 6 of the 7 tests fail; with the fix, `helm unittest packages/system/monitoring-agents` passes 26/26. `hack/monitoring-agents-vm-cleanup-hook.bats` runs the rendered script against a fake `kubectl` that keeps the objects in a state file, with fake `date` and `sleep`. It checks that only objects annotated for this release are deleted and patched, that with the operator gone the hook polls through the grace period and then clears the finalizers, that a live operator's objects are never patched, that a missing CRD still exits 0 and cleans the other kinds, and that a release with nothing left exits 0 without deleting anything. Removing the ownership condition, making the patch a no-op, or inverting the polling condition each turns this file red. The hook has not yet been run on a live tenant cluster. Backport: labelled `kind/backport` (release-1.7). The release-1.6 backport is done by hand in #5158. The commit cherry-picks onto `release-1.6` with a single trivial conflict in `values.yaml` (context only: 1.6 lacks the `prometheus-node-exporter` resources block the new `kubectlImage` block sits after on `main`). Resolved locally, the chart's tests pass on 1.6 (9/9); 1.6's chart renders the same VictoriaMetrics kinds. Note that `release-1.6`'s monitoring-agents `Makefile` has no `test:` target, so CI does not run this chart's unit tests on that branch. ### Screenshots Not a UI change. ### Downstream repositories I walked the trigger map against the diff (a system chart's templates, `values.yaml` and tests): no package added or renamed, no platform values, no app schema, no CRD, no tooling or convention change. I also searched cozystack/website for a documented VMAgent finalizer workaround that this would make obsolete and found none. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(monitoring-agents): disabling the Monitoring Agents addon of a tenant Kubernetes cluster no longer leaves the monitoring-agents release stuck in UninstallFailed when the VictoriaMetrics operator is removed first; a pre-delete hook now releases the VictoriaMetrics objects' finalizers, and the VPA is only rendered when its CRD is served. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Bug Fixes** * VictoriaMetrics resources belonging to a release are cleaned up during uninstall, including resources that remain because of finalizers. Cleanup is limited to resources owned by that release. * Vertical Pod Autoscaler resources are rendered only when the cluster supports the required API. * **Configuration** * Added an image setting for the uninstall cleanup job, including a pinned image version and pull policy. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
583f18437d |
test(clickhouse): give the credentials-wait BackupJob a UID
The test pins that a backup held for late bucket credentials keeps its archive name, and a run's archive name now carries part of the BackupJob's UID, so the fixture needs one for the name it expects to be the name the controller mints. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
dae76d186c |
Merge branch 'main' into fix/clickhouse-idle-before-backup
main pins the archive name a backup Pod keeps across a wait for late bucket credentials, against the controller's archive helper. That helper now names a run's archive with part of the BackupJob's UID, so the merged test has to give its BackupJob one. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
5c6fc10a23 |
test(postgres): pin three guards no test caught when changed
A job whose own Backup exists skips the wait for another job's plugin rollout, and the size probe keeps the strategy's Secret key names while it takes the Secret from the record, adding a credentials block only when the strategy has none. Each of these could be changed with the suite still green: dropping the own-Backup check, or forcing the missing-credentials branch either way. Cases for a job with a running Backup next to a recent foreign attach, a strategy without credentials and a strategy with custom key names now fail on each change. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
238b3ee491 |
fix(seaweedfs): enforce S3 bucket lifecycle rules (#5228)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does SeaweedFS accepts and stores an S3 bucket lifecycle configuration, but only the `s3_lifecycle` job of its plugin worker applies it, and that job is scheduled by the admin server. Cozystack deployed neither, so a lifecycle rule returned 200 and no object ever expired, on the platform backup bucket and on tenant buckets alike. That also made the documented way to bound backup storage a no-op. This PR adds the admin server and a worker to Simple and MultiZone SeaweedFS instances, behind a new `lifecycle.enabled` value that is **off by default**. A Client instance is unchanged, because its rules are enforced where the instance it connects to runs. The worker registers for `s3_lifecycle` only, so vacuum, balancing and erasure coding are not started as a side effect. The admin keeps its job state on a 1Gi volume on `lifecycle.storageClass`, or on the cluster default when that is empty; `volume.storageClass` is not reused, because in MultiZone it can name a class local to one zone or node. The admin UI is protected by a generated password, because a tenant namespace admits traffic from the whole cluster. The dashboard Role lists the two new WorkloadMonitors. Enabling the worker alone was not enough on SeaweedFS 4.45. Its chart renders no `[grpc.s3]` section, so the s3 gateway served its internal gRPC in plaintext while the worker dials it with mTLS, and every lifecycle delete failed the TLS handshake. The section is backported from upstream (seaweedfs/seaweedfs#11157, released in 4.46) as a chart patch, applied with `--forward` so the next `make update` stops at the point where the patch must be dropped instead of silently reversing it. The s3 pods are restarted once so they load the section. This also repairs IAM cache propagation between s3 replicas, which failed in the same way, and it applies whether or not lifecycle is enabled. The data-plane release now retries a failed install or upgrade in place (`RetryOnFailure`) instead of rolling it back. With the previous default remediation and unlimited retries, an upgrade that waited out its timeout, for example on a tenant quota with no room for the admin volume, rolled back and retried forever, restarting the s3 pods twice per cycle. The worker's hostname anti-affinity is relaxed to a preference so a rollout finishes on a single-node cluster. #### Why lifecycle is off by default The daily walk that SeaweedFS uses to expire objects sends its deletes without an object identity, and the s3 server skips its compare-and-swap when no identity is given; it does not re-check the object's age either. A key overwritten between the walk's listing and the delete loses its new content. This path is unchanged from 4.45 through 4.52 and on upstream master (`weed/s3api/s3lifecycle/dailyrun/walker_dispatcher.go`, `identityMatches` in `weed/s3api/s3api_internal_lifecycle.go`). Backup buckets write unique keys and are barely exposed; a tenant bucket that rewrites a fixed key is. On top of that, turning enforcement on would start applying, on upgrade, every rule that was stored and ignored until now. Both are retention changes an operator should choose, so nothing changes for existing instances until `lifecycle.enabled` is set. Making it the default again is left to a separate PR that carries object identity on the walk path; upstream report: https://github.com/seaweedfs/seaweedfs/issues/12451. #### Turning lifecycle on and off When `lifecycle.enabled` is set, rules that have been stored but ignored start applying within minutes, and objects they have already expired are deleted on that first pass, so review existing bucket lifecycle rules first. Set `lifecycle.storageClass` before enabling, and prefer a replicated class: the class is part of the admin StatefulSet's volume claim template, which cannot change, so changing it while lifecycle is on makes every upgrade of the instance fail until it is reverted. Moving the volume means disabling lifecycle, deleting the `admin-data-seaweedfs-admin-0` PVC, changing the class and enabling lifecycle again. Each instance gains two small pods and a 1Gi PVC; when the tenant quota has no room for them, the data-plane release keeps retrying in place until it has. When `lifecycle.enabled` is turned off again, the parent release removes the admin credentials Secret while the data-plane release removes the admin StatefulSet only on its own reconcile, shortly after. An admin pod that restarts in that window fails to start on the missing Secret until the StatefulSet is gone; nothing is lost, and the order is left as is. The admin PVC stays until the instance is deleted, and a later re-enable starts the admin with a new password. Verified on a test cluster with the 4.45 chart: without the backport, the worker found the expired objects and every delete failed with `tls: first record does not look like a TLS handshake`; with this branch and lifecycle enabled, a rule with a past expiration date removed the matching objects and left the others, both on a manual run and on the first scheduled pass. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up needed, not opened yet: `operations/services/backup-classes.md` (v1.7 and next) advises an S3 lifecycle policy, which the bundled SeaweedFS applies only once `lifecycle.enabled` is set; the SeaweedFS reference page gains `lifecycle.enabled` and `lifecycle.storageClass` from the regenerated README on the next release - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The terraform provider does not model the SeaweedFS application, so the new values do not reach it. ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(seaweedfs): S3 bucket lifecycle rules can now be enforced. Simple and MultiZone SeaweedFS instances run the SeaweedFS admin server and a lifecycle worker when the new `lifecycle.enabled` value is set; it is off by default, so existing instances keep their current behaviour. It stays off by default because the upstream daily expiry walk deletes without checking that the object it listed is still the one stored, so an object overwritten in that window can be deleted although it is new. When enabled, rules that were stored but ignored start applying within minutes and objects they have already expired are deleted on that first pass; review existing rules first. Each enabled instance gains two small pods and a 1Gi volume, placed on `lifecycle.storageClass` or the cluster default. Independently of the switch, the s3 gateway now serves its internal gRPC over mTLS, which repairs IAM cache propagation between s3 replicas and restarts the s3 pods once on upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Added optional, daily S3 bucket lifecycle-rule processing for supported SeaweedFS configurations. It is disabled by default; Client instances rely on lifecycle rules configured on the connected SeaweedFS. * Added configurable storage for lifecycle job state and dashboard visibility for lifecycle components. * Enabled the S3 gRPC server to use its internal TLS identity when security is configured. * **Bug Fixes** * Improved storage-class handling so numeric-looking names remain strings and unset classes are omitted where appropriate. * Clarified backup cleanup guidance, including when lifecycle rules apply and how to reclaim storage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
3badbdef74 |
fix(clickhouse): let a running backup sidecar pick up system-bucket credentials (#5232)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does A ClickHouse release with `backup.useSystemBucket: true` can start before the platform projects `cozy-backups-creds` into its namespace. Its `clickhouse-backup` sidecar took the S3 settings from environment variables, which are fixed when the container starts, so the first BackupJob or RestoreJob of such a release failed with `no EC2 IMDS role found` until the Pod was restarted, and a rotated key stayed stale the same way. The projector now also writes a `clickhouse-backup.yml` key into `cozy-backups-creds` (bucket, `https://` endpoint, region when set, keys, `force_path_style` from `backupStorage.forcePathStyle`). On the system-bucket flow the chart mounts that key as an optional Secret volume and points `CLICKHOUSE_BACKUP_CONFIG` at it, instead of setting `S3_*` variables. clickhouse-backup reloads its config file in each API handler that reaches S3 (among them the `create_remote`, `restore_remote` and `list` calls the strategy makes; the `status` and `actions` reads it polls with do not) and kubelet rewrites a mounted Secret in place, so credentials projected after the Pod started, or rotated since, reach the running sidecar. `S3_PATH` stays in the environment because the prefix belongs to the release. Kubelet takes about a minute to refresh the volume, so the projector stamps `backups.cozystack.io/credentials-updated-at` when the projected content changes, and the Altinity driver holds a new backup or restore Job for up to two minutes after that stamp, with `Ready=False` and reason `WaitingForCredentials`; the condition is dropped once the Job runs. A stamp that cannot be read, or one further ahead than clock skew explains, is logged and ignored rather than holding the namespace's jobs for good. The hold applies to every Altinity job in a namespace whose projected Secret just changed, including releases on their own Secret, where it is only a delay. The new e2e Test `clickhouse-3-system-bucket-roundtrip` starts a ClickHouse on the system bucket with no projected Secret, then runs a backup and an in-place restore through `cozy-default` and checks that the sidecar that started without credentials served both jobs. In CI the system bucket's advertised host `s3.example.org` neither resolves nor gets an ACME certificate, so for the duration of its own step the Test serves a SeaweedFS-CA certificate under the S3 Ingress TLS Secret name and adds a marked `rewrite name exact` line for that host to the host CoreDNS Corefile, removing both on exit; it proves the served certificate before any job starts. The bug was reproduced on a test cluster running the code before this change: a ClickHouse started before the projection failed its first BackupJob with `no EC2 IMDS role found`. The new e2e Test has not run green yet outside CI, because it edits the host CoreDNS, which is shared on that cluster. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#799 (draft until this PR merges): `operations/services/backup-classes.md` for the mounted config file, the credentials wait and the upgrade window, without the Pod restart this change removes - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(clickhouse): a ClickHouse release on the platform system bucket (`backup.useSystemBucket: true`) now backs up and restores without a Pod restart when its namespace had no projected backup credentials at start, and picks up rotated credentials while running. The sidecar reads its S3 settings from a `clickhouse-backup.yml` file in `cozy-backups-creds` instead of environment variables, so the upgrade rolls every such ClickHouse Pod once. The first Altinity backup or restore in a namespace after its projected credentials change waits up to two minutes (`WaitingForCredentials`) for them to reach the sidecar. Upgrade the platform as a whole: a system-bucket BackupJob started while the new chart runs against the previous controller fails, because that controller does not write the new key. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * ClickHouse system-bucket backups now use a projected configuration file for S3 settings, supporting HTTPS endpoints and optional region and path-style settings. * Backup and restore jobs wait briefly for recently updated projected credentials before starting. * Path-style settings accept common true and false values and are normalized consistently across backup storage integrations. * **Tests** * Added end-to-end coverage for system-bucket backup and restore, verifying restored data without restarting the ClickHouse Pod or backup sidecar. * Added checks for TLS certificate validation and safe restoration of shared network and ingress settings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
1d22013646 |
fix(postgres): hold a backup until the plugin sees its new destination
The plugin-barman-cloud sidecar reads the ObjectStore through its own cache, kept for ten seconds (DefaultTTLSeconds in plugin-barman-cloud v0.13.0) and invalidated only by the sidecar's own writes, and WAL archiving keeps it warm. The pass that changed the destination created the cnpg.io/Backup a moment later, so the base backup could be taken against the previous destination while the artifact names the new one, and a restore from it finds nothing. On a live cluster with the shipped plugin, a Backup created two to four seconds after switching the destination landed in the old bucket in two of three tries; with a fifteen second wait it landed in the new one every time. The apply that changes the ObjectStore configuration now stamps the time on the ObjectStore itself, so the record cannot be lost apart from the change, and the job creates its Backup only once that change is older than the cache can be. A change is judged on the configuration a server-side dry run of the same apply returns, so values the apiserver defaults or another field manager owns do not read as a change on every pass. That wait leaves a job with the destination written and no Backup yet, which the queue, looking only for Backups in flight, did not see: a second job with another destination wrote its own, the first took that as a change and stamped again, and the two restarted each other's wait forever. Every apply of the ObjectStore therefore also names the job that made it, which only a job without a Backup of its own and let through by the queue does, and the queue holds every other job while that one runs and has no Backup yet. Once it has one, that Backup holds the destination for as long as CloudNativePG runs it, as before, so a job stuck after its Backup finished does not hold every later backup of the application. This also covers a job waiting for the rollout of the plugin it attached, during which a second job used to switch the destination as well, including when its own write left the configuration as a restore had re-rendered it. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
aa97451bc2 |
fix(platform): leave tenant-root's ingress empty when the root tenant has none (#5226)
## What this PR does Enabling SeaweedFS on the root tenant without ingress no longer hangs. `cozystack-basics` wrote `_namespace.ingress: tenant-root` for the root tenant whatever its `spec.ingress` said, so `seaweedfs-system` depended on an `ingress-nginx-system` release that nothing creates, and the platform backup bucket waited with it. Every other tenant already derives the key in `packages/apps/tenant/templates/namespace.yaml` and leaves it empty without ingress. `cozystack-basics` cannot read the root Tenant, so the platform now carries its ingress state the same way it carries monitoring: the new platform value `ingress.rootEnabled` reaches the `cozystack-values` Secret as `_cluster.ingress-enabled`, and `_namespace.ingress` is written only when it is true. The default is `true`, which keeps the value every existing cluster renders today, and a Secret written before the key existed keeps it as well. An operator running the root tenant without ingress sets `ingress.rootEnabled: false`, kept in sync with `Tenant/root spec.ingress` by hand, as with `monitoring.rootEnabled`. With the key empty, `seaweedfs-system` drops the dependency through the check it already had (`eq $ingress .Release.Namespace`), and nothing reads live cluster state for it. An empty `_namespace.ingress` now reaches tenant-root and every tenant inheriting from it, so the consumers that would break on it are adjusted in the same PR. The API rejects `ingressClassName: ""` (checked with a server-side dry run), and two consumers quote the value: - `kubernetes`: the proxied ingress-nginx addon Ingress is not rendered without an ingress; its Service still is. - `monitoring`: Grafana's `spec.ingress` is left out, since grafana-operator copies it into the Ingress verbatim. - `seaweedfs`: the S3 Ingress is disabled instead of rendering with no controller to serve it. The `KamajiControlPlane` `network.ingress` block is unchanged on purpose: its `hostname` is what the Kamaji provider uses as the control plane endpoint, and Kamaji itself skips an empty class name. Harbor, bucket, bootbox and alerta write the class unquoted, so an empty value renders as an unset class, which the API accepts. Tests: `packages/core/platform/tests/apps_ingress_enabled_wiring_test.yaml` pins the flag reaching `_cluster` both ways and with `ingress` set to null; `packages/system/cozystack-basics/tests/cozystack-values-secret_test.yaml` pins `_namespace.ingress` for a missing, string, bool and `"false"` flag, and that the other keys stay when ingress is off; `packages/extra/seaweedfs/tests/ingress_dependency_test.yaml` pins the dependency and the S3 Ingress for an own, empty and inherited ingress; the kubernetes and monitoring suites pin the gated Ingress. Reverting any of the gates, or flipping the missing-key default, fails the suites. Not covered here: - `_namespace.etcd` has the same hardcoded shape (#4783) and is left for its own change. - Toggling `Tenant/root spec.ingress` without changing `ingress.rootEnabled` leaves the value stale, as `monitoring.rootEnabled` does today. ### Screenshots Not a UI change. ### Downstream repositories The diff adds a key to `packages/core/platform/values.yaml`, which the website documents by hand. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#800 documents `ingress.rootEnabled` in the platform Package reference - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(platform): a root tenant without ingress no longer leaves seaweedfs-system waiting for an ingress-nginx-system release that is never created. Set the new platform value ingress.rootEnabled to false to match Tenant/root spec.ingress: false. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Added a root-tenant ingress setting, enabled by default, that can be turned off. * **Bug Fixes** * Kubernetes and Grafana ingress resources are now rendered only when ingress is configured. * SeaweedFS S3 ingress is enabled only when ingress is configured. * Ingress-dependent components now respect the root-tenant setting, while other platform services remain available when root ingress is off. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
76a34d3e53 |
fix(seaweedfs): bump the COSI driver to v0.4.0 for the lost-grant fix (#5284)
## What this PR does This PR bumps seaweedfs COSI driver to v0.4.0, because v0.3.1 can lose a grant for good: BucketAccess reports `accessGranted` and its Secret is written, but S3 gateways keep rejecting the key with `InvalidAccessKeyId`. Fixes #4912. Driver v0.3.1 grants access by rewriting legacy `/etc/iam/identity.json`, and every S3 gateway migrates this file to per-identity files and renames it away on each change, so grant written during migration is dropped. v0.4.0 also writes `/etc/iam/identities/<user>.json` on grant (seaweedfs/seaweedfs-cosi-driver#15). First version of this PR built the driver from that upstream commit, now upstream released v0.4.0 so it's just a tag bump. Rest of v0.4.0 doesn't reach the platform: - `readwrite-governance-bypass` access policy (seaweedfs/seaweedfs-cosi-driver#16) is opt-in, none of our BucketAccessClasses use it. - Bucket deletion is refused while objects hold an active lock (seaweedfs/seaweedfs-cosi-driver#18), only for object-lock buckets. All `-lock` BucketClasses are `Retain`, and sidecar calls DriverDeleteBucket only for `Delete`. - Filer errors (seaweedfs/seaweedfs-cosi-driver#20) are now matched by exact not-found message, I checked it against seaweedfs 4.45 filer and formats match. One behavior change: bucket deletion doesn't ignore errors of the recursive delete anymore, so sidecar retries instead of reporting success. How I tested (seaweedfs 4.45, two S3 gateways): Bucket with one user, then three more users added in one change, after a minute ListObjects with each user key, rejected key retried once more after 30s. Five runs for each driver, the fixed one is a build of `5efd120`, the commit v0.4.0 takes the grant fix from: | driver | keys rejected for good | | --- | --- | | v0.3.1 | 5 of 20 (4 runs out of 5 lost at least one) | | `5efd120` | 0 of 20 | Same failure broke v1.7.1-rc.1 e2e, RabbitMQ backup round-trip got `403 InvalidAccessKeyId` for freshly granted key: https://github.com/cozystack/cozystack/actions/runs/42980741995 Keys already lost on running clusters stay broken after upgrade. Workaround from #4912 still works: remove user from Bucket, wait until its BucketAccess is gone, add it back. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(seaweedfs): bucket users granted together could get S3 keys that never work (403 InvalidAccessKeyId). COSI driver is bumped to v0.4.0, which writes each grant to per-identity IAM file. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Updates** * Updated the SeaweedFS COSI driver to version `v0.4.0`. * The updated driver writes IAM files per identity, helping preserve grants when identity files are updated. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c63e83466a |
ci(e2e): raise the E2E job limit from 215 to 265 minutes
A green run of the full Chainsaw suite no longer fits in 215 minutes: the last four nightly runs were cancelled at the limit, and pull requests that select the full suite end the same way while every suite that got to run passed. All lanes that run the suite keep one figure, so a change that passes on a pull request is not cut off on nightly or on a tag. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
0b284a65d5 |
test(release): isolate report deduplication from workflow attempts (#5282)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, security (a fix that comes through a security advisory), docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Fixes merged to `main` reach the release lines the backport batch runs for automatically, so do not open backport PRs by hand: https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md --> ## What this PR does Pins the report deduplication test to the first workflow attempt. In a CI rerun, the inherited attempt otherwise adds `/attempts/2` to the synthetic run URL and fails the existing exact-URL assertion. Production run URLs and the deduplication assertions stay unchanged. Validation: a source-derived check of the test setup and original URL assertion fails with an inherited second attempt and passes with the proposed first-attempt setup. Controls with the pin after preview or set to 2 still fail. The full Git-backed test was not run locally; CI must verify it. ### Screenshots Not applicable; no UI changes. <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note test(release): make the report deduplication test independent of the surrounding workflow attempt. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Tests** * Updated automated test configuration to account for GitHub Actions run attempts. No user-facing behavior changes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
5a95bcdd64 |
fix(clickhouse): keep a run's archive name within the old name's length
The archive name is part of every object key, a key is capped at 1024 bytes, and the legacy s3PathOverride prefix has no length limit. The full UID made the name 37 characters longer than before, so a prefix that fit the old name pushed the manifest key past the cap and failed the backup. The run's name now drops "clickhouse-" and the "T" and "Z" of the timestamp and spends the room on the first 12 hex digits of the UID, so it is exactly as long as the old name for every application. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
6a93765351 |
fix(seaweedfs): bump the COSI driver to v0.4.0 for the lost-grant fix
seaweedfs-cosi-driver v0.3.1 grants access by rewriting the legacy /etc/iam/identity.json. Each S3 gateway migrates that file to per-identity files and renames it away whenever it changes, so a grant written while a gateway is migrating is lost: the BucketAccess reports accessGranted and its Secret is written, but every gateway rejects the key with InvalidAccessKeyId and never recovers. Users added to a Bucket in one change lose a key regularly. v0.4.0 also writes the per-identity file on grant. The rest of v0.4.0 does not reach the platform. The readwrite-governance-bypass access policy is opt-in and no BucketAccessClass uses it. The refusal to delete a bucket that still holds locked objects applies only to object-lock buckets, and their classes are Retain, so the sidecar never calls DriverDeleteBucket for them. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
67fcadf689 |
fix(seaweedfs): disable the S3 Ingress in a tenant without ingress
The S3 Ingress kept rendering with no ingress-nginx to serve it. Now that seaweedfs-system no longer waits on ingress-nginx-system there, the release completes and the unserved Ingress would stay behind. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
275c840a8c |
fix(monitoring): leave Grafana's ingress out in a tenant without ingress
grafana-operator creates the Ingress from spec.ingress verbatim, so an empty _namespace.ingress reached the API as ingressClassName "" and was rejected. With no ingress-nginx in the tenant there is nothing for that Ingress to attach to. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
c61e6a842e |
fix(kubernetes): skip the proxied addon Ingress in a tenant without ingress
An empty _namespace.ingress means no ingress-nginx serves the tenant, which tenant-root now reports when the root Tenant has ingress off. The Ingress would carry ingressClassName "", which the API rejects, and the release would fail to install where it used to leave an unserved Ingress behind. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
b26001b6f9 |
test(seaweedfs): pin the ingress-nginx dependency to the namespace's ingress
seaweedfs-system waits on ingress-nginx-system only when _namespace.ingress names its own namespace. With tenant-root's key now empty when the root Tenant runs no ingress, an empty value has to drop the dependency, and a value inherited from a parent must not add it. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
5b6708c7f9 |
fix(platform): leave tenant-root's ingress empty when the root has none
cozystack-basics wrote _namespace.ingress: tenant-root whatever the root Tenant's spec.ingress said, so applications in tenant-root were told an ingress-nginx runs there when nothing deploys it. SeaweedFS then waited forever on ingress-nginx-system, and the platform backup bucket with it. Every other tenant already derives the key and leaves it empty without ingress. cozystack-basics cannot read the root Tenant, so the platform carries its ingress state the way it carries monitoring: ingress.rootEnabled reaches the Secret as _cluster.ingress-enabled and gates the key. The default keeps the value existing clusters render, and a Secret written before the key existed keeps it too. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
9978771858 |
feat(observability-proxy): serve tenant namespace logs through an authorizing proxy (#5214)
## What this PR does Adds `observability-proxy`, a new system component that lets a tenant user read the logs of their own namespace from VictoriaLogs with the same Kubernetes credentials they already use for the console, and with Kubernetes RBAC deciding what they can read. This is the first part of the design proposals [Tenant application logs](https://github.com/cozystack/community/pull/95) and [Tenant resource-consumption metrics](https://github.com/cozystack/community/pull/94); this PR implements the namespace logs root only. Metrics and the per-application roots follow in separate PRs. Why a proxy: today the log stores are shared per root tenant and have no notion of a caller, so tenants cannot be given direct read access without seeing other tenants' logs. The proxy authenticates the caller, authorizes the request from its path, and rebuilds the upstream query so that its scope cannot be widened by the client. Backward compatibility: this adds a new package and a new HelmRelease; nothing existing is renamed, removed or changed, so no migration is needed. What is included: - `cmd/observability-proxy` and `internal/observabilityproxy`: the proxy. - Serves `/apis/cozystack.io/v1alpha1/namespaces/<ns>/logs/select/logsql/<endpoint>` for the VictoriaLogs read endpoints the console needs (`query`, `hits`, `stats_query_range`, `tail`, `field_values`, `stream_field_values`, `facets`). Anything else under the root answers `404`. - Authentication by `TokenReview`, authorization by one `SubjectAccessReview` built from the path (`list` on `cozystack.io/logs` in the namespace), both cached briefly. - The upstream request is rebuilt from an allowlist of parameters; the namespace scope is always forced as `extra_stream_filters`. Client-supplied `extra_filters`, `extra_stream_filters`, any LogsQL `options(...)` block, and tenant headers such as `AccountID`/`ProjectID` are not forwarded or are refused. - Non-canonical paths (`..`, `//`, encoded separators) are refused with `400` before any authorization is built, because the console nginx matches on the normalized path but forwards the raw one. - Time window, step, bucket count and result limits are clamped; per-user rate, concurrency and open-tail limits answer `429`. Invalid time, step and limit parameters answer `400`. - The store is resolved from the namespace's `namespace.cozystack.io/monitoring` label. - Live `tail` streams are passed through unbuffered, and the upstream tail request disables compression so headers reach the client before the first line. - `packages/system/observability-proxy`: chart and image build, wired into the root `make build`. - Platform wiring: a new `cozystack.observability-proxy` PackageSource, enabled by default on the full variants and opt-in on the slim ones, the same way as `monitoring-agents`. - RBAC: an aggregated `cozy:observability:view` ClusterRole grants `list` on `cozystack.io/logs` to tenant `view` and above. No API server serves this resource; the rule exists only for the proxy's `SubjectAccessReview`. The proxy's own ClusterRole is limited to the calls it makes (`tokenreviews`, `subjectaccessreviews`, `list`/`watch` on namespaces). - Console nginx: when the proxy is not installed or cannot be reached, the observability roots answer `404` with a Kubernetes `Status` body, which the console shows as not available on this cluster; the proxy's own `502`/`504` for an unreachable store pass through unchanged. - Console: one regex `location` in the console nginx routes the four observability roots to the proxy ahead of `/apis`. The upstream is resolved at request time, so the console starts even when the proxy is not installed; those paths then answer `404` with a Kubernetes `Status` body. The console Deployment now receives the cluster domain for that resolution. - E2E: a new Chainsaw suite `hack/e2e-chainsaw/observability-proxy/` with two tenants, covering `401`/`403`/`200`, scope-widening attempts, path normalization and a long-lived `tail`. #### How it was tested - `go test -race` and `go vet` for the proxy packages. - helm-unittest for `observability-proxy`, `dashboard`, `tenant` and `core/platform`; bats tests for the console nginx routing. - The Chainsaw suite was not run as-is: it deploys its own VictoriaLogs as the root store and refuses to run when a real one exists. Its cases were replayed by a script against the real store on a test cluster. - Verified on a test cluster: - `401` without a token and with an invalid token, `403` for another tenant's namespace, `200` for the caller's own. - Client `extra_stream_filters`/`extra_filters` (GET and POST), an `AccountID` header and another tenant's stream named inside the query do not widen the scope; `options(...)` answers `400`; a path with `..` pointing at another tenant answers `400`; an endpoint outside the allowlist answers `404`. - Through the full browser chain (OIDC login, gatekeeper, gateway, console nginx), a single `tail` stayed open for more than two minutes without reconnecting, and requests from the same session to other tenants' roots answered `403`. - Not verified: the `429` limits under real load. `golangci-lint` has not been run locally. #### Known limitations - The proxy is reachable only through the console route; tenant workloads have no egress rule to it. Every request takes its concurrency or tail slot before the body is read; the token and access reviews and the body read run under `--request-timeout`, so a stalled kube-apiserver answers `503` rather than holding the request; and writes to the client carry a deadline (renewed per line for tails). `--request-timeout=0` disables those bounds and should not be used in production. - The first lines a newly started pod writes can be missing from an already open `tail`, although they are in the store. The likely cause is that Fluent Bit picks up a new log file later than the VictoriaLogs live-tail window; later lines of the same pod arrive. A history query shows them. - The per-application roots (`.../<plural>/<name>/log`, `.../metrics`) and the namespace metrics root answer `404` in this PR. ### Screenshots No UI changes. ### Downstream repositories - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/issues/787 (docs land once the implementation PRs merge) - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Downstream: `cozystack/website` is reached. A new platform component (`packages/core/platform/sources/observability-proxy.yaml`) triggers `content/en/docs/next/guides/platform-stack/_index.md` and `operations/configuration/licenses.md`; adding it to the system bundle (`packages/core/platform/templates/bundles/system.yaml`) may also need a line in `operations/configuration/variants.md`. A user-facing page for the logs API and the new tenant `view` permission is not required by the trigger map but would be useful. Tracked in cozystack/website#787. Other repositories: no trigger matched. `packages/core/platform/values.yaml` is not changed (only a test fixture), no package under `packages/apps/` gains a values field, and no CRD changes. The root `Makefile` `build` target builds one more image; this does not touch `hack/` or the make targets `ccp` relies on, so `ccp` is not considered affected, but a reviewer may want to confirm. ### Release note ```release-note feat(observability-proxy): add an authorizing proxy that lets tenant users read their namespace logs from VictoriaLogs with their Kubernetes credentials, under `/apis/cozystack.io/v1alpha1/namespaces/<ns>/logs/`. Access requires the tenant `view` role or higher. The component is enabled by default on full variants and opt-in on slim variants. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Added an observability proxy for tenant-scoped log access through the console, including live log streaming. Requests are authenticated, access-controlled, and subject to configurable rate and concurrency limits. * Added platform packaging and deployment support, with console routing for observability requests. * **Tests** * Added automated coverage for access control, tenant scoping, query handling, streaming, proxy behavior, and deployment configuration. * **Documentation** * Updated end-to-end test guidance with host requirements for running the observability proxy suite. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
9fe06a874b |
test(release): isolate report deduplication from workflow attempts
CI reruns expose GITHUB_RUN_ATTEMPT to a test that expects a URL for
the first attempt. Keep its synthetic run identity independent of the
surrounding workflow.
Fixes:
|
||
|
|
35823c6c9c |
test(clickhouse): list the system-bucket round-trip in the suite header
The header names the backup Tests of the suite and why they are split; it still described two of them after the third was added, so a reader starting from it would miss the only Test that covers the system-bucket path. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
8ba0380043 |
docs(clickhouse): name the clickhouse-backup handlers that reload the config
clickhouse-backup reloads its config file in each handler that reaches S3, among them the create_remote, restore_remote and list calls the strategy makes. The status and action-log endpoints the strategy polls read in-memory state and do not reload it, so the mounted file reaches the sidecar on the calls that need credentials rather than on every request. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
3b4db69f8e |
feat(backups): record the RDB snapshot location and size for Redis backups (#5046)
## What this PR does Builds on the shared S3 probe from #5223 (MariaDB and CNPG), which is on `main`. Refs #4316. Backups made by the Redis strategy carry no `Backup.status.artifact` at all: no URI and no size. This covers both the Redis and Valkey apps, which share the driver. The snapshot is a single object per backup, `<namespace>/<app>/<backup>.rdb` (`redisObjectKey`). The driver already records that key in `driverMetadata`. The default strategy uploads the object using the S3 coordinates from the backup credentials Secret, which the controller projects into the application namespace (`endpoint`, `bucketName`, `region` and the access keys). When a backup completes, the driver now: 1. reads that projected Secret (`CredentialsConfig.TargetSecretName`) in the application namespace. The projector stores its `endpoint` without a scheme, and the strategy script reaches it over https; an endpoint with a path prefix is refused by minio-go, which leaves the artifact unset; 2. reads the TLS trust from the pod spec of the completed backup Job, in the order the script applies it: `ca.crt`, then `tls.crt`, from the Secret mounted as the `s3-ca` volume (`backupStorage.endpointCASecretName`). Only when there is no `s3-ca` volume does it honour `S3_INSECURE_SKIP_VERIFY=true`, and only when every container of the pod sets it, since the probe cannot tell which container uploaded. The Secret is read at probe time; the default strategy mounts `s3-ca` only when an endpoint CA is configured, so when the volume yields no CA now (the Secret is gone or lacks both keys, or the volume is not a Secret), the probe leaves the artifact unset rather than trust the system roots or skip verification where the Job may have verified. The CA is trusted on its own, as `curl --cacert` does, not on top of the system roots; 3. runs `StatObject` on the recorded key; a response without `Content-Length`, which minio-go reports as size -1, counts as unmeasured; 4. records `status.artifact.uri = s3://<bucket>/<key>` and `status.artifact.sizeBytes` = the object's size. An empty object is recorded too, but `sizeBytes` is `omitempty`, so on the Backup it reads like an unmeasured size; since a valid RDB snapshot always carries a header, a Warning `EmptySnapshot` Event on the BackupJob flags it as a broken backup. The artifact is set **only when the object is found**. A custom strategy that writes elsewhere would otherwise get a URI pointing at an object that does not exist. On any failure (Secret missing or incomplete, endpoint invalid, CA unreadable, object not found, TLS error) the artifact stays absent and the backup still succeeds. The full error is logged at info, and once the Backup is created a Normal `ArtifactNotMeasured` Event on the BackupJob gives a fixed reason, also when a failed existence check skipped the probe; the Event lives in the application namespace, so it never carries the raw error, which may name the platform's endpoint and bucket. The probe shares the 10s timeout of the MariaDB and CNPG probes. The retry check is shared with those drivers too: a `Backup` that already exists is returned without probing, a failed existence check creates the Backup without probing, a `Create` collision is resolved by reading past the cache, and an existing Backup that cannot be read back is retried with `Ready=False` instead of failing the BackupJob. The size is useful because an RDB snapshot is the compressed dataset. It shows growth, and a snapshot that is suddenly tiny is a broken backup. Kafka is deliberately left out. Its backup is a topic-metadata export of a few kilobytes, so the size would say nothing about the data and might suggest the data itself was backed up. Tests: - the stat uses the projected credentials; - every path that leaves the artifact unset: object missing, Secret missing, bucket, endpoint, access key or secret key missing, no target Secret configured, a size the endpoint did not report; - trust resolution: `ca.crt` wins over `tls.crt` and skip-verify; `items` remapping of the volume, and a key the `items` do not project is never trusted; the Redis CA volume mounted next to `s3-ca` is not taken as the S3 CA; an `s3-ca` volume that yields no CA (Secret gone, Secret without the keys, a ConfigMap source) is refused with and without skip-verify; skip-verify applies only on an exact `true` of that variable in every container, the last entry of a repeated variable winning; - the real minio client against a TLS stub: its certificate is trusted only through the `s3-ca` volume or skip-verify, a CA that did not sign it is refused even with skip-verify set on the Job, and the shared client enforces a configured CA with and without the system roots; the client sends a signed `HEAD` and parses `Content-Length`; - the Events: an unmeasured artifact or a failed existence check raises `ArtifactNotMeasured` without the raw error; an empty snapshot is recorded and raises a Warning `EmptySnapshot`; a measured artifact, a rejected `Create` and a `Create` collision raise none; - a reconcile test pinning that the completed Job's pod spec reaches the probe; a Backup that already exists is returned without a probe or a `Create`; the `Create` collision is read back from the API server past the cache. The suite fails if any guard of the trust lookup (volume name, Secret source, variable name, every-container rule, refusal of an unreadable CA), the CA enforcement in the transport, the negative-size check, the Events, the reason-only message, their place after `Create`, the empty-snapshot warning, the existing-Backup check, the wiring, any term of the Secret check, or the API-server read-back is removed. ### Screenshots N/A — no UI changes. ### Downstream repositories I walked the trigger map against the diff. The change is Go-only in `internal/backupcontroller`, with no API, CRD, chart, value or schema change. Existing `Backup` fields are now filled for one more driver. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(backups): Redis and Valkey backups now record `status.artifact` with the `s3://` location and the size of the RDB snapshot. If the snapshot cannot be found with the platform backup credentials, the artifact stays unset and a Normal `ArtifactNotMeasured` Event on the BackupJob says why. An empty snapshot raises a Warning `EmptySnapshot` Event. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Redis backups now include artifact location and size when they can be measured. * Backups are still created when artifact measurement fails, with status reporting that the artifact was not measured. * **Bug Fixes** * Improved handling of existing backups and temporary read-back failures to support reliable reconciliation. * Improved compatibility with S3 endpoints using custom certificate authorities or configured TLS verification settings. * Zero-size snapshots are now reported with a warning. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c26d2773a5 |
test(clickhouse): name what really leaves the round-trip's Secret behind
The projected Secret the preconditions delete is left by whichever BackupJob ran in tenant-root before this Test, from this suite or another: Chainsaw does not fix the order of the Tests in a suite, so no particular one can be named. The restore budget is also not spent on a credentials hold: by then the stamp is older than the backup's own two-minute wait. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
fb5ac1826a |
test(observability-proxy): prepare seed stubs in both unit runners
The POSIX runner does not invoke Bats setup hooks. Initialize the
capture stub from the test body so the compatibility lane stays offline.
Fixes:
|
||
|
|
66acdb80e1 |
ci(e2e): skip final cleanup on disposable PR runners (#5260)
## What this PR does Remove the final sandbox teardown from the two PR E2E workflows. The runners are documented as disposable VMs, so destroying their local containers and zpools after testing delays the result without protecting a later run. Cleanup between preparation attempts and the Makefile targets for reused hosts stay unchanged. The job limit stays at 215 minutes. Suite-length budgeting, including nightly and tag runs, remains separate in #5261. This change relies on the existing disposable-runner contract; it does not change runner provisioning. ### Screenshots Not applicable. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **CI Workflows** * Removed automatic sandbox and cluster teardown after end-to-end report uploads. * Added checks that cluster cleanup runs before environment preparation is retried. These changes affect automated test workflows and do not alter user-facing product features. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
f5114ae04c |
fix(seaweedfs): keep the metadata database storage class a string
db.storageClass went unquoted into the values of the database release and again into its CNPG Cluster. A class name made only of digits, which Kubernetes allows, became a number that the Cluster rejects, and a line break in the value could add keys to the database release values. It is quoted at both steps. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
e6eb3345c6 |
fix(seaweedfs): escape pool and zone storage classes in the release values
Pool and zone storage classes were wrapped in quotes by hand, which does not escape a quote or a backslash in the value: a quote closes the string early and the data-plane release values stop parsing, with a YAML error that does not name the field. They are now quoted like the volume and admin classes. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
f83db514b0 |
fix(seaweedfs): quote every storage class the upstream claims carry
The master, filer, volume index, log and all-in-one claims of the upstream chart write storageClassName unquoted, like the data claims did, so a class name made only of digits renders as a number and the claim is rejected. None of them is set by Cozystack today, but they are values of the same chart that a later change can start handing on, and the patch already exists for the data claims. They follow the same rule: quoted, and left out when no class is named so that the cluster default applies. The deprecated filer.enablePVC claim is left alone: it renders only with filer.data unset, and the template fails earlier on an unset filer.data, so it cannot be reached. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
c6cba19946 |
feat(release): keep a line's backport record and reports across batch pull requests (#5264)
## What this PR does This PR keeps backport batch record of a line across batch pull requests. Before it lived only in body of open batch PR, so once that PR merged or closed it was gone, and a conflict deferred in merged batch showed up as missing in `check` and on cut. Three `check --cut` results also did not match the docs, all three are fixed here. - `check --cut` fails when workflow original is the only thing left waiting. Before it passed as soon as pick next to it got `backport/skip` or landed by hand. - `run`, `check`, `fold` and `notify` read line record from newest trusted batch PR in any state (opened by bot or by someone with write or admin access). If newest record is in doubt (author no longer trusted, bot batch without block, unreadable block, closed record newer than open batch) command stops and doesn't fall back to older one. - Record is written only while its PR is open. Merged and closed batch PRs are never edited or commented, so run that picks nothing with no open batch writes nothing and says so. - Every run makes report: each original merged to `main` since line branched in exactly one category, what the run read (heads, policy, tool commit, Actions run, flags) and each drop with its reason and merge commit. Latest report sits in body next to record and each one also goes to immutable comment. Report ID is hash of classification without PR number, so identical runs post one comment. `backport-batch report --pr N [--id ID]` reads it back, merged batches too, it needs `main` in local clone to check drops. - `run --defer --push` of last pick used to refuse with "close batch pull request instead". Now it writes report and record with reason while PR is open, then closes PR and deletes branch, both only while head is still the same. If close stops halfway next `run --push` finishes it, and it never deletes branch that moved or that new batch uses. - `run --drop N --reason R` keeps original off one line, `--undrop` withdraws it. Drop is stored in line record bound to merge commit and carried to next batches. Original the line already carries can't be dropped, and `backport/include` doesn't lift a drop. - Output names policy rules instead of internal numbers (no more "rule 101"), `--push` prints PR URL. Reasons and file names from record are accepted only in the clean one-line form a run writes them, and everything printed is escaped. Record version 2 reads unversioned blocks that open batches carry today. I ran new tool next to current one without `--push` on today's history for release-1.7 and release-1.6, every original got same category and `check` output is the same. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(release): backport batch keeps line record and report of every run across batch pull requests, check --cut fails on workflow change left waiting alone, deferred conflict stays deferred after its batch merges, --defer of last pick closes the batch, and new run --drop keeps a fix off one line. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Maintainers can permanently exclude a change from a release line with `--drop`, or undo that decision with `--undrop`. * Added a `report` command to view validated batch reports in JSON or a readable summary. * Deferring every remaining change can close an emptied batch. A later `run --push` can complete an interrupted close. * Batch reports now show change classifications, including deferred and dropped items. * **Bug Fixes** * Checks more clearly distinguish missing, deferred, skipped, and carried changes, with cut checks enforcing outstanding workflow requirements. * Batch operations refuse edited or untrusted records and avoid writing to closed pull requests. * **Documentation** * Updated command and release-process guidance for drops, reports, cut checks, and batch closure. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
e7b5a5a1a9 |
fix(seaweedfs): keep storage class names strings in the volume claims
The upstream admin and volume StatefulSets write storageClassName unquoted, so a class name made only of digits, which Kubernetes allows and the parent chart now hands on as a string, turned back into a number there and the claim was rejected. The field is quoted, and left out when no class is named: quoting an empty value would set an empty storageClassName, which disables dynamic provisioning instead of using the cluster default. Upstream 4.52 and master still write both fields unquoted, so this is a chart patch, applied with --forward so make update stops once upstream carries an equivalent change. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
7c9aa2fe10 |
fix(seaweedfs): hand storage class names to the data plane as strings
The volume and admin storage classes were written unquoted into the values of the data-plane release. A class name made only of digits, which Kubernetes allows, arrived there as a number, and a value with a line break could add keys next to the field, past the schema that is the tenant's input gate. Both are now quoted. lifecycle.storageClass, new in this release, also carries the StorageClass name rules in its schema, so a value that cannot name a class is refused when the application is written instead of leaving the admin claim pending. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
5a2582e21c |
test(observability-proxy): preserve JSON seed request bodies
Send JSON seeds with their content type so VictoriaLogs does not consume
log records while parsing form parameters before ingestion.
Fixes:
|
||
|
|
9cd51cac2e |
ci(e2e): skip final cleanup on disposable PR runners
Destroying runner-local state after a completed suite can push the job
past its deadline even though the runner is discarded. Keep cleanup
between preparation attempts and for reused hosts instead.
Fixes:
|