mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-08 21:14:56 +09:00
4d7f9e485ab46e401bde15e7f3c09adebbbfc657
7040
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
4d7f9e485a |
docs(ci): say why the overlay's ancestry tie goes to main (#4469)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does This fixes the comment on the overlay's ancestry tie, which described the wrong reason for the tie. No behaviour changes. The comment said a line fresh off main is the same as main. That holds only for the graph. `cozystack-packages:main` is built from main's tip, so once main moves past the cut, a stack on such a line gets newer images than the line. The comment now says so. It also says why the tie still goes to main: a branch forked from main before the cut ties the same way, and that case is far more common. This follows up on a review note in #4460 and is stacked on it. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note docs(ci): explain why the base-branch overlay sends an ancestry tie to main. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified how artifact selection works for stacked pull requests on release lines without line-specific commits, including why `main` is preferred in a tie and how that may affect image freshness. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a66907f535 |
fix(backups): share the walkthrough wait helpers and fix the S3 preflight cleanup (#4635)
## What this PR does This fixes the S3 access preflight and a few backup walkthroughs, and moves the walkthrough wait helpers into one shared file. The access preflight now keeps its evidence and cleans up after itself. When `mc alias set` failed, it dropped the port-forward log, and now it prints it like the other exits do. The exit trap used to kill the port-forward before `mc rm`, so the delete always hit a closed port and the warning gave no reason. The delete now runs first, and the warning carries mc's own error, or says the delete timed out when mc was killed before writing one. A PUT that timed out (rc 124) may have landed, so it counts as uploaded. A PUT the server refused does not, so access that never went live gets no delete and no warning. The wait helpers were copied into every walkthrough and had drifted. `wait_for_field`, `wait_hr_ready` and `wait_deleted` had 12, 8 and 2 copies with 5, 3 and 2 different bodies. They now live in `examples/backups/_lib/wait-helpers.sh`, and each `00-helpers.sh` sources it. Call sites and wait budgets are unchanged. Where copies differed, the shared body keeps every diagnostic any copy had. What changes per walkthrough: - `wait_for_field` in every walkthrough but redis now shows kubectl's stderr, so a Forbidden or transport error is visible instead of reading as an empty field. `--ignore-not-found` keeps a resource that does not exist yet from printing an error on every poll. Redis already kept stderr. - `wait_for_field` in foundationdb, nats and vmi gains the optional 7th terminal-value argument. None of their callers pass it, so nothing changes at runtime. - vmi loses its progress dots. - `wait_hr_ready` in redis and kafka-metadata now prints every condition and the release history on Stalled or timeout. It also says whether the release never appeared, was deleted, or could not be looked up. - The kafka `wait_deleted` timeout message reads "it is still present" instead of "still present". `hack/backup-examples-shared-helpers.bats` goes red if a walkthrough defines one of these helpers again or stops providing them, and it pins both sides of the `wait_for_field` read: kubectl's reason for a failed read is shown, a missing resource prints nothing. The byte-identity pin between copies in `hack/mariadb-first-boot-waits.bats` is gone, and its errexit test runs through the mariadb walkthrough's helpers. The kafka topic-data demo had the empty-read race from #4400. `kafka_run` read topic state through a throwaway `kubectl run -i` Pod. It now execs into a long-lived CLI Pod that keeps the restricted security context, and `cleanup.sh` removes it by label. The Pod name comes from its own `KAFKA_TOPIC_CLI_POD`, so setting the kafka-metadata demo's `KAFKA_CLI_POD` does not move it. #4400 itself was already fixed on main for nats, kafka-metadata and rabbitmq. The nats and kafka-metadata demos could exec into a Pod they had just refused. They read CLI output inside `$(...)`, where bash does not apply errexit, so the ownership refusal was ignored. A failed read of the Pod's phase was treated as no Pod, and the demo went on to delete and replace it. Every setup step now returns on failure. Their "exec fails with kubectl's own error" tests could never fail: the runner's trace wrote the expected message into the same stderr the test grepped. The stub now prints the message itself. Each fix has a test that fails on main. For each fix I also reverted it and checked that only its own test goes red. One gap stays open outside this PR. `hack/select-e2e.sh` maps `examples/backups/<app>/` paths by app name, so a later PR that changes only `examples/backups/_lib/wait-helpers.sh` would select no suite. This PR is fine because it touches every walkthrough. The rule belongs to the select-e2e owners and is tracked in #4626. While reviewing I also found that the kafka README puts steps that create and exec into Pods under the tenant persona, and the tenant role can do neither. That is #4625, left for its own change. Left out on purpose, because they are outside what this diff changes: - The kafka topic demo and the kafka-metadata demo each carry a CLI Pod helper, and the kafka-metadata one runs without the restricted security context. - The walkthrough READMEs do not say that a directory now needs `../_lib` next to it. - The errexit test in `hack/mariadb-first-boot-waits.bats` loops over one file. I kept it so the rebase against #4471 stays mechanical. Closes #3757, closes #4243, closes #4244 ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backups): the backup walkthroughs share one set of wait helpers that keep kubectl errors visible, the kafka topic-data demo no longer reads empty replies from a throwaway Pod, the nats and kafka-metadata demos no longer exec into a CLI Pod they refused, and the e2e S3 access preflight says why client setup or cleanup failed and deletes its test object reliably. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Kafka topic backup walkthroughs now reuse a named CLI Pod, with restricted security settings and checks to prevent using a Pod owned by another demo. * Backup walkthroughs can wait for resources to reach a desired state, Helm releases to become ready, or resources to be deleted. * **Bug Fixes** * Kafka and NATS walkthroughs now stop and report errors when CLI Pod lookups, creation, deletion, or command execution fail. * S3 preflight cleanup now attempts removal after a timed-out upload and reports cleanup failures, including timeouts. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a9dc106a81 |
fix(e2e): bound post-install-prep calls and make its failures name their cause (#4632)
## What this PR does The post-install-prep step of the e2e install could hang without failing, and several of its failure paths gave up without saying why. The script runs in the background of the install test, which waits on it, so a kubectl call that never returns keeps the install open until the job's own timeout. This PR fixes both problems in `hack/e2e-post-install-prep.sh`. - Every call that does work now has a wall-clock bound: the pool listing and creates, both applies, the CDI patch, the reads in the LINSTOR node loop, and the diagnostic reads on every failure path. Plain requests also get a smaller `--request-timeout`, so kubectl can say why a request failed before the outer timeout kills it. When a bound fires, the log names the call. - `kubectl wait --timeout` turned out to be no bound at all for the GET it runs first, so each wait attempt got an outer timeout too. If that timeout cuts an attempt, the timeout line says so. - The wait loop keeps kubectl's stderr now. An expired credential or an RBAC denial used to look the same as "not created yet" and ended in a timeout on the object. Each error is printed once when it changes, and the timeout line repeats the last one. - When the linstor wait times out, the script lists every HelmRelease first. The linstor release can be just waiting for a release further up the dependency chain, and the Ready messages name which one. - The MetalLB CRD wait and the CDI StorageProfile wait used to exit silently. Now they print a timeout line and one last read of the missing object with its error. The MetalLB wait also prints the metallb HelmRelease. - A failed or killed storage pool listing stops the script before any pool is created. Before, it read as "no pools exist", and the creates that followed failed with an unrelated error. All of this is tested in `hack/post-install-prep.bats` with the existing stub PATH and virtual clock. One test checks that every kubectl call on the success path and on each timeout path runs under a bound. The literal grep in `hack/linstor-storage-mode_test.bats` is replaced with a test of the behaviour. The rest of #3680 lives outside this script and moves to #4628: the empty `{"error":{}}` in helm-controller's Helm log, previous-container logs on passing runs, and node and pod resource state during install. Closes #3708, closes #3707, closes #3713, closes #3714, closes #3680. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(e2e): bound every call in the post-install-prep step and make its failures name their cause, so a hung call fails the install instead of holding it until the job timeout ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Reliability** * Cluster operations now have bounded wait times, with clearer diagnostics when resources or endpoints are delayed or unavailable. * Storage setup stops when pool listings fail instead of masking the error. * **Tests** * Expanded coverage for timeout handling, error reporting, and failures during storage and resource setup. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d8ec8f0af1 |
fix(ci): run every bats unit file after a failure and cover build-*.sh in the cross-compile check (#4633)
## What this PR does Two fixes to the tooling of the unit lane. A local run of the bats unit tests stopped at the first red file and never ran the files after it, while the output looked like a finished suite. Each bats file is its own make target, and without `--keep-going` make stops scheduling targets after the first failure. CI passes `-k`, so it was not affected. The target now runs the files in a `--keep-going` sub-make. That sub-make still exits non-zero if any file failed, and it shares the caller's `-j` slots and `--output-sync`. The new `hack/bats-unit-keep-going.bats` drives the real Makefile against two fixture files. It checks that both run and the target fails when the first one fails, that the target exits 0 when both pass, and that the sub-make still finds the root Makefile when `MAKEFILES` names another file. I split the exact-line check on this target in `hack/bats-runner-coverage.bats` in two to match the new recipe. A workflow comment and a test name that still called each file a prerequisite now call it a target. The cross-compile check never looked at the multus-cni builder. It treated only a script named exactly `*build.sh` as a compile step, and multus runs upstream `./hack/build-go.sh`. The pattern now allows a suffix after `build` but no `/`, and each case has a fixture test. Among the Dockerfiles the check scans under `packages/`, this matches exactly one new line, the multus stage, and that stage passes. Closes #3692 for the make-level truncation. The in-file half, where `hack/cozytest.sh` ends a file at its first failing test, is tracked in #4620. Closes #4500 Things I left out: - The multus stage passes because a `curl ... TARGETARCH` step sits in the same stage. The check does not see that `build-go.sh` reads `TARGETPLATFORM`. The test header already documents this per-stage limit. - The sub-make reads only the root Makefile, so an extra `make -f other.mk` given to the parent no longer reaches the bats targets. Command-line variables and `MAKEFILES` still do. Nothing in the repo calls it that way. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(ci): `make bats-unit-tests` runs every bats file after a failing one and still fails at the end, and the image cross-compile check now covers build scripts such as `build-go.sh`. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Unit tests now continue running across individual test-file failures while still reporting an overall failure if any file fails. * Build-stage detection now recognizes build scripts with names such as `build-go.sh`. * **Tests** * Added coverage for unit-test results and build-script detection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
cdc4b88a90 |
fix(ci): close e2e suite selection gaps in select-e2e and select-install (#4634)
## What this PR does The e2e test selector picked the wrong Chainsaw suites for some diffs. In one case it lost an escalation, in another it ran the full suite where a subset was enough, and in two more it missed a suite or ran nothing. The fixes are in `hack/select-e2e.sh`, with the `.yml` part also in `hack/select-install.sh`. - A changed file in a directory under `hack/e2e-chainsaw/` that holds no suite escalated only when it was the whole diff. Next to any path that selected a suite, the escalation was dropped with nothing on stderr. The directory name is now checked against the suite list right where the path is classified, like the `hack/e2e-apps/` rule already did. The empty-selection backstop can't be reached anymore. I kept it as a guard, and the doc says its line has no test. - A seaweedfs change ran the full suite, because `cozystack.seaweedfs-application` mapped to a `seaweedfs` suite that does not exist. Now it selects the suites that use tenant-root's object storage: bucket, the backup round-trips that write through the in-cluster `seaweedfs-s3` endpoint, and harbor, which keeps its registry in a BucketClaim served by the seaweedfs COSI driver. The issue says this mapping would break the round-trip test. It doesn't, since that test only walks suite to source to suite, and `vm-disk-application` already maps to `vminstance` the same way. The test takes the expected set from the suites' own yaml. - A suite written as `chainsaw-test.yml` was invisible to both selectors. Chainsaw v0.2.15 loads that name when there is no `chainsaw-test.yaml` (`pkg/discovery/load.go`), so such a suite runs but never gets selected. Both scripts list it now. `select-install.sh --validate` also passed when `find` failed, and now it fails. - An edit under `examples/backups/_lib/` was read as an app named `_lib` and selected nothing. It escalates now, same as `hack/e2e-chainsaw/_lib/`. - The selector comments quoted bats file counts and escalation-branch counts. All of them were stale, so I removed them. The doc half of #4120 was already fixed on main. Each fix has a test that fails on main. I also mutated each fix: its own test goes red and no other test does. Three related gaps outside these lines are tracked in #4630: the `packages/core/testing/Makefile` comment that names only `chainsaw-test.yaml`, suites built from numbered step files, and a file placed directly in `examples/backups/`. Two things worth knowing before merge: - Any file except the `.disabled` one in a parked suite's directory, like `hack/e2e-chainsaw/kubernetes-proxmox/tenant.yaml`, now always runs the full suite. That is the safe direction. If it gets expensive, those directories could be made inert. - The `examples/backups/_lib/` rule and its doc line point at a directory another PR adds. If this PR merges first, the rule just has nothing to match yet. Closes #3816, closes #3710, closes #4283, closes #4120, closes #4205, closes #4626. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(ci): e2e test selection no longer drops an escalation for a non-suite path under hack/e2e-chainsaw/, selects the suites that use seaweedfs instead of the full run, sees suites written as chainsaw-test.yml, and escalates edits to examples/backups/_lib/. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * End-to-end test selection now recognizes suites configured with either supported YAML filename and reports discovery failures instead of silently proceeding. * Changes to shared backup walkthrough helpers and unrecognized test-suite directories now trigger broader test coverage. * Storage-related changes in SeaweedFS now select all relevant test suites. * **Documentation** * Updated testing guidance to describe suite discovery, file-change matching, and when the full test suite is selected. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
scooby87
|
031ded71ee |
feat(seaweedfs): alert before an instance stops taking writes (#4573)
## What this PR does A SeaweedFS instance that runs out of room stops taking writes without anything paging. A volume server below its `minFreeSpacePercent` reserve turns every volume it holds read-only; before that, it can run out of slots, and a bucket whose volumes are all full then cannot get a new one, even with half the disk free. Backups archiving into the instance fail in turn, and a Postgres cluster doing continuous WAL archiving keeps the segments on its own volume until that fills too. The generic `KubePersistentVolumeFillingUp` alert does not catch this: it pages below 3% free, while a volume server stalls at its 5% reserve or, when slots run out, with the disk still half empty; and its early warning needs the disk to keep filling, which stops once writes fail. This PR adds per-instance rules to the SeaweedFS system chart, on signals the servers already publish and the platform already scrapes, rendered under the same `monitoring.enabled` gate as the chart's ServiceMonitors: - `SeaweedFSVolumeServerDiskLow` (warning): a volume server has less than 10% of its disk left above the reserve. - `SeaweedFSVolumeServerFull` (warning): a volume server cannot host another volume, because it is at the reserve or has no slot left. - `SeaweedFSWritesFailing` (critical): the master has failed to place a write in every 5-minute window for 15 minutes. Writes may still land in between; the usual cause is a bucket with no writable volume left that the master cannot grow. A new bucket's first writes fail a few picks while its volumes grow, which the 15-minute hold does not reach unless new buckets keep arriving. Every expression is pinned to the release namespace, because the root vmalert evaluates each instance's rules against every namespace's metrics. It also adds `docs/operations/seaweedfs-capacity-for-backups.md`: how to size an instance for continuous WAL archiving, what the rules mean, and how to grow an instance. The expressions were validated live rather than by unit tests, since the repository has no rule-evaluation tooling: on a test cluster, a standalone SeaweedFS 4.45 on a 1 GiB volume was scraped by the platform's vmagent and the rules rendered from this chart were evaluated by the root vmalert. All three loaded healthy and each reached `firing`: `SeaweedFSVolumeServerDiskLow` with 5.7% of the disk left above the reserve, `SeaweedFSVolumeServerFull` both at the reserve and on slot exhaustion, which stopped one bucket at 50% disk use, and `SeaweedFSWritesFailing` under continuous failing writes, resolving once they stopped. A new bucket's first successful writes on a free disk did fail a few picks, which is why that rule holds for 15 minutes. On the healthy platform instance all three stay quiet. helm-unittest pins what a render can break: the namespace selector of each rule, the shared label set of the two arms of the full-server rule, the gate, the critical rule's hold, and that alert templating reaches vmalert unrendered; each assertion was checked to fail when its line is mutated. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches a system chart template, its test and a document under `docs/operations/`; no app values, platform values, ApplicationDefinition, tooling or `hack/` path from the trigger map. ### Release note ```release-note feat(seaweedfs): alert when a SeaweedFS volume server nears its free-space reserve, cannot host another volume, or keeps failing to place writes, and document how to size an instance for continuous WAL archiving. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added namespace-scoped SeaweedFS monitoring alerts for low or exhausted disk capacity, unavailable volume slots, and write-placement failures. Capacity alerts use a 10% disk threshold, and alerts trigger after 15 minutes; write failures are marked critical. - **Documentation** - Added guidance on estimating SeaweedFS storage needs for Postgres backups, interpreting capacity alerts, and expanding storage when supported by the StorageClass. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
97fc4359ad |
feat(scheduler): set scheduling-class label on mutated pods (#2621)
## What this PR does
Extends the lineage-controller-webhook so that, on every Pod it mutates
with a `SchedulingClass`, it also sets a label of the same name (in
addition to the existing annotation).
Today, when a SchedulingClass is in effect on a tenant, the webhook
injects:
- `spec.schedulerName: cozystack-scheduler`
- annotation `scheduler.cozystack.io/scheduling-class: <name>`
This is enough for the cozystack-scheduler to look the class up and
apply its `nodeAffinity`, `nodeSelector`, and topology rules. However,
it is **not** enough to write a `SchedulingClass` whose
`spec.podAffinity` rules `labelSelector` targets pods across multiple
applications, because `labelSelector` matches labels and not
annotations.
This PR adds the label, leaving the annotation untouched for backward
compatibility:
```go
podLabels := obj.GetLabels()
if podLabels == nil {
podLabels = make(map[string]string)
}
podLabels[schedulerapi.SchedulingClassLabel] = schedulingClass
obj.SetLabels(podLabels)
```
The constants `SchedulingClassLabel` and `SchedulingClassAnnotation`
exported by `github.com/cozystack/cozystack-scheduler/pkg/apis/v1alpha1`
already share the same value
(`scheduler.cozystack.io/scheduling-class`), so no new key is
introduced.
**Why this is useful**: a single SchedulingClass can now express
"co-locate every pod that uses me at the same topology" with a rule
like:
```yaml
apiVersion: cozystack.io/v1alpha1
kind: SchedulingClass
metadata:
name: co-region-prod
spec:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
scheduler.cozystack.io/scheduling-class: co-region-prod
topologyKey: topology.kubernetes.io/region
```
When several applications share this class, the second-and-later pods
will be constrained to the region the first one landed on — without
hard-coding any region in the class itself.
### Tests
Adds a `webhook_test.go` (none existed in the package). Five unit tests,
using `clientfake.NewClientBuilder` +
`dynamicfake.NewSimpleDynamicClientWithCustomListKinds`, cover:
- Label, annotation, and `schedulerName` are all set when a class
exists.
- Existing pod labels are preserved (no clobbering).
- Non-Pod objects are a no-op.
- Pods in a namespace without a scheduling-class label are a no-op.
- Pods whose referenced `SchedulingClass` CR does not exist are a no-op
(the existing defensive behavior, preserved).
### Release note
```release-note
feat(scheduler): the lineage-controller-webhook now sets a scheduler.cozystack.io/scheduling-class label on mutated pods (in addition to the existing annotation), enabling SchedulingClass authors to write podAffinity rules that match pods across applications.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Pods with a valid scheduling class now receive its label alongside the
scheduler configuration and annotation.
- Existing Pod labels are preserved when the scheduling-class label is
added.
- **Bug Fixes**
- Scheduling-class settings are applied only when the namespace
references a scheduling class that exists. Non-Pods and namespaces
without a scheduling-class label are left unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
mattia-eleuteri
|
250f89dc72 |
feat(scheduler): set scheduling-class label on mutated pods
The lineage-controller-webhook now sets a scheduler.cozystack.io/scheduling-class label on mutated pods (in addition to the existing annotation of the same name). This unlocks SchedulingClass authors writing podAffinity rules whose labelSelector targets pods across different applications that share the same class. A natural pattern becomes: define a class whose name is the affinity-group identifier, attach it to several applications, and have the scheduler co-locate them at the chosen topology key (region, zone, hostname). Behavior preserved when the SchedulingClass CR is missing: pods are left untouched (no scheduler injection, no label) so they do not stay Pending on a non-existent scheduler. Tests cover label/annotation/schedulerName injection, existing label preservation, non-Pod no-op, no-class no-op, and missing-CR no-op. Signed-off-by: mattia-eleuteri <mattia@hidora.io> |
||
|
Aleksei Sviridkin
|
873d9f4c8e |
fix(backups): stop the nats and kafka-metadata demos using a Pod they refused
Both demos read their CLI output through command substitutions, where bash does not apply errexit. When the ownership check refused a same-named Pod another party owns, the refusal was ignored and the exec went into that very Pod. A failed read of the Pod's phase or owner was likewise read as "no Pod", so the helper went on to delete and replace a Pod whose owner it had not seen. Every step of the Pod setup now returns on failure, and the phase and owner reads tell a missing Pod (--ignore-not-found) from a failed read. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a3caa62e47 |
test(backups): make the nats and kafka-metadata exec-error cases able to fail
Both cases passed the expected kubectl error to the stub through an environment variable named on the command that runs the helper. The unit runner traces that command, with the value expanded, into the very stderr the case then greps, so the case stayed green with the helper's stderr sent to /dev/null. The stub now prints the message itself and the variable is only a flag. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ead7048b35 |
refactor(backups): define the walkthrough wait helpers once
Every backup walkthrough carried its own wait_for_field, and most their own wait_hr_ready and wait_deleted: twelve, eight and two copies, which had already drifted into five, three and two bodies. A fix made in one copy did not reach the others, and a diagnostics change carried by hand between two of them once went into the wrong function and was caught only by reading the diff. The helpers now live in examples/backups/_lib/wait-helpers.sh, sourced by each walkthrough's 00-helpers.sh. Where copies differed, the shared body keeps the diagnostics any of them had: wait_for_field leaves kubectl's stderr attached, and wait_hr_ready prints the conditions and history dump on Stalled or timeout. Call sites and budgets are unchanged. A new unit test fails when any walkthrough defines one of these helpers again, which replaces the byte-identity pin between copies. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5318b5e507 |
fix(backups): exec into a CLI Pod in the kafka topic-data demo
kafka_run seeded and read topics through a throwaway `kubectl run -i` Pod per call. That Pod's stdout returns over an attach that carries only what the container writes after the attach registers, so a CLI that finishes first reads as an empty reply with exit 0 -- the empty-read defect already fixed in the redis, nats and kafka-metadata demos. The topic count, dump and replication-factor checks all read that reply. Run one long-lived CLI Pod, keeping the restricted security context, and exec into it. It carries its own name and owner label so it cannot collide with the kafka-metadata demo's Pod in a shared namespace, and cleanup.sh removes it by that label. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
735486fc36 |
fix(e2e): say when a post-install-prep wait attempt hit its bound
When the outer timeout cut a `kubectl wait` attempt, nothing was printed: `timeout` is silent and kubectl dies on the signal. The timeout line then read "last kubectl error: none", which says the client had nothing to report rather than that it was killed mid-request. The attempt's exit status now names that case. The errors the loop prints also carry the script's prefix. The first NotFound shows up on every green run, since the release appears about a minute in, and without the prefix it reads like a failure when the log is scanned after one. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5b4380dd3b |
fix(e2e): delete the failed S3 preflight object through a live tunnel
The exit trap stopped the port-forward before running mc rm, so the delete went to a closed port and failed every time, leaving the object behind and printing a warning that said nothing about why. The delete now runs first and the warning carries mc's own error, or says the delete timed out when mc was killed before it could write one. A PUT that timed out is now also treated as uploaded: it can land and still outlive its time bound, and the trap never tried to remove such an object. A PUT the server refused is not, so the common case of access that never went live does not end in a delete doomed to the same refusal. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b6cd6ab682 |
fix(e2e): print the port-forward log when the S3 preflight client fails
mc alias set reaches the endpoint through the port-forward, so a tunnel that came up and then broke is a likely cause of that failure, and its log is the only record of it. The readiness and retry-timeout exits already print it; this one threw it away. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
71a943277f |
build(make): keep running bats unit files after one fails
Each bats file is its own make target, so a plain `make bats-unit-tests` stopped scheduling files at the first red one and never ran the rest, while the output looked like a finished suite. CI passes -k and was not affected; a local run was. The target now runs the files in a --keep-going sub-make, which still exits non-zero when any file fails and shares the caller's job slots and output sync. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f023e2fa38 |
test(build): count build-*.sh scripts as compiling in the cross-compile check
The compile pattern only matched a script named exactly *build.sh, so the multus-cni builder, which compiles through upstream ./hack/build-go.sh, was never classified as compiling and the check skipped the stage while appearing to cover it. Allow a suffix after "build" in the script name, but not a directory separator, so a path through a build/ directory is not mistaken for a build script. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
0592d0e324 |
chore(ci): add area/openbao to the label set and the scope mapping (#4182)
## What this PR does Every openbao PR lands on `area/uncategorized`. The labeler has no scope entry for it, and there would be no label to apply if it did, so both #4168 and #4177 are sitting on the fallback right now. `labels.yml` puts the bar for a new area at three open issues on the topic, and openbao is at three: #2787, #2483 and #2793. The shape follows what is already there for a single shipped component, next to `area/kamaji`, `area/keycloak` and `area/cilium`. The mapping takes the plain `openbao` scope and `openbao-central` alongside it, because the platform instance lives in its own package and carries its own scope. Checked that every `area/*` and `kind/*` the labeler references is declared in `labels.yml`: 27 referenced, none missing, before and after this change. zizmor and the pre-commit hooks pass, and the embedded script still parses under github-script semantics. ### Screenshots No UI change. ### Downstream repositories Walked the trigger map against the diff. Nothing matches: the change is two entries under `.github/`, and no trigger in the map is keyed on the label set or the labeler. No package added, renamed or removed, no platform component or variant, no `values.schema.json`, no `ApplicationDefinition`, no `hack/` layout change, no namespace rename. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added an `area/openbao` label for tracking issues and pull requests related to OpenBAO. - Updated automatic pull request labeling to apply this label to changes scoped to OpenBAO and OpenBAO Central. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
c96dcaa11a |
fix(e2e): bound the lookup behind each kubectl wait in post-install-prep
`kubectl wait --timeout` looked like a bound on the wait loop's calls, but it covers only the watch. The command first resolves the object with a plain GET that has no deadline, since no --request-timeout is set, so an apiserver that accepts the connection and never answers holds the wait loop, the most frequently repeated call in the script, just like the calls that were unbounded outright. Each attempt now runs under an outer timeout set a little above the attempt's own --timeout, so the wait still owns its normal expiry. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
08011cef09 |
fix(ci): escalate an edit to the shared backup walkthrough helpers
examples/backups/<app>/ selects that app's suite, and a directory whose name is no suite selects nothing. A helpers directory shared by every walkthrough, examples/backups/_lib/, falls into the second case: an edit to it alone produces an empty selection, which both lanes read as "skip Chainsaw", so a change reaching every backup round-trip would run none. It now escalates to the full suite like hack/e2e-chainsaw/_lib/. The set of suites that run a walkthrough is spread across the suites' own scripts, and a derived subset that misses one would fail open. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
603ff2871e |
docs(hack): drop file and branch counts from the selector comments
The comments explaining the unit/e2e bats split quoted how many files each side held, and the escalation helper how many branches call it. Every figure had gone stale: nothing checks them, and they move each time a bats file or an escalation is added. The Makefile expression quoted beside the bats claim is what defines the set, so the explanation stands without the number. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a6ec30d56f |
fix(ci): discover Chainsaw suites written as chainsaw-test.yml
Chainsaw loads chainsaw-test.yml when a directory has no chainsaw-test.yaml, but select-e2e and select-install enumerated the .yaml name only. A suite spelled that way ran whenever the whole directory was handed to chainsaw, yet no escalation listed it and no edit to it could select it, so it silently dropped out of every narrowed run. select-install also sent find's errors to /dev/null inside a pipeline that reported sort's status, so a failed listing validated no suite at all and passed. It is now captured and fails validation. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d777f2a0b3 |
fix(ci): select the suites that use seaweedfs instead of all of them
cozystack.seaweedfs-application mapped to a seaweedfs suite that does not exist, so every seaweedfs change reached nothing runnable and paid for the whole Chainsaw run. The suites that can regress from it are bucket, the backup round-trips that write to the tenant-root instance through its in-cluster S3 endpoint, and harbor, whose registry storage is a BucketClaim served by the seaweedfs COSI driver with no graph edge between the two. bucket alone would miss the CA and service-name paths the round-trips depend on. The test derives the expected set from the suites' own yaml, so a new suite that names the endpoint or a BucketClaim there fails it until it is mapped. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e070bc7d1d |
fix(ci): escalate a per-suite Chainsaw path that names no suite
A path under hack/e2e-chainsaw/<dir>/ put <dir> into the selection without checking that it is a suite, leaving the final intersection to drop it. Alone the path escalated through the empty-selection backstop, but beside any path that contributed a suite the escalation vanished and the run narrowed to that suite with nothing on stderr. Shared material next to _lib/ or a suite nested deeper than discovery looks then went untested in exactly the pull requests that also touch an app. The directory name is now membership-tested where the path is classified, as the hack/e2e-apps/ arm already does, so the decision is a property of the path and names it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
7e0bd9c403 |
fix(e2e): stop post-install-prep on a failed or killed pool listing
The storage pool listing was piped straight into awk, so its exit status was lost: a listing that failed, or one killed by its time bound, read as "no pools exist yet". The script then sent a create for every node, and on the nodes whose pool did exist the create failed with a reason unrelated to the actual failure. The listing now runs on its own, so a failure stops the script before any pool is created. A bound that fires leaves no text of its own: `timeout` prints nothing and kubectl dies on the signal before it can. The bounded calls therefore go through a wrapper that names the call it killed, so the log does not end on the step's announcement line. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ff2a677fc0 |
fix(e2e): say why the MetalLB CRD and CDI StorageProfile waits failed
Both waits ran as `timeout N sh -ec 'until kubectl get ...'` with the read's output discarded. On elapse `timeout` exits 124 and `set -e` ends the script, so the last line in the log is the one announcing the wait: CRDs that are late and an operator that never got far enough to install them read the same. A shared deadline loop now polls quietly and, on the way out, reads the objects once more with stderr kept, so the log names which object is missing. The MetalLB wait also lists the metallb HelmRelease, whose state separates those two cases. The loop runs on the script's own clock, which lets the unit suite drive it with the virtual one. The unit suite's runaway guard in the sleep stub now kills its caller as well, because a failing exit status does not stop a loop that runs where set -e is off, and a guard that cannot stop the loop turns a broken deadline into a hung test. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b9a3fd0e8b |
fix(e2e): name the blocking release when post-install-prep times out
When the linstor HelmRelease never goes Ready, the script reported a timeout on linstor and dumped the cozy-linstor pods and events. In two failing install runs the linstor release only sat behind a dependency chain (kubeovn -> piraeus-operator-crds -> piraeus-operator -> linstor), so the evidence it printed described the wrong release and said nothing about the one that was stuck. The timeout now lists every HelmRelease first. Each Ready message names the dependency it waits on, so the post-install-prep log names the release at the head of the chain. Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM |
||
|
Aleksei Sviridkin
|
3378711fb9 |
fix(e2e): keep kubectl stderr in the post-install-prep wait loop
The loop discarded stderr to stay quiet while an object is not created yet, since `kubectl wait` exits non-zero at once on NotFound. An expired credential or an RBAC denial exits the same way, so the loop retried a client that could never succeed for the whole budget and then reported a timeout on the object, pointing at the cluster while the cause sat in the client. Each attempt's stderr is now kept and printed whenever it changes, and the timeout line repeats the last one. A NotFound repeated every poll still costs a single line. The text is printed rather than matched to fail fast: apiserver restarts during install produce the same connection and auth errors transiently, and a match list would turn a slow link into a false failure. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d07099f13c |
fix(e2e): bound the calls that do work in post-install-prep
The install step blocks in `wait` on this script, so a kubectl call that never returns does not fail the step: it holds it open until the job's own ceiling cancels the run, which leaves different evidence behind than a failed test. Only the waits carried a budget. The execs that create storage pools, the applies, the CDI patch, and the reads inside the LINSTOR node loop did not, and the loop's deadline is checked between iterations, on the other side of a call that hangs. The reads that explain a node-loop timeout were unbounded too, on the one path that runs only when the cluster is already unhealthy. Each call now runs under an outer timeout, and plain requests carry a client timeout strictly inside it so kubectl names the reason before the kill lands. The helpers take the bound as a prefix that stays empty when unit tests source them, since `timeout` would bypass the kubectl function those tests substitute. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
9ac67d76a2 |
test(tests): preserve Bats failure reports during fixture cleanup (#3848)
## What this PR does Test-level `EXIT` cleanup overrides the Bats TAP handler, so a failed assertion can disappear from test results. This moves cleanup in eight unit files after their assertions. Failed tests keep scratch files for inspection. Prerequisite for [the Bats runner change](https://github.com/cozystack/cozystack/pull/3849). Existing subshell traps in the OpenAPI and CAPK tests remain supported. Closes #3623 ### Validation `make unit-tests test-controllers -j4 --output-sync=target` passed. The converted files passed under Bats 1.14.0 and `dash`. A deliberately failed assertion produces `not ok` after this change, and the trap audit rejects an added test-level handler. ### Downstream repositories Test cleanup and its documentation only. No downstream consumer changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` |
||
|
Aleksei Sviridkin
|
15f57be6f9 |
feat(controller): expose bucket class on BucketClaim Workloads (#3501)
## What this PR does `WorkloadMonitorReconciler.reconcileBucketClaimForMonitor` builds a `Workload` for each `BucketClaim` but never read `spec.bucketClassName`, so nothing on the resulting `Workload` told a consumer which bucket class was in use (e.g. an encrypted class vs. a plain one). This PR reads `BucketClaim.spec.bucketClassName` and sets it as a `workloads.cozystack.io/bucket-class` label on the `Workload`, following the same `workloads.cozystack.io/*` labeling convention already used for KubeVirt instance-type/profile metadata elsewhere in this controller. A regression test (`TestReconcileBucketClaim_BucketClassLabelPropagated`) asserts the label is present with the correct value. Closes #3419 ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(controller): expose bucket class on BucketClaim Workloads via the workloads.cozystack.io/bucket-class label ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Workloads created from ready BucketClaims include the associated bucket class label when a valid value is specified. * **Bug Fixes** * Invalid or empty bucket class values are omitted from Workloads. * Stale bucket class labels are removed when the associated claim does not provide a valid value. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
3111fa7802 |
chore(ci): add area/openbao to the label set and the scope mapping
Every openbao PR lands on area/uncategorized, because the labeler has no scope entry for it and there would be no label to apply if it did. #4168 and #4177 are both sitting on the fallback right now. labels.yml sets the bar for a new area at three open issues on the topic, and openbao is at three: #2787, #2483 and #2793. It also follows the shape already used for a single shipped component, next to area/kamaji, area/keycloak and area/cilium. Maps the plain `openbao` scope and `openbao-central` alongside it, since the platform instance lives in its own package and gets its own scope. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
9d3661416b |
ci: drop unused packages:write from the pull-request workflow (#3422)
## What this PR does The `build`, `build-talos` and `finalize` jobs in `pull-requests.yaml` each requested `packages: write`, but none of them touches GHCR: the workflow's `REGISTRY` is OCIR and they authenticate with `OCIR_USER` / `OCIR_TOKEN`. The only GHCR login in the file is in `verify-release-candidate`, which has `packages: read` and is not touched here. The scope was dead. It is worth removing rather than leaving alone. GHCR pushes elsewhere in CI authenticate with `GITHUB_TOKEN` plus `packages: write` rather than with a stored secret, so on those paths the scope *is* the whole credential, so granting it on the `pull_request` lane hands every same-repo PR run a token carrying GHCR write authority that lane has no use for. Whether a given package would actually honour that write depends on its Actions access setting, which lives outside this repo, so this is least-privilege hygiene rather than a fix for a demonstrated exploit. It also brings the three jobs in line with the `permissions: contents: read` default the workflow already sets at the top level. The e2e job keeps its `packages: read`; only the write scope is removed. Verified: `build`, `build-talos` and `finalize` log in to OCIR only (`docker/login-action` with `OCIR_USER` / `OCIR_TOKEN`), the sole `ghcr.io` login is in `verify-release-candidate` with `packages: read`, unchanged. The file still parses as YAML after the change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map in `contributing.md` against the diff. It removes three permission lines from one workflow and changes nothing any downstream repository restates. The nearest entry is the `ccp` one on release-prep behaviour, which is scoped to `tags.yaml` and untouched here. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated the build, Talos build, and finalization workflows to request read-only repository access. * Removed package write permissions from these workflows. * Registry login and image publishing remain configured separately and are unchanged. No other workflow behavior or processing steps changed. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
c983b5c404 |
fix(linstor): use severity warning instead of warn in piraeus-datastore alerts (#3799)
## What this PR does
Seven of the piraeus-datastore alert rules label themselves `severity:
warn`. Alerta, which the monitoring package ships as the alert sink,
validates that value and `warn` is not on its list. Every webhook
delivery for these alerts dies with a 500.
```
Severity (warn) is not one of security, critical, major, minor, warning,
indeterminate, informational, normal, ok, cleared, debug, trace, unknown
```
Alertmanager retries seven times and drops the notification, and the
alert never shows up anywhere. The affected rules are the LINSTOR/DRBD
ones, such as storage pool at capacity, lost quorum, unintentional
diskless and stuck resync. These are exactly the alerts you least want
silently dropped.
Found this on a v1.4.6 cluster where `linstorStoragePoolAtCapacity` was
genuinely firing. The cluster had 153 dropped notifications in 40
minutes and nothing in Alerta. Renaming the label to `warning`, which
the other three rules in the same file already use, stopped the failures
immediately and the alerts came through.
The same labels sit on `main` and in `v1.6.1`, so a backport to
`release-1.6` would be appreciated.
The test commit adds a bats guard so the next one gets caught in CI. It
walks every `PrometheusRule` and `VMRule` under `packages/` outside
vendored `charts/`, helm templates included, and fails on a severity
outside Alerta's default alarm model, which is exactly where `warn`,
`info`, `error` and `fatal` fall. Helm actions are stripped before the
file is read, so a rule behind a `{{- if }}` or an unquoted `{{
.Release.Namespace }}` is checked rather than skipped, and a file that
still does not parse fails the contract instead of thinning it out. On
the current tree that is 194 alerting rules in 41 scanned files.
The guard found one more on `main`. `DatabaseAutoscalerAtMax` in
`packages/system/keda/templates/alerts.yaml` carries `severity: info`,
which Alerta refuses the same way. It is renamed to `informational`, the
value the model has for that level and the one seven other rules in the
tree already use.
Moving that alert off `info` also stops `InfoInhibitor` in
`packages/system/monitoring-agents/alerts/general.rules.yaml` from
firing. That rule matches any `severity="info"` alert and carries
`severity: major`, so on `main` it usually reached Alerta whenever
`DatabaseAutoscalerAtMax` fired, even though the keda alert itself was
refused. After this change no rule carries `info`, so operators no
longer see that `InfoInhibitor` notification.
Fixes #2411.
### Screenshots
Not a UI change.
### Downstream repositories
Walked the trigger map against the diff. It touches severity labels
inside `packages/system/piraeus-operator/alerts/` and
`packages/system/keda/templates/`, plus a bats file under `hack/`. No
package is added or renamed, and there is no values schema, CRD, tooling
the satellites gate on or node contract change. Nothing downstream
restates these labels.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
fix(linstor): LINSTOR/DRBD alerts carried severity `warn`, which Alerta rejects, so they were never delivered. They now use the standard `warning` severity, and the keda at-max autoscaler alert moves from `info` to `informational` for the same reason.
```
|
||
|
harshitrwt
|
ffb6564f8b |
fix(controller): drop stale bucket-class label from BucketClaim labels before applying validated value
maps.Copy(workload.Labels, bc.Labels) can bring in a workloads.cozystack.io/bucket-class label directly set on the BucketClaim, which would survive unmodified when spec.bucketClassName is empty or invalid, leaving the Workload reporting a class that does not match the source BucketClaim. Signed-off-by: harshitrwt <harshit.rawat1804@gmail.com> |
||
|
Aleksei Sviridkin
|
2c2f2c194e |
fix(dashboard, foundationdb): show the connection ConfigMap to tenants (#4612)
## What this PR does Fixes #4611. Since #4148 a tenant may read the `<release>-config` ConfigMap that holds the FoundationDB cluster file, but the console has no view for ConfigMaps, so the connection string was still reachable only with kubectl. Chart: - `<release>-resourcemap` is rendered regardless of `monitoring.enabled`, because the console now reads it; - the tenant Role grants `get`, `list` and `watch` on `<release>-resourcemap` next to `<release>-config`, still by `resourceNames`, with no wider ConfigMap access. Console: - the application page gets a ConfigMaps tab when the application's resource map lists at least one ConfigMap in the same namespace; - each listed ConfigMap is read and watched with a `metadata.name` field selector, which RBAC checks against `resourceNames`, so a tenant never needs `list` on all ConfigMaps; - values are shown as plain text with a copy button, the cluster file needs no decoding; - a resource map that is missing or forbidden hides the tab, a listed ConfigMap that cannot be read shows the API error instead of looking empty, and a malformed map or an invalid name is reported. The tab is generic, but today only FoundationDB ships a resource map, so it is the only application that gets it. #3490 keeps `monitoring.enabled` as the switch for the resource map and asks whether the map should render unconditionally. This PR answers that for FoundationDB by rendering it always, so the wording in #3490 would need to follow if both land. Tests: `helm unittest` for the chart (6 tests), `tsc --noEmit`, `vitest run` (438 tests in 60 files) and `eslint` on the changed files pass. The new tests cover reading only the listed names in the active namespace, the watch, copying, a clipboard failure, a read denial, and malformed maps. The chart renders with `monitoring.enabled` true and false. ### Screenshots The ConfigMaps tab of a FoundationDB application, rendered by the console from this branch against a local stub API with synthetic data:  ### Downstream repositories Walked the trigger map against the diff: it changes a FoundationDB chart template and its test, and the console. No `values.yaml`, `values.schema.json`, README, CRD or `hack/` file is touched. The website FoundationDB page is generated from the chart README and says nothing about connecting, and the Terraform provider does not read these ConfigMaps. The console is vendored in this repository. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(dashboard, foundationdb): the application page shows the FoundationDB connection ConfigMap with the cluster file to tenants who may read it ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Application details now include a ConfigMaps tab when configuration maps are available. View their values and copy them to the clipboard. * The tab displays loading, empty, unavailable, and error states. * **Bug Fixes** * The resource map remains available when monitoring is disabled, and its access permissions are included. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
scooby87
|
51155f1003 |
feat(postgres-operator): alert when the WAL archive backlog does not drain (#4572)
## What this PR does `LastFailedArchiveTime` compares a cluster's last failed WAL archival with its last successful one, so it resolves as soon as any segment gets through. Archiving that fails intermittently, or runs slower than the cluster writes WAL, keeps clearing that alert while the queue of segments waiting on the Postgres volume grows, until the volume is full and the database stops. The alert is also only a warning. This PR adds `CNPGWALArchiveBacklog`, a critical alert that fires when a primary's `archive_status` queue has not been empty once in 30 minutes. A healthy archiver empties the queue within seconds of each WAL switch, so a queue that never drains means point-in-time recovery is falling behind, whatever the archiver reports. Idle clusters and replicas have an empty queue and stay quiet. Validated live, since the repository has no rule-evaluation tooling: on a test cluster, two CNPG clusters archived into an object store that refused connections and into one that accepted them and never answered, while writing WAL continuously. The rule, evaluated by the root vmalert, fired for both. Over the previous seven days of the same cluster's metrics, none of its healthy clusters would have matched; those clusters carry light load, so a cluster whose archiver runs near its limit all the time has not been observed and would page, which is the intended reading of a queue that never drains. The unit test pins only that the rule ships in the rendered PrometheusRule as critical, since the rules reach the cluster through a file glob that drops a moved file silently. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches an operator chart's alert rules and its test; no path from the trigger map. ### Release note ```release-note feat(postgres-operator): page when a Postgres cluster's WAL archive queue has not drained for 30 minutes, which the existing archiving alert misses while some archives still succeed. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a critical alert for sustained PostgreSQL WAL archive backlogs. It triggers when the minimum observed ready archive status remains above zero over a 30-minute window, then stays pending for 10 minutes before firing. The alert describes potential impacts on point-in-time recovery and storage usage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
c5865061a3 |
feat(dashboard): show a connection hostname for LoadBalancer services (#4035)
## What this PR does The Services tab on the application page shows a connection hostname under the External IP of every LoadBalancer Service, as `<service-name>.<suffix>`, also while the IP is still Pending. ClusterIP Services get nothing. The suffix is a new platform value, `publishing.serviceDomain`. It travels the same path as `cluster-domain` and `branding`: `_cluster.service-domain` in the cozystack-values Secret (leading and trailing dots stripped there), then `serviceDomain` in the dashboard's `config.json`, written only when non-empty. Empty by default, and with it empty the tab shows no hostname. There is deliberately no fallback to a tenant-derived name: the app charts certify `<release>.<tenant host>` while the LoadBalancer Services are named `<release>-external-write` and similar, so a name built from the tenant apex would match neither DNS nor the certificate. Showing nothing beats showing a name nobody publishes. The value is display-only. Nothing in the platform creates DNS records for it. Set it when your DNS already resolves `<service-name>.<serviceDomain>` to that Service's LoadBalancer IP, external-dns with a matching fqdn template for example. A CoreDNS `k8s_external` zone does not fit, it answers `<service>.<namespace>.<zone>`. The shown name is not the one in the app certificates, so a TLS client may see a SAN mismatch unless DNS and certificates are aligned by the operator, and two tenants with a release of the same name get the same hostname. The values comment says all of this. Left for follow-ups: reading `external-dns.alpha.kubernetes.io/hostname` from the Service as the first source (only the opensearch chart stamps it today), and a per-Service override annotation. Parked from review, not blocking: the Admin External IPs page does not show the hostname yet, the column header still says "External IP", and surrounding whitespace in the value is not trimmed. Closes #3112 ### Screenshots <img width="1200" height="360" alt="services-tab" src="https://github.com/user-attachments/assets/01e1f3e0-836e-41cc-b916-f675b62580cd" /> ### Downstream repositories `packages/core/platform/values.yaml` gains a key, so the hand-written table in `content/en/docs/next/operations/configuration/platform-package.md` on cozystack/website needs a row for `publishing.serviceDomain`. I will open that website PR once this one merges. No other repository restates this key: ansible-cozystack passes a fixed list of `publishing.*` keys that does not include it, and the terraform provider models app schemas, not platform values. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: a `publishing.serviceDomain` row in the platform-package table, opened after this PR merges - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(dashboard): the application Services tab shows `<service>.<suffix>` under the external IP of LoadBalancer services when the new platform value `publishing.serviceDomain` is set ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added an optional service domain setting for displaying LoadBalancer service hostnames in the dashboard. - When configured, services display hostnames in the format `<service-name>.<service-domain>` alongside external IP information. - Hostnames are shown only for LoadBalancer services and remain hidden when no service domain is configured. - Dashboard configuration now receives the service domain setting automatically, including support for custom domain formatting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
harshitrwt
|
f41a500b07 |
test(controller): cover invalid bucket class label in reconcileBucketClaimForMonitor
A BucketClassName longer than 63 chars (valid as a Kubernetes object name, up to 253 chars) is not a valid label value, so the bucket-class label is skipped with an Info log rather than failing the Workload write. This can happen with SeaweedFS pool class names (<namespace>-<pool>-lock) when the tenant and pool names are both long -- a missing label in that case does not mean no class was set. Signed-off-by: harshitrwt <harshit.rawat1804@gmail.com> |
||
|
Aleksei Sviridkin
|
52c74beedd |
fix(api): let the garbage collector finalize Applications (#4610)
## What this PR does An application deleted with foreground or orphan propagation (`kubectl delete --cascade=foreground`, or any client that sends that `propagationPolicy`) could stay in deletion forever. Helm uninstalled the release and helm-controller removed `finalizers.fluxcd.io`, but the HelmRelease kept `foregroundDeletion` in `metadata.finalizers`, so the HelmRelease and its Application stayed terminating. We hit this on v1.6.4 with a Bucket and an OpenBAO application. An Application and its HelmRelease share one UID, so the garbage collector keeps a single graph node for both and may finalize it through the Application endpoint. The API never returned finalizers on an Application, and every update restored the HelmRelease finalizers from the live object. [`removeFinalizer`](https://github.com/kubernetes/kubernetes/blob/v1.33.1/pkg/controller/garbagecollector/operations.go#L85-L123) reads the owner, does not find the finalizer there and returns as if it had already been removed, so nothing ever took it off the HelmRelease. This PR: - returns `foregroundDeletion` and `orphan` in `metadata.finalizers` of an Application, and only those two; - carries an add or a remove of either of them from an Application update to the HelmRelease, as a change against the finalizers the update started from, so the result is the same when helm-controller or the garbage collector changes the list during the conflict retry; - keeps controller finalizers such as `finalizers.fluxcd.io` owned by the live object, so `helm uninstall` still runs before the release goes away; - copies the deletion timestamp and grace period from the live object into the rebuilt HelmRelease, so an update of a terminating release stays valid. `rest_garbage_collection_test.go` covers both finalizers with Helm finishing first and with the garbage collector finishing first, concurrent finalizer changes during the conflict retry, controller finalizers surviving an update, the cached HelmRelease not being mutated by the conversion, and an `UpdatedObject` that edits the object in place. All five tests fail on the current `main` without this change. `go test -race ./pkg/registry/...` passes. The tests use the controller-runtime fake client, so this has not been run against a live garbage collector with a patched cozystack-api yet. The same bug is in release-1.6. There the Update path has no finalizer carry-over at all, because #2914 is not backported yet, and its automatic backport #4577 currently carries conflict markers. A 1.6 backport of this fix should go on top of a working backport of #2914. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff: it only changes how cozystack-api handles `metadata.finalizers` of Applications in `pkg/registry/apps/application`. No package, values, schema, CRD, `hack/` file or convention is touched. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(api): an application deleted with foreground or orphan propagation no longer leaves its HelmRelease stuck with the foregroundDeletion or orphan finalizer ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Application updates now preserve Kubernetes deletion behavior, including foreground deletion and orphaning, even when concurrent changes occur. * Helm-managed protection metadata is retained during updates, and deletion timing details remain intact. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
harshitrwt
|
882929ff86 |
fix(controller): skip bucket-class label when BucketClassName is not a valid label value
Signed-off-by: harshitrwt <harshit.rawat1804@gmail.com> |
||
|
harshitrwt
|
0c37fff5f4 |
feat(controller): expose bucket class on BucketClaim Workloads
Signed-off-by: harshitrwt <harshit.rawat1804@gmail.com> |
||
|
Aleksei Sviridkin
|
97fa2a619f |
feat(dashboard): show a connection hostname for LoadBalancer services
Users who provision a service through the dashboard had no hostname to connect to: the Services tab showed only the raw LoadBalancer IP, while the pre-1.4.0 UI composed <service>.<root host> for every Service. Render <service>.<suffix> under the External IP of every LoadBalancer Service (also while the IP is still Pending), and nothing for ClusterIP Services. The suffix is config.json serviceDomain, set by the platform value publishing.serviceDomain. There is deliberately no fallback: the app charts fix their external name as <release>.<tenant host> while the LoadBalancer Services are named <release>-external-write and similar, so a name derived from the tenant apex would match neither the DNS nor the certificate. Without a configured suffix nothing is rendered. Runtime config previously reached only the Shell as a prop; it is now published through AppConfigContext and read from there by the Shell and the Services tab alike, so a single test on the rendered version pins the wiring. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
67adfbce3c |
feat(dashboard): surface service-domain in the console config.json
The console loads config.json once at startup and treats a missing serviceDomain key as "show no hostname", so the chart copies _cluster.service-domain into config.json only when it is non-empty rather than shipping an empty string the SPA would have to special-case. The helm-unittest suite pins the unset, empty and set cases. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d463e00da1 |
feat(platform): add publishing.serviceDomain for the dashboard Services tab
The dashboard has no way to show a connection hostname for a LoadBalancer Service: before 1.4.0 the old UI composed one from the Service name and the cluster's root host, and operators whose DNS publishes LoadBalancer IPs under a dedicated zone still need that. Add the suffix as a platform value and propagate it into the cozystack-values map as service-domain, next to cluster-domain and branding, so the dashboard chart can read it. The value is display-only and opt-in: empty (the default) means the console shows no hostname, because nothing in the platform publishes DNS for a guessed name. Leading and trailing dots are stripped here, the one place every consumer reads from, so an operator's ".svc.example.org." cannot render as "demo..svc.example.org."; the value is coerced to a string first because an unquoted numeric scalar would otherwise abort the whole platform render. The helm-unittest suite pins the empty default, a configured value landing in the Secret, the dot trimming and the numeric case. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
0bc3f9f25c |
fix(kubernetes): name TalosConfigTemplate by content hash (#3523)
## What this PR does Fixes #3515. `TalosConfigTemplate.spec` is immutable — the `vtalosconfigtemplate.cluster.x-k8s.io` webhook denies any mutation with `TalosConfigTemplate.Spec is immutable` — but the `talos-reconcile` Job applied it under a fixed `<release>-<group>` name on every reconcile. Once the template existed, **no change to the rendered worker machine config could ever reach the cluster**: the apply was rejected server-side while the HelmRelease stayed `Ready`. Reported with `machine.network.searchDomains` after a `clusterDomain` correction; the same held for `certSANs`, `machine.registries` and nameservers. - Name the `TalosConfigTemplate` `<release>-<group>-<6-char hash of its spec>`, mirroring the `KubevirtMachineTemplate` content-hash idiom the chart already uses, and point `MachineDeployment.spec.template.spec.bootstrap.configRef` at that name. A content change now creates a **new** template instead of attempting a rejected mutation, and CAPI rolls the workers onto it. - Extract the machine config into a single named template (`_talosconfigtemplate.tpl`) shared by the two consumers that have to agree on the name byte-for-byte: the Job, which embeds it in the heredoc it `kubectl apply`s, and the `MachineDeployment`, which hashes it into `configRef`. - Assemble the node group once, in `kubernetes-nodes.group`, and read it back at both call sites. Sharing the rendering was not enough: each file also built its own `$group` dict, so the two names agreed only by coincidence of the keys the spec helper happens to read. - The runtime values the Job substitutes (`${TALOS_CA_B64}`, `${SVC_IP}`, `${COREDNS_IP}`, tokens) stay shell placeholders and are deliberately **not** part of the hash: they are facts of an already-provisioned cluster, not chart inputs, so a certificate rotation or a Service re-address does not roll workers. ### What the shared spec carries Worker node pools live in the `kubernetes-nodes` chart, which owns `_talosconfigtemplate.tpl` outright: `make update` copies nothing from the parent chart. The helper renders everything `main` puts into the worker machine config, so the hashed spec is the one the cluster gets: - The heredoc escaping from #3513: backslash, dollar and backtick escaping on `talosVersion`, `installerRepository` and `schematicID`, the escaped `machine.registries.mirrors` block, and the `INVARIANT` comment above `data:`, which belongs in that file since it is where the next field will be added. The spec is still emitted from an unquoted heredoc in a host-cluster Pod, so an unescaped copy would be command injection. - The per-pool install image. `_talosconfigtemplate.tpl` calls `kubernetes-nodes.resolveOsImage` itself instead of reading `talos.schematicID` and `talos.version`, so a pool on `osImage.builtin` or `osImage.factory` reinstalls the schematic and version it actually booted. The install image is part of the hashed spec, so the hash has to be taken over the resolved image, or an `osImage` change would render a different config under an unchanged name. - The Proxmox branches: `nameservers` from `proxmox.dnsServers` where a KubeVirt pool points at the cluster's CoreDNS, and the kubelet's `cloud-provider: external`, without which the Proxmox cloud-controller-manager never initialises the Node. - `machine.kernel.modules`, emitted only when the pool resolves to a non-empty list. `kubernetes-nodes.group` carries every key those read, `podCpuLimit`, `podCpuRequest`, `osImage`, `proxmox` and `kernelModules` included. A key missing there is the failure this PR exists to prevent: with `kernelModules` read at one call site only, an explicit module list hashes to two different names and the `MachineDeployment` points at a template nothing creates. The `kubectl apply` stays `--server-side --force-conflicts` (#4163), so the Job is not OOMKilled on the merge path; with a content-hash name the apply is normally a create or a no-op, and server-side apply keeps the no-op cheap too. ### The replace-on-immutable path and the Role The runtime values stay outside the hash, so one failure is left that the hash cannot absorb: a new apiserver address renders a different spec under an unchanged name, and the webhook rejects it. The replace path from #4089 handles exactly that case and is kept, with its `kubectl delete` pointed at `${TCT_NAME}`. The talos-reconcile Role pins its TalosConfigTemplate `get`, `patch`, `update` and `delete` rights to the pool's own template, so it names the hashed template too. A server-side apply of a template that does not exist yet is still a PATCH on its name: a rule left on `<release>-<group>` would leave the Job Forbidden and the pool with no template to scale onto, with nothing turning red. The name-agreement guard below therefore covers the Role as well. ## Upgrade impact — please read **Existing clusters roll their Talos workers once.** The `configRef` moves from `<release>-<group>` to the hashed name; CAPI creates a new `MachineSet` for it and rolls machines onto it under the existing `RollingUpdate` strategy (`maxSurge: maxReplicas`, `maxUnavailable: 1`) plus the group's `MachineHealthCheck`. The rendered machine config itself is byte-identical to `main` for the same values, so this particular roll delivers no config change — it is the one-time cost of adopting the content-hash name, the same cost `KubevirtMachineTemplate` already paid. Proxmox pools roll the same way: the `ProxmoxMachineTemplate` name does not move, but the `configRef` does. **Until the new template exists, the pool cannot scale.** The Helm upgrade points the `MachineDeployment`'s `configRef` at the hashed name before the `talos-reconcile` Job has created that template, and there is no render-time ordering fix, since the template needs values Helm does not have. Existing workers are untouched in that window, but the pool is frozen: in CAPI v1.10.1, which this repository ships, `getTemplatesAndSetOwner` records the missing bootstrap template, and `rolloutRolling` then fails in `getAllMachineSetsAndSyncRevision` with `cannot create a new MachineSet when templates do not exist` and returns before it scales either the new or the old `MachineSet`. A replica change from the cluster-autoscaler therefore reaches no `MachineSet`: scale-up and scale-down both wait. On an upgraded cluster the Job's inputs already exist, so the window is normally a few seconds. **A failed `talos-reconcile` Job now blocks autoscaling, and nothing turns red.** After `backoffLimit: 30` the Job is not retried until the next Helm upgrade of the pool release, and the HelmRelease stays `Ready` meanwhile, as it did in #3515. Before this PR the same failure left the pool on its old template and still scaling; with it, the pool cannot scale at all until the template exists. Where to look: - the `MachineDeployment` `kubernetes-<cluster>-<group>`: its `ScalingUp` condition under `status.v1beta2.conditions` reads `Scaling up from N to M replicas is blocked because TalosConfigTemplate does not exist` (or `Scaling up would be blocked because …` while no scale-up is pending), and the capi-controller-manager logs carry `cannot create a new MachineSet when templates do not exist` for it; - whether a `TalosConfigTemplate` named after its `spec.template.spec.bootstrap.configRef.name` exists in the tenant namespace; - the `talos-reconcile` Job's status and Pod logs — but the Job has `ttlSecondsAfterFinished: 600`, so a failed Job and its logs are deleted ten minutes after it gives up. Past that, the `MachineDeployment` condition and the missing template are the only evidence left. The pre-existing fixed-name template is **not** deleted — the outgoing `MachineSet` still references it — and is garbage-collected with the `KamajiControlPlane` when the cluster is deleted. Hashed templates therefore accumulate, one per distinct machine config. Pruning them safely is filed as #4355 rather than widened into this PR. **The one roll is not the only one.** Hashing the whole rendered spec makes anything inside it a roll trigger, including two things that are not edits to this cluster's resource. For a pool sized by `instanceType`, the kubelet reservations come from a live `lookup` of the `VirtualMachineClusterInstancetype`, so a later platform-side edit to that instancetype rotates the name and rolls the pool again — and the `KubevirtMachineTemplate` hash this mirrors does not have that property, since it embeds the instancetype by name and never its resolved size. And `talos.registryMirrors` is inside the hashed spec, so setting or changing a mirror now rolls the pool rather than applying only to nodes created later; the field's documentation said nothing about this and now says it. The same holds on Proxmox for `proxmox.dnsServers`, and on every substrate for `kernelModules`: both are inside the hashed spec, so changing either rolls the pool, and the `kernelModules` documentation now says so instead of describing the fixed name as the current state. **The roll is not free on a node group holding a scarce resource.** From exercising this same rollout path on a production fleet of 13 tenant clusters, by rotating the `KubevirtMachineTemplate` hash on a GPU node group. `maxSurge: maxReplicas` is dangerous when the resource is constrained: with 6 GPUs and 1 free, CAPI created all 5 new machines at once and 3 sat `Pending` on `Insufficient nvidia.com/...`, each having already imported a 200 GiB disk. It converges, but it wastes a lot of import. The natural reflex, `maxSurge: 0` with `maxUnavailable: 1`, deadlocks instead: CAPI computes `maxScaledDown = available - (desired - maxUnavailable)`, which was `4 - 4 = 0`, so it removed no old machine while the new ones waited for exactly those machines to release their GPUs; nothing moved until `maxUnavailable` was loosened to 2. And the `MachineHealthCheck` amplifies both — at `maxUnhealthy: 100%` and `nodeStartupTimeout: 10m`, machines queuing for a GPU were declared unhealthy and remediated in a loop, three deleted and recreated while perfectly healthy, each iteration re-importing the disk. The per-node-group `nodeStartupTimeout` override fixes that, but the default is shorter than a plausible queue wait. Anyone upgrading a fleet with a GPU pool hits all three, and this PR is what starts the roll. ## Validation - `helm unittest packages/apps/kubernetes-nodes` passes, snapshots included. Pointing `configRef` back at the fixed name fails `nodegroup_test.yaml`, the snapshot and the name-agreement bats; dropping `osImage` from `kubernetes-nodes.group` fails the `worker_image_*` suites; removing the Proxmox nameserver branch fails `substrate_branches_test.yaml` and the Proxmox heredoc case. - `hack/check-talosconfigtemplate-name-agreement.bats` is new. The unittest suites pin each name against its own literal in its own file, so a divergence shows up as one literal having moved, which reads as a stale fixture and invites re-pasting the value. The guard renders the chart once and compares the Job's `TCT_NAME`, the Role's TalosConfigTemplate `resourceNames` and the `MachineDeployment`'s `configRef.name` against each other, so the failure states the invariant rather than a number. Asserted on a GPU-shaped pool as well as the default one, because the default renders `gpus []`, `kubelet {}` and `instanceType ""` as falsy and a consumer reading a subset of the group is indistinguishable there from one reading all of it; on an explicit `kernelModules` list, which neither of those shapes sets; and a last case pins that the shapes hash differently, so the others cannot hold vacuously. Verified by injecting each divergence: dropping `gpus` at one call site, dropping `kernelModules` at one call site, and pointing the Role back at `<release>-<group>` each fail it, the last one also failing the Role cases in `talos_tct_immutable_test.yaml`, as does pointing the `delete` back at the fixed name. - `hack/cozytest.sh hack/talos-reconcile-heredoc_test.bats` passes, the Proxmox cases included. This is the execution-level evidence that the escaping survived the move into `_talosconfigtemplate.tpl`: it renders with a hostile `registryMirrors` endpoint and a hostile `installerRepository`, runs the extracted heredoc through a real shell, and asserts the values come out literal rather than command-substituted; hostile `schematicID` and `version` values, and `proxmox.dnsServers` entries that are not addresses, are asserted refused at render; and the Proxmox heredoc is asserted to emit the nameservers it was given. A rendered-string regex cannot catch a heredoc the shell refuses to emit; this does. - `make generate` in `packages/apps/kubernetes-nodes` — no diff. - The "byte-identical to `main`" claim was measured against `main` on six pool shapes: default, GPU, an explicit `kernelModules` list, `kernelModules: []` on a GPU pool, `osImage.factory` and Proxmox. For each, the `TalosConfigTemplate` manifest the Job applies differs only in its name (`${RELEASE}-${GROUP_NAME}` → `${TCT_NAME}`); the spec is unchanged, the Job's `TCT_NAME` equals the `MachineDeployment`'s `configRef.name`, and the `KubevirtMachineTemplate` and `ProxmoxMachineTemplate` names do not move. - **Not verified:** no live-cluster or e2e run (needs bare-metal nodes). The upgrade sequence above, the scaling freeze included, comes from reading the CAPI v1.10.1 `MachineDeployment` controller source and the chart's existing `KubevirtMachineTemplate` precedent, not from watching it end to end. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Every box is left empty on purpose, including the first, because I am not confident about one line of the map and would rather a maintainer decided than have me tick it. The change adds no field and changes no default: the only schema movement is the description text of `talos.registryMirrors` and `kernelModules`, regenerated from their `values.yaml` doc comments. Nothing else downstream restates this chart's templates, the hash naming is internal to the chart, and no image reference, CI behaviour or CRD shape moves. The open question is whether `terraform-provider-cozystack` restates field descriptions as well as field shapes; if it does, those description changes reach it and want a follow-up. ## Release note ```release-note fix(kubernetes): name the worker TalosConfigTemplate after a hash of its content so machine-config changes (searchDomains, certSANs, registries, nameservers) actually propagate instead of being silently rejected by the immutability webhook. Upgrading rolls existing Talos workers once, on KubeVirt and Proxmox pools alike. Until the talos-reconcile Job has created the new template, the pool cannot scale, so a failed talos-reconcile Job now blocks cluster-autoscaler scale-up and scale-down while the HelmRelease stays Ready; check the MachineDeployment's ScalingUp condition ("blocked because TalosConfigTemplate does not exist") and the Job status, which is deleted ten minutes after it fails. On a node group holding a scarce resource such as GPUs, review the MachineDeployment surge settings and the MachineHealthCheck nodeStartupTimeout first: `maxSurge: 0` can deadlock the roll and the default startup timeout can trigger spurious remediation while machines queue for the resource. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Worker Talos configuration templates now use content-hashed names for consistent reconciliation and bootstrap references. * Changes to worker machine configuration, including GPU and registry mirror settings, automatically roll existing workers. * **Documentation** * Updated registry mirror documentation to explain that configuration changes roll all workers in the pool. * **Tests** * Added coverage for hashed template names, configuration-triggered worker rotations, and matching references across worker resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Yan Bondarenko
|
2fa7fea29b |
fix(dashboard): show ConfigMaps listed in the application resource map
FoundationDB keeps its connection string in a ConfigMap that its operator writes, and tenants may read it, but the console had no view for ConfigMaps. Tenant roles grant ConfigMaps by name only, so the tab takes the names from the application's resource map and reads each one with a metadata.name field selector, which RBAC checks against resourceNames. An application without a readable resource map shows no tab, while a listed ConfigMap that cannot be read is reported instead of looking empty. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
e3ae8033e6 |
fix(foundationdb): let tenants read the dashboard resource map
The resource map is the only object that names the connection ConfigMap for a client such as the dashboard. It was rendered only with monitoring enabled, and the tenant role could not read it, so a tenant session had no way to find the ConfigMap it is allowed to read. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
46b5a542fe |
fix(api): let the garbage collector finalize Applications
An Application and its HelmRelease share one UID, so the garbage collector keeps a single graph node for both and may finalize it through the Application endpoint. The API never returned finalizers on an Application, and every update restored the HelmRelease finalizers from the live object. When the collector came to drop foregroundDeletion or orphan through the Application, removeFinalizer found no such finalizer and returned as if it were already gone. The HelmRelease then kept foregroundDeletion after Helm had uninstalled the release, and the deletion never completed. The Application now carries the two garbage collection finalizers, and an update that adds or removes one of them applies the same change to the HelmRelease. Controller finalizers such as finalizers.fluxcd.io stay owned by the live object, so helm uninstall still runs before the release goes away. The deletion timestamp and grace period are copied from the live object, which keeps an update of a terminating release valid. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Aleksei Sviridkin
|
4f043b572e |
fix(api): pass the stored object to UpdatedObject in tenantsecret Update (#4603)
## What this PR does A PATCH against an existing TenantSecret returned 404 because `REST.Update` passed `nil` as the old object to `objInfo.UpdatedObject`, and the apiserver patch handler builds the patched object from that old object alone. `Update` now passes the stored Secret (converted with `secretToTenant`) and keeps `nil` only for the `forceCreate` path. The nil also made the patch reach admission as CREATE instead of UPDATE. Validation: the new `TestUpdate_PassesStoredObjectToUpdatedObjectInfo` fails before the change and passes after; `go test ./pkg/registry/...` and `go build ./pkg/... ./cmd/...` pass. I did not run a live `kubectl patch` or the real apiserver patch handler. Known limitation, unchanged by this PR: `tenantToSecret` prefers `Data` over `StringData` when `Data` is non-empty, so a patch that only sets `stringData` on a secret that already has data is a no-op, as a GET-then-PUT edit of `stringData` already is. Patching `data` or metadata works. Fixes #3601 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(api): PATCH, `kubectl label` and `kubectl edit` on an existing TenantSecret no longer fail with 404 NotFound. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Tenant secret updates now correctly apply changes using the existing stored value, including label updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
