mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-22 04:33:36 +09:00
f83db514b0eb88cefc2923d17aad59daccb4e358
7777
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
422b6f65ea |
test(vpn): add default render helm unit tests
vpn was one of the remaining user-facing apps with no helm-unittest coverage. Signed-off-by: Akanksha Trehun <akankshatrehun@gmail.com> |
||
|
|
a4547a6fe0 |
test(kube-ovn): compare image copies against upstream's Dockerfile
packages/system/kubeovn/images/kubeovn/Dockerfile is a hand-kept copy of upstream's dist/images/Dockerfile, and `make update` rewrites only its ARG VERSION and ARG TAG. A binary upstream starts shipping therefore does not reach our image on its own, and nothing read the two files together: kube-ovn-bfdd-supervisor went missing that way with the whole suite green. hack/check-kubeovn-image-copies.sh normalizes the COPY instructions of both files and reports either side's surplus. It compares only the stages built FROM kubeovn/kube-ovn-base, because our builder stage clones and patches kube-ovn and its COPY lines have no upstream counterpart, and it treats an empty result as a broken check rather than agreement. `make update` now runs it against the tag it pulls, so the next bump stops on a mismatch, and hack/check-kubeovn-image-copies.bats runs it on every CI run against a verbatim copy of upstream's Dockerfile at the vendored appVersion. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
b59510abe6 |
fix(kube-ovn): ship kube-ovn-bfdd-supervisor in the image
v1.15.26 adds cmd/bfdd_supervisor and builds it as dist/images/kube-ovn-bfdd-supervisor (makefiles/build.mk:33), and upstream's dist/images/Dockerfile:12 copies it into /kube-ovn. Ours, which the comment above its COPY block names as a copy of that file, stopped at kube-ovn-controller. pkg/controller/vpc_egress_gateway.go:47 hardcodes /kube-ovn/kube-ovn-bfdd-supervisor. A VpcEgressGateway with spec.bfd gets a bfdd container that takes the binary as its command (:1014) and probes it for liveness (:1107), from the kube-ovn image itself, so without the COPY that container cannot exec. The vpc-egress-gateways.kubeovn.io CRD ships from this chart, so the path is reachable. v1.15.10 has neither the binary nor the container, so this arrives with the bump rather than predating it. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
41ef4a6437 |
fix(ci): teach the overlay to repair an unstamped images/*.tag ref (#4362)
## What `hack/overlay-main-images.sh` walks `images/*.tag`, but every branch of its line classifier needs either a YAML key or an `@sha256:`. A `.tag` file holds one bare reference and nothing else, so until a build stamps it the line has neither, the drift guard reads it as a non-ref change, and the committed reference survives. For a package onboarded with the `:v0.0.0` placeholder this repo already uses for a new image, that surviving reference is one no registry serves. The overlay is the only thing that would replace it for a PR that does not rebuild the package, so once such a package is on main every full-suite run that skips its build keeps the placeholder. The same placeholder inside a `values.yaml` repairs itself, because there it sits under an `image:` key the classifier recognises. The difference is the file shape, not the placeholder. ## Reproduction A fixture holding a committed bare tag against an artifact copy carrying a stamped one: ``` $ cd fixture/fakerepo && sh ../../hack/overlay-main-images.sh ../mainpkgs '[]' '' overlay: packages/system/x/values.yaml -> current-main drift (non-ref change) in packages/apps/x/images/x.tag -> keeping committed ref ``` Both files differ only in their reference. The `values.yaml` one is repaired; the `.tag` one is not. Adding any digest to the same placeholder makes it pass, which is what narrows this to the classifier rather than the file. ## Why it has not bitten yet 17 of the 19 `.tag` files in the tree carry `@sha256:` and pass on that branch alone. The only other digest-less one is `packages/extra/seaweedfs/images/seaweedfs-cosi-driver.tag`, and it survives only because it is third-party: `build-main` never rebuilds it, the artifact copy is byte-identical, and the classifier is never reached. The first digest-less first-party `.tag` is what turns the gap live, and one is coming in #3426. ## The fix `hack/promote-rewrite-tags.sh` already carries a second match branch for exactly this shape, with a comment saying why ("the second branch is what catches a bare `.tag` file, whose content has no YAML key at all"). This gives the overlay the same one, scoped to `*.tag` so an unkeyed line in a `values.yaml` still counts as drift, because only in a `.tag` file is a whole line known to be a reference and nothing else. Two tests in `hack/overlay-main-images_test.bats`: the unstamped placeholder is overlaid, and a `.tag` whose non-reference line differs still counts as drift, so the rule cannot quietly widen into "every line in a `.tag` file is a reference". Removing the fix fails the first one. ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Fixed image overlay handling for `.tag` files containing bare image references, allowing current references to be applied correctly. - Preserved drift detection when `.tag` files also contain unrelated, non-reference changes. - **Documentation** - Clarified how image-reference tooling handles bare references and package-tree processing. - **Tests** - Added coverage for bare-reference overlays and mixed reference/comment changes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
78f01d5542 |
docs(rabbitmq): point the preset section at the shared size matrix
The preset table in this README listed the seven legacy aliases only, and its CPU column disagreed with cozy-lib on every row, so a reader looking up what the new s1.nano default buys them found neither the name nor a correct number. Nine app READMEs already replaced that table with the two paragraphs that describe the <series>.<size> convention and link the full matrix in docs/operations/resource-presets.md; this README now carries the same text. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
|
6d5666f7e6 |
fix(rabbitmq): describe the preset as filling what resources leaves unset
cozy-lib merges the preset under resources key by key, so a partially set resources map still takes the preset for everything it omits: with resources.cpu=700m alone the broker renders 700m of CPU against the preset's 512Mi of memory, and this upgrade moves that broker from 128Mi to 512Mi as well. The description said the preset applies when resources is omitted, and cozyvalues-gen copies it verbatim into values.schema.json, the README parameter table and the cozyrd openAPISchema the dashboard renders, so one sentence made the claim on three surfaces. The suite covered a fully omitted resources map and one that sets both cpu and memory, which is the one shape that does not exercise the merge. The new case pins it: reverting defaultingSanitize to replace the preset instead of merging over it turns that case red and leaves the other three green. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
|
03394bd5fe |
chore(rabbitmq): drop the test target the rebase duplicated
main already carries a test target at the bottom of the Makefile; the rebase kept the earlier copy at the top and GNU make warns about the override. The Makefile matches main again. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
|
cf749988a4 |
fix(rabbitmq): right-size the default resources preset to s1.nano
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> Assisted-by: LLM |
||
|
|
c0ff783026 |
fix(hack): make bats unit tests pass on a BSD/macOS checkout (#4284)
## What this PR does Refs #4272 (Defects 1 and 2). On a clean checkout of `main`, `make unit-tests` is red out of the box on macOS for reasons that have nothing to do with the change under test: the BSD userland shipped with macOS differs from the GNU tools on the Linux CI runners, and CI never sees any of it. This PR closes the two worst of those, both scoped to `hack/`. **Defect 1 — the runner cannot parse a non-ASCII `@test` title.** `hack/cozytest.sh` rebuilds each test's function name by walking its title one character at a time with awk `substr()`. BSD awk aborts that walk on the first multibyte byte (`towc: multibyte conversion failure`) rather than returning it, so a file carrying a non-ASCII `@test` title never parses and not one of its tests runs — silently, because the suite only reports that the runner errored. Four `hack/*.bats` files carry such a title today. The fix pins `LC_ALL=C` on all three awk passes (title parse, run selection, and the exit-code scan of the captured log) so each reads the input byte-wise; the parse and selection passes must share the ctype or they would derive mismatched function names for one title. A new regression test, `hack/cozytest-nonascii-title.bats`, builds a multibyte-titled fixture at runtime and asserts the runner actually runs it; it is red against the pre-fix runner (mutation-verified) and green after. **Defect 2 — GNU-only `sed -i`.** `hack/nightly-mirror.sh`, `hack/promote-rewrite-tags.sh`, and the fixture builder in `hack/promote-rewrite-tags_test.bats` edit files in place with `sed -i`. BSD sed reads the argument after `-i` as a mandatory backup suffix, so `sed -i -e` and `sed -i -E` swallow the following flag as that suffix and the edit either errors or silently no-ops. Each is rerouted through a temp file, portable to both seds. The `nightly-mirror.sh` rewrite additionally relied on the GNU `\?` optional quantifier, which BSD BRE reads as a literal `?` so a bare host was never matched; it is replaced with the POSIX `\{0,1\}`, equivalent on both. All three suites are green on macOS after the change, and behaviour on GNU sed is unchanged. ### Scope This PR is deliberately scoped to Defects 1 and 2. Defect 3 from the issue — a GNU-only `date -d` in `hack/seaweedfs-naming-audit.sh` — is tracked separately as #4273 and is left untouched here. For transparency: with Defect 1 fixed, several `hack/*.bats` suites that previously never ran now run, and a handful still fail on macOS because they depend on GNU coreutils `timeout`, which base macOS lacks (`capture-dataplane`, `cozytest-capture-gate`, `run-kubernetes-join-timing_test`, `run-kubernetes-runner-cpu_test`, `runner-identity`). Those failures are pre-existing — identical before and after this change when the runner is swapped in place — and are a distinct defect from the two fixed here, so they are out of this PR's scope. A fully green `make unit-tests` on macOS additionally depends on #4273 and on that separate `timeout` gap. ### Verification All fixes were verified on a genuine macOS host (Darwin, BSD awk 20200816 / BSD sed). `hack/cozytest-nonascii-title.bats`, `hack/nightly-mirror_test.bats` (5 tests), and `hack/promote-rewrite-tags_test.bats` (17 tests) are green; the two files the issue reports as running zero tests (`overlay-main-images_test.bats`, `promote-gate-contract.bats`) now yield 13 and 15 `Test OK`, matching the issue's numbers. Each fix is mutation-proven: reverting it turns the corresponding suite red again. ### Downstream repositories The diff touches only internal `hack/` tooling — the bats test runner and two release scripts. No downstream repository vendors or restates these: `cozystack/ccp` and `cozystack/external-apps-example` gate on `hack/package.mk`, `hack/common-envs.mk` and `hack/update-crd.sh`, none of which this PR changes, and nothing renames or moves a file or alters what a make target does. - [x] No downstream repository is affected by this change ### Release note ```release-note fix(hack): make the bats unit-test runner parse non-ASCII `@test` titles under BSD awk, and route the in-place edits in `nightly-mirror.sh` and `promote-rewrite-tags.sh` through a temp file so they work under BSD sed — closing two of the host-tool gaps that make `make unit-tests` red on a clean macOS checkout ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved test runner compatibility with non-ASCII test titles and output. * Improved text-processing and file-rewrite behavior across BSD and GNU environments. * Preserved file permissions and metadata during automated rewrites. * Improved handling of host values and release tags across supported platforms. * **Tests** * Added coverage for non-ASCII test titles. * Updated rewrite tests for portable BSD and GNU tooling. * **Documentation** * Clarified supported BSD and GNU command-line tool compatibility. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8849aa13de |
fix(mongodb): keep a completed archive reachable and a legacy Backup unblocked
Once the operator CR is ready the archive exists, and the Backup object is the only thing that can ever reach it again. A transient failure creating that object (etcd timeout, quota, webhook) failed the job terminally with no artifact, and the terminal-phase guard stopped any retry, so the dump sat in the shared bucket with nothing to reclaim it. The failure is now retried with backoff for a window after the dump completed, and when it does give up the job names the operator CR that still holds the archive. Cleanup read the operator CR before deciding whether the Backup owned anything, so a legacy Backup — an unconditional no-op before this driver — could be held Terminating by a throttle or a revoked verb on a CR the driver never owned. The snapshot records the flow, so a legacy Backup now releases before any apiserver read; a missing or undecodable snapshot still falls through to the read. The opt-out refusal recognised the lingering injected entry only by the strategy's current credentialsSecret and only while the strategy still carried s3, so a platform change to the strategy after a tenant opted out let a legacy CR be minted onto the platform bucket. The platform default Secret name is now recognised as well, independent of the strategy's current block. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
f004b35fb4 |
fix(mongodb): record the CR's flow on the artifact and bound the two open loops
Reading a backup's flow off its CR's finalizer had reached the state switch but not the artifact: the snapshot was still written from the app flag, re-read on the completing reconcile. A flag flipped to false while a system-bucket dump streamed produced a Backup whose snapshot called a cozy-backups archive legacy, so the point-in-time refusal and the S3 backfill both switched off for it. The artifact now takes the same flow the switch does, pinned by a test that flips the flag between mint and completion. Two waits had no upper bound. A live-state read that kept failing at the deadline deferred every poll behind a debug line with no condition, so an unreadable CR pinned the job Running with nothing to act on; it is now a named wait and fails after a grace, with the CR left in place rather than deleted blind. The storage-race retry minted a fresh operator CR per poll while the previous ones terminated, for the whole deadline; it now stops after a small budget and surfaces the operator's error. On useSystemBucket the chart already guarantees backup.enabled, so a strategy without s3 coordinates failed with a remedy the tenant had in place; the wait names the missing coordinates instead. The missing-credentials message no longer blames useSystemBucket on a release that never enabled it. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
6a34c9f037 |
fix(opensearch-operator): name the component reconciler a stall sits in
The cluster controller runs nine component reconcilers in sequence and none of them logs unconditionally when it is entered: the TLS one's only line above its work sits on the no-security early return, which the chart in this repository never renders, and the rolling restart one opens at V(1), off at the verbosity the operator ships with. So a call that stops returning leaves nothing behind at all. The operator log ends on whatever ran before it, the object is silently no longer reconciled, and from the outside that is indistinguishable from an operator with nothing to do. Two e2e runs ended exactly that way, the operator going quiet at the moment the chart rewrote the CR and never coming back: https://github.com/cozystack/cozystack/issues/4332. Each component is now wrapped so a call still in flight after a minute reports its own name and how long it has been running, and keeps reporting on that interval while it is stuck. A single line dated at the start of a stall does not answer the question a reader has five minutes in, which is whether the call is still there. Nothing is written while a component behaves, so a healthy cluster costs no extra lines. The finalizer path is left alone: every DeleteResources at this version returns immediately, so a watchdog there could only spawn goroutines for calls that cannot block. A diagnostic rather than a fix: drop it once the stall itself is understood. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
bad152443e |
fix(kafka): correct the stale co-namespaced migration render error
The brokerPoolName render guard still told operators to "migrate one at a time or use separate namespaces" when a sibling cluster owned the shared "kafka" pool. That predates automatic name freeing: the migration hook now drains the first cluster off the shared pool with a Cruise Control rebalance and releases the name, and the waiting cluster's HelmRelease retries into it — every co-namespaced Kafka reaches KRaft, no manual action required. Reword the fail message and the helper comment to describe the transient wait-and-retry, and update the unit test's error pattern to match. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
0b9332877e |
test(kafka): drop the false-green ZK->KRaft migration chainsaw suite
The kafka-zk-to-kraft-migration suite did not exercise the pre-upgrade hook, it reimplemented a simplified flow beside it: a controller pool named "controller" instead of the hashed c-<hash>, no strimzi.io/next-node-ids pin, and a single broker so the controller-vs-broker node-id collision could never occur. That is exactly the defect the hook exists to prevent, so the suite stayed green on a revision whose in-place migration lost every message — reproduced by hand 2/2 on a live cluster. A suite that cannot fail on this code is worse than no suite: it reads as migration coverage and invites trust it has not earned, while costing ~380s on every Kafka PR and asserting only that a PVC object exists, never reading a record back. The migration hook's bats suite and the live upgrade runs are the real coverage; the Strimzi bump would delete this suite regardless. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
1c88014661 |
fix(installer): keep node_modules out of the packages artifact (#4290)
## What this PR does `flux push artifact` walks the whole `packages/` tree and reads neither `.gitignore` nor `.sourceignore`, so a working copy where somebody ran `pnpm install` under the vendored console ships the console's dependency tree inside the bundle. Helm then refuses the dashboard chart on every cluster that pulls it, because one dependency file sits above the 5MB per-file limit, and the build itself reports nothing: ```text Could not load chart: decompressed chart file "dashboard/images/console/node_modules/.pnpm/@rolldown+binding-darwin-arm64@1.0.0-rc.15/node_modules/@rolldown/binding-darwin-arm64/rolldown-binding.darwin-arm64.node" is larger than the maximum file size 5242880 ``` Passing `--ignore-paths` replaces flux's built-in list rather than extending it, so the defaults are restated alongside `node_modules/`. `hack/check-packages-artifact-ignores.bats` holds both halves: dropping `node_modules/`, dropping one of the defaults, or declaring the variable without passing it to the recipe each turn the test red. Its second case re-reads the defaults out of the installed flux, so an upstream addition surfaces as a failure rather than as a pattern quietly missing from the artifact. Measured on a tree with the console's dependencies installed: the artifact goes from 80MB carrying 32397 `node_modules` entries to 6.7MB carrying none, with the dashboard chart still in it. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change The two workflows that push the same artifact themselves (`e2e-fork.yaml`, `nightly.yaml`) run from a clean CI checkout, where the tree has no `node_modules` to pick up. ### Release note ```release-note fix(installer): the packages OCI artifact no longer picks up node_modules from a developer's working copy, which made the dashboard chart unloadable on every cluster that pulled the bundle. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Package artifacts now exclude unnecessary files, including version-control metadata, common media and archive formats, and dependency directories. * This helps prevent oversized files from exceeding Helm’s per-file size limit. * **Tests** * Added validation to ensure package artifact ignore patterns remain complete and aligned with the packaging tool’s defaults. * Optional checks report when the packaging tool is unavailable without failing the test suite. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
ac92b6b95c |
fix(e2e): cap the bucket verify S3 reads so the step honors its timeout
A Chainsaw `script` op runs its shell under exec.CommandContext with stdout and stderr wired to in-process buffers. When the op timeout fires it SIGKILLs the shell process alone, not its descendants, and cmd.Wait() does not return until every process that inherited the pipe fds closes them. An mc that keeps retrying an S3 call after the shell is gone holds the pipe open, so the op runs to mc's own exhaustion instead of stopping at its ceiling. This is why the bucket suite's verify-every-bucket-takes-a-write step ran 606s under a 300s ceiling. verify-s3-access shares that shape, so wrap each S3 and apiserver read in its own `timeout`: a hung read dies at a known budget and the op timeout stays a backstop. The 30s kubectl cap matches the terminal `k` wrapper the redis suite carries. The two `mc cp` PUTs take 45s, the budget verify-every-bucket-takes-a-write already gives a write, because a working PUT can still wait while the master grows a volume for a freshly created bucket; 15s left too little headroom on that grow and could turn a slow-but-healthy write red on an unrelated PR. The 15s cap on the mc reads has no terminal precedent: the backup-access preflight caps `mc cp` at 15s too, but inside a retry loop where a 124 costs one attempt rather than the step, so it is not one. Land each secret's encoded value in a file before base64 -d so a timed-out kubectl read (exit 124) stops the step under set -e rather than masking as an empty-credential mc alias set further down. The readonly-upload deny probe branched on mc directly (if mc cp ...), which suspends set -e, so a timeout there (exit 124) was indistinguishable from the access-denied non-zero the probe expects: a hung deny path would exit 124, take the false branch, and reach a green cleanup, the exact window this change bounds. Capture the status instead: exit 0 is the readonly regression, 124 is a timed-out probe, both fail; any other non-zero is the expected denial. The suite's own catch read (the `kubectl get pods` in namespace cozy-objectstorage-controller) carried neither an op timeout nor an inner cap, so it inherits timeouts.exec (5m) and can hold the pipe the same way; give it a 30s inner cap and a 1m op timeout so the rule the rest of the file follows holds for it too. The .chainsaw.yaml notes still read the op ceiling as a reliable cut, the old SIGKILL as a de-facto cap and a smaller timeout as guaranteeing a kill; correct them, since the shell-only kill plus the pipe wait are why an inner cap on every read is required rather than optional. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
a8c4119050 |
fix(multus): move kube-ovn CNI plugins marker to v1.15.26
The cni-plugins-staging-contract test pins the kube-ovn release whose CNI_PLUGINS_VERSION was last checked against multus's, and bumping kube-ovn to v1.15.26 trips it while the marker still reads v1.15.10. kube-ovn v1.15.26's base image (dist/images/Dockerfile.base) still uses CNI_PLUGINS_VERSION=v1.9.1, identical to multus, so portmap and macvlan stay in lockstep across the two daemonsets. Only the marker moves; the plugins pin and per-arch checksums are unchanged. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
75eebb5f3c |
feat(kube-ovn): bump kube-ovn to v1.15.26
Update kube-ovn from v1.15.10 to v1.15.26. v1.15.26 removes the checkDBClusterIntegrity check from the OVN leader checker (pkg/ovn_leader_checker/ovn.go). In v1.15.10 that check called os.Remove on a healthy local raft DB whenever a leader observed fewer members than expected (ovn.go:367). That count drops below the target transiently during cluster growth, a rolling restart, or recovery after a node address change, so the check could delete a valid DB and leave ovn-central unable to elect a leader. The function is gone in v1.15.26, so the DB is no longer deleted. The chart is re-vendored with `make update`, which re-pulls charts/kube-ovn from the v1.15.26 tag, re-applies patches/mtu.diff, and regenerates the image build args. This brings the full v1.15.26 chart: the reworked master-node affinity helpers, the widened kube-ovn CRD schema (bandwidth ingress/egress become int-or-string, u2oFeatures added), and the new OVN_CENTRAL_MODE (cluster/single) support with its central-svc, central-pvc and central-hcp-rbac templates. OVN_CENTRAL_MODE defaults to cluster (raft on hostPath), so an in-place upgrade keeps the existing topology. Two vendored patches are dropped as now upstream in v1.15.26: fix-keepvmip-portgroup.diff (hasAliveSiblingVMPod, pod.go:2437) and fix-vmim-scheduling-retry.diff (scheduling requeue, kubevirt.go:195). fix-vmim-target-pod-retry.diff is kept: v1.15.26 still returns nil at kubevirt.go:215. The image ref stays digest-pinned (repository + tag@sha256), which the promote-retag release contract requires for a cozystack-built image. The digest is carried over from v1.15.10 as a placeholder and is refreshed to the real v1.15.26 image by the release-candidate digest-vendoring step; no PR or main job rebuilds it. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
b18281ef0c |
feat(dashboard): serial console for virtual machines, and paste into the VNC console (#4356)
## What this PR does Adds a serial console tab to the VM detail page, and makes the VNC console accept a paste. Moving text into a VM was the starting point, and the two consoles answer it very differently. The VNC console carries pixels and key events and nothing else, so a paste there has to be typed, one emulated key press per character. The serial console carries text, so a paste is a single write and the guest's output is real text the user can select and copy. Measured on a running VM, 1000 characters, checked by piping the pasted string through `md5sum` in the guest and comparing with the digest computed locally: | | time | integrity | | --- | --- | --- | | serial console | **0.87 s** | digest matched | | VNC, 25 ms per key event | 56.6 s | digest matched | | VNC, no pacing | 10.3 s | **characters dropped and reordered** | The VNC rows were taken by driving the typing loop directly rather than through a real keypress, so they measure the typing and not the shortcut. The pacing is the measured floor either way: with none, the guest loses and reorders characters. ### Serial console New tab backed by KubeVirt's `virtualmachineinstances/console` subresource and xterm.js. A VM's tabs now lead with its two consoles — serial first, labelled Console, with VNC beside it — because reaching a console is why people open a VM. VNC keeps its place: the serial port does not reach a desktop session, a bootloader, or a Windows guest, and Windows guests are supported here, down to their own scheduling in the VM charts. Copy stays on the terminal's own chord, Ctrl+Shift+C or Cmd+C, so a plain Ctrl+C still interrupts the foreground process. Paste needed one thing handled, and only off a Mac: xterm turns Ctrl with a letter into a control byte and cancels the keydown, so Ctrl+V never reached the browser's paste and the guest received SYN (0x16) instead. Reproduced on a real terminal, where the shortcut put a single `0x16` on the wire and nothing else. The key handler now returns that chord to the browser, at the cost of the terminal's quoted-insert. On a Mac the browser pastes on Cmd+V, so there the terminal keeps the key. One detail worth knowing before touching this code: virt-api negotiates the `plain.kubevirt.io` subprotocol and reads **binary frames only**. A text frame is accepted by the socket and then silently dropped, which is indistinguishable from a dead console. That is why the stream is a small module of its own with its own tests, rather than xterm's stock attach addon, which sends text. ### Paste in the VNC console The graphical console still matters where the serial port cannot reach — a desktop session, a bootloader, Windows — so a paste works there too, typed key by key. **The shortcut is the platform's own: Cmd+V on macOS, Ctrl+V everywhere else.** That means Ctrl+V no longer reaches the guest on Linux and Windows, where it is visual block in vim or the guest's own clipboard. Taking both chords bought nothing, because no browser pastes on Ctrl+V on a Mac, and cost the guest a key it has its own use for. Nothing is added to the interface: the toolbar's existing status line reports `Pasting 12/270` while the text goes in, then returns to the connection state. Four constraints shaped the rest, each found by running it rather than by reading: - `navigator.clipboard.readText()` puts a permission prompt in front of **every** paste. Instead the shortcut moves the focus to a hidden field and lets the browser paste into it, which costs nothing and asks nothing. - noVNC calls `preventDefault()` on keydown. Letting the shortcut reach it would cancel the browser's paste outright, so the shortcut is taken on the capture phase and only its propagation is stopped. If no paste follows, the focus returns to the console instead of being parked in an invisible box. - The paste starts while the user is still holding the chord, and noVNC told the guest that modifier is down the moment it was pressed. Typing into that state sends chords rather than characters — under Ctrl an `m` is Return — so the host's modifiers are released before the first character. On Windows noVNC sits on the first Ctrl for 100 ms and delivers it on a timer, after that release has already happened, so the release is sent once more when the pacing has covered that window. - A character the keymap cannot produce changes what the rest means, and the newline after it still submits: `rm -rf /srv/данные` would become `rm -rf /srv`. Nothing is sent unless the whole paste can be typed, and the toolbar says how many characters were in the way. The keymap covers US only. A guest in another layout is the case the serial console answers, where text travels as bytes and no layout is involved. ### Screenshots The serial console on a running VM: the tabs lead with it, the terminal is live, and the middle line was pasted from the browser clipboard. <img width="2252" height="1606" alt="cozystack-serial-console" src="https://github.com/user-attachments/assets/6fac5f95-c142-405b-9b54-8ed040d65167" /> ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: <!-- The diff touches only packages/system/dashboard/images/console. Walked the trigger map in docs/agents/contributing.md file by file: no entry matches a console-only change, and the map states outright that UI changes belong in this repository, in the same PR. --> ### Notes for the reviewer - New dependencies: `@xterm/xterm` and `@xterm/addon-fit`. The tab is split at the route with `React.lazy`, the way the YAML editor already is, so both land in a 347 KB chunk fetched when a console is opened. Built both revisions with the same `node_modules`: the entry chunk every dashboard page loads goes 935.91 KB → 896.39 KB, so it ends up smaller than before this branch. - The serial console needs a getty on `ttyS0` in the guest. Cloud images enable it by default; a Windows guest or a graphical desktop is the VNC console's job. - The serial console opens with the user's own identity — nginx forwards the session's `Authorization` untouched — and `virtualmachineinstances/console` is already granted next to `/vnc` in `cozy:tenant:use:base`, so no RBAC change is owed. - Verified against a live VM on a dev cluster: connection, typing, a 1000-character paste round-tripped through `md5sum` on the serial console, text typed before the serial socket opens (held and flushed), a real Ctrl+V on the VNC console arriving intact, and two pastes in quick succession (the second is refused rather than interleaved). - The Windows modifier path is read from the pinned noVNC in `node_modules`, not run: there is no Windows host here. The test fake models that delivery, so the behaviour is at least held by a test. - There is no cancel affordance for a paste in progress. It stops on its own when the session drops or the tab is left, and the toolbar then says how far it got; Reconnect is the manual way out. A length cap is worth adding separately — at 25 ms per character a large clipboard types for a long time. - 418 tests pass; build and typecheck are clean. `eslint .` reports 54 pre-existing problems across twenty files, none of them in the files this PR touches. ### Release note ```release-note feat(dashboard): virtual machines gain a serial console tab, where pasting is instant and the output can be selected and copied. The VNC console now pastes the host clipboard on the platform's own shortcut — Cmd+V on macOS, Ctrl+V elsewhere — typing it into the guest on a US keyboard layout; Ctrl+V no longer reaches the guest on Linux and Windows. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a serial console for running virtual machines, including terminal input, output, resizing, reconnection, and connection status. - Added clipboard paste support to the VNC console with keyboard-layout handling, progress feedback, and unsupported-character reporting. - Added platform-aware paste shortcuts for macOS and Windows/Linux. - **Improvements** - Console content now loads on demand with a loading indicator. - Added handling for stopped and unsupported virtual machine states. - **Documentation** - Documented keyboard, clipboard, and console testing behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
5e3b8c217b |
fix(mongodb): refuse useSystemBucket without backup.enabled and pin the retry's flow
The pbm agents run only when backup.enabled is true, so useSystemBucket with backups disabled renders a cluster that can never service a BackupJob, and the tenant learned that thirty minutes into the first job when the driver's precondition gave up. The chart already fails closed on the bootstrap clash; do the same here so the combination is a render-time error with the remedy in it. The storage-race retry keyed on the app flag while everything else in the state switch had moved to the flow recorded on the CR's own finalizer, and that flag term was the only thing keeping a legacy tenant's operator CR from being deleted and retried on the same operator error — with no test holding it. Key it on the CR's flow like its neighbours and pin the legacy side: the error is terminal and the CR survives. The retry matches the operator's own error wording, which is correct at the pinned crVersion and nothing else ties together: a bump that rewords it turns the retry off with the suite still green. Name the coupling at the pin and at the matcher so a bump re-checks it. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
4233e90792 |
fix(mongodb): decide a backup's flow from its own CR, not the app flag
The opt-out refusal ran before the lookup of the job's existing operator CR, so it was a per-poll gate rather than a mint-time one: flipping backup.useSystemBucket to false while a dump streamed routed every later reconcile into the refusal, failed the job at the deadline and left the CR minted under the old flag streaming into the platform bucket with a finalizer no Backup would ever resolve — the stranded archive the running exemption exists to prevent. The refusal now applies only while the job has no CR, and everything after the mint reads the flow off the CR's delete-backup finalizer, which is fixed at mint time, instead of the mutable app flag. A late dump on the legacy flow is likewise handled by its CR's flow: the deadline fails the job and leaves the operator CR, as before this driver existed. The refusal also identified the injected entry by bucket name and told the operator to wait for the chart to re-render it, which the docs say never happens. The entry is now recognised by its credentialsSecret, the one field on it the driver chose (bucket names collide with an admin's external bucket and drift when the platform bucket is re-provisioned), and the message names the manual removal the docs prescribe. At the pinned operator `requested` is set right after the command is sent to pbm, so deleting a CR in that state cannot recall it and a late dump would land with nothing to own it. Only an unobserved or queued CR is cancelled; a requested one is left as the handle to whatever pbm writes, and the job fails. The credential pre-check failed at once where every sibling precondition waits for the restore deadline, and it re-ran against a restore already replaying, so a Secret pruned mid-replay marked an in-progress restore Failed. It now waits like its neighbours and is skipped once the operator restore exists. The status-write suppression on the waiting helper is pinned by counting status updates across repeated reconciles. The legacy deadline's justification claimed psmdb task retention covers the archive; the operator prunes only its own scheduled backups, so the object is the tenant's to reclaim, and the comments say that instead. The bootstrap refusal names the value that actually clears it (bootstrap.enabled is not reset after a bootstrap restore). Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
42a29e3597 |
fix(mongodb): keep the legacy deadline on a running dump and name the wait
Exempting state=running from the wall-clock deadline was justified by the shared bucket: on the useSystemBucket flow a failed BackupJob leaves pbm streaming an archive no Backup object represents. The exemption was applied to both flows, though, so a legacy dump whose operator or node died mid-run went from failing on the deadline to polling forever with no condition and no Event. Gate the exemption on the flow: a legacy archive lands in the tenant's own bucket under psmdb task retention and the pre-existing contract failed the job while leaving the operator CR alone, so that is restored. On the system-bucket flow the wait is now named on the BackupJob (Ready=False with the start time) instead of being silent; the waiting helper writes status only when the condition changes so a dump that runs for hours does not cost a status write per poll. The point-in-time refusal read the snapshot flag with the decode error discarded, so an undecodable snapshot fell open and a recoveryTime restore of a system-bucket archive reached the operator to die on a pbm-level message. Treat a decode failure as "flow unknown" and refuse. Adopting a target credential only for a storage on the same bucket left the source's own reference in place when nothing matched, and opting an app into the system bucket prunes exactly that Secret while older Backups still name it. Check the reference resolves before the operator sees it and fail with a named reason instead of a missing-Secret error deep in the restore. Opting back out stops the injection but leaves the driver's storage on the live cluster until the chart re-renders; a BackupJob in that window passed the presence-only precondition, minted a CR without the delete-backup finalizer and wrote an object nothing owns into the platform bucket. Refuse to mint onto a storage whose bucket is the platform's while the flag reads false, bounded by the deadline like the other waits. Also realign the snapshot literal for gofmt, and fix two doc slips: the pbm agents start on backup.enabled alone (a declared storage is needed only to service a backup), and the opt-in cross-reference targeted the parent heading instead of its subsection. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
69f80dbdc2 |
fix(mongodb): drop the false-positive stall detector and harden the cancel
The pinned psmdb operator persists a backup's status only when the state string or error changes, so status.lastTransition freezes at the instant a dump enters running and never advances while it streams. A stall detector keyed on that field fires on every dump that outruns its window and raises the same signal a genuine wedge would, so it cannot be acted on either way. Drop it: a running dump is polled quietly and never cancelled, since this CR carries no field a driver-side timer can use to tell a slow dump from a wedged one. The deadline cancel re-read the live state but could not tell "could not read" from "not started": the helper returned "" for a nil client, a throttle, a 403 and a real absence alike, and a live ready or error state also fell through, so an unanswerable read or a just-completed CR was deleted, taking a finished archive with it via the delete-backup finalizer. The helper now returns an error, and the cancel deletes only on a positive not-started answer, deferring on anything else. The restore's point-in-time refusal keyed on the resolved credentialsSecret name, which a BackupClass can set to any Secret; record the useSystemBucket flag on the backup snapshot and key the refusal on that instead. The storage-race retry deleted the errored CR, but the lookup kept returning it while it lingered Terminating, so a fresh CR was never minted; skip a CR carrying a deletion timestamp in the lookup. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
2293c0c073 |
fix(mongodb): bound and surface the useSystemBucket backup race-recovery waits
The useSystemBucket driver injects storage onto the live cluster at BackupJob time and works around the psmdb operator resolving that storage from a cache that has not yet observed the apply. Several of those waits were unbounded or silent, and a PITR restore of a system-bucket backup reached the operator despite no oplog ever being captured on that flow: - the first-injection requeue now fails terminally once the backup deadline passes, instead of polling a cluster whose cache never reflects the storage - a state=error the operator raises because it has not yet seen the injected storage is retried (delete the CR, requeue) within the deadline rather than failing the BackupJob on the driver's own race - a backup wedged in state=running past the stall window surfaces Ready=False and a Warning once its status stops transitioning, so it no longer parks later BackupJobs in silence, and is never cancelled (that would strand the partial archive pbm keeps writing to the shared bucket) - the timeout-cancel path re-reads the CR live before deleting, so a waiting CR whose live state just turned running at the boundary is not deleted mid-dump - a failed delete-backup finalizer strip now emits a Warning naming the stuck object instead of the success-case event - a recoveryTime restore over a system-bucket backup fails with a named error rather than handing the operator a point-in-time target it cannot serve Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
0aaf5e1e05 |
docs(mongodb): condense the psmdb-backups RBAC comment so it does not ship into the ClusterRole
The delete/patch grant justification had grown to an 11-line `#` block, which
(unlike a `{{/* */}}` comment) renders verbatim into every cluster's ClusterRole
and drifts out of date as the code moves. Condense it to a short pointer at
reconcileMongoDB / cleanupMongoDBBackup and the docs, where the detail is
versioned; keep only enough to explain the non-obvious grant.
Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
|
d091c9951c |
fix(mongodb): requeue after a fresh storage injection so the operator's cache can observe it
The first BackupJob of a useSystemBucket app SSA-injected the storage and minted the operator PerconaServerMongoDBBackup in one reconcile. The psmdb operator resolves spec.backup.storages from a cached cluster read when it services that CR, and a cache that has not yet observed the apply takes the not-found branch and latches the CR at State=error — which the operator never re-drives (it returns for a terminal-state CR and watches only the Backup CR and Pods) and the driver reads as a terminal failure. Only the first job per app is exposed; later ones re-apply an already-declared storage. When the pre-apply cluster did not already declare the storage, requeue after the apply instead of minting in the same pass. The next reconcile finds the storage in this driver's own cache (a good proxy that the operator's has caught up too) and only then mints the CR — one 5s poll per app, well inside the deadline. A fake client has no informer lag, so this is pinned by asserting the first pass requeues and mints nothing rather than by reproducing the timing. Also correct the RBAC delete-grant comment: the 30m timeout branch cancels a not-yet-started operator CR on either flow, not just the useSystemBucket cleanup, so a Failed BackupJob never leaves a CR the operator would run later. And pin the wantBucket=="" restore-credential short-circuit with a no-bucket source case. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
e653a4fa5d |
fix(mongodb): correct the finalizer-prune claims, pin the pre-read release CR, and fix the retention field text
The delete-backup finalizer prunes storage only for a `ready` backup at v1.22.0; a timed-out backup never reaches `ready`, so the deadline-cancel comment no longer claims the finalizer "takes anything partially written" — nothing is written until the backup runs, which the cancel prevents. The psmdbBackupTimedOut godoc no longer says the operator "has not started" the backup, which is wrong for `requested` (the operator sets it as it dispatches); it now describes the not-yet-running states it actually gates. The pre-read CR handed to releaseMongoDBCleanup on the terminating path had no test that failed when the argument was dropped, because the existing cases either call the function directly or let the second Get succeed. Added a cleanupMongoDBBackup case whose operator-CR Get fails after the first: the release must strip from the passed object, not re-read, or the teardown wedges. The useSystemBucket field said "retention move[s] to the Plan", but a Plan has no retention field; corrected to "scheduling moves to a Plan; this flow has no retention yet" and regenerated the schema, README and mongodb-rd. Softened the "never pins a namespace in Terminating" wording in the cleanup godoc and RBAC comment: the driver only bounds its own finalizer, the operator's release-lock finalizer can still hold the CR while the operator is down. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
a89321fa8c |
fix(mongodb): keep retentionPolicy a genuine no-op, and cancel a timed-out backup so it cannot orphan an archive
Every prose surface (the useSystemBucket field, the retentionPolicy field, and backup-classes.md) tells a tenant that schedule and retentionPolicy are ignored on this flow, and cozyvalues-gen copies that into the schema, README and the dashboard. A render-time reject of a non-default retentionPolicy contradicted all of them: a tenant who followed the documented opt-in with a retentionPolicy still set got a failed render and a HelmRelease that stopped converging, pointed at the very doc that says the field does nothing. Drop the reject so retentionPolicy is inert exactly as documented, handled the same way as schedule (gated out, no error). The absence of retention on this flow is real but belongs to a follow-up, so the docs now point at the BackupRetentionPolicy design (#4192) and the teardown-GC issue (#3966), and recommend an S3 lifecycle policy until they land. Cancel a timed-out operator backup instead of only failing the BackupJob. The deadline fires for a backup the operator has not begun streaming; but `waiting` is the state pbm reports for a backup queued behind another, and the operator starts it once the slot frees — after the BackupJob is Failed and never reconciled again, writing an archive into the shared bucket that no Backup object represents. The deadline branch now deletes the operator CR before failing, which prevents that late run entirely: at v1.22.0 the operator returns for a CR with a deletion timestamp before it dispatches to pbm, so nothing is written. The delete is best-effort so it never blocks the failure. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
f8554d289d |
fix(mongodb): stop the system-bucket flow stranding archives, and reject an inert retentionPolicy
Two ways the shared cozy-backups bucket could accumulate objects nothing prunes. A dump that outran the 30-minute deadline was declared failed while pbm kept writing. The BackupJob's terminal-phase guard then never runs the branch that mints the Cozystack Backup, so the archive lands with no Backup object to represent it and no retention can reach it (the operator CR carries no ownerRef either). Leave a running dump to finish — a real dataset routinely outruns 30m, and a broken one still terminates through the operator's error/rejected state. The deadline keeps bounding the case it was written for: the operator never starts the backup (state stays "", requested or waiting), where nothing has been written and failing strands nothing. retentionPolicy is inert on this flow (the chart's tasks[].keep is gated out and no Plan-level retention exists yet), so a value a tenant set expecting it to work would grow the bucket silently. Reject a retentionPolicy changed from the chart default when useSystemBucket is on, rather than ignore it; the default still renders so an untouched install is unaffected. Also on the cleanup path: hand the terminating-namespace release the operator CR the caller already read and confirmed owned, so the finalizer strip is not gated on a second Get whose transient failure would silently drop it and leave the CR finalized in a namespace being torn down. Correct the ownership comments — the delete-backup finalizer is not one the operator never sets (it finalizes task-minted backups on the legacy flow); what keeps it unambiguous is that cleanup only ever resolves the one CR named in driverMetadata. Add the missing isolation test that a legacy app's own s3-storage is never overwritten by the injection call site, and move the strategy's storage comment into a Go-template comment so it no longer ships into every rendered Strategy CR. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
cc51e040ae |
fix(mongodb): keep the cleanup escape hatch off legacy operator CRs, and surface a stuck finalizer
The skip-artifact-cleanup annotation reaches releaseMongoDBCleanup before the ownership check, and the operator-CR delete sat outside the finalizer gate, so a legacy backup (no delete-backup finalizer) lost its PerconaServerMongoDBBackup record when a tenant set the annotation to release a stuck Backup or applied it across a namespace during teardown. That record is what a psmdb restore-by-name resolves against, and both the RBAC comment and the docs promise it is never touched. Gate every write in the release on the finalizer being present: an unowned (legacy) CR or one that cannot be read at all is left intact, and the Cozystack Backup still releases. Surface a failed finalizer strip. The release must proceed even against an unreachable CR, but the swallowed patch error left the operator CR holding percona.com/delete-backup — pinning its namespace in Terminating, the outcome the branch exists to prevent — while the Event reported the success case. Emit a FinalizerNotStripped Warning naming the object so an operator can find it. Mark the sibling backup fields (schedule, retentionPolicy, destinationPath, endpointURL, insecureSkipTLSVerify, s3AccessKey, s3SecretKey) as ignored under useSystemBucket, so a tenant editing one instead of the flag itself is not misled by a description that describes only the legacy flow; regenerate the schema, README and mongodb-rd. Pair the insecureSkipTLSVerify rendering assertion with a case that flips the platform knob on, so the two together discriminate the rendered value from a hardcoded default. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
d58322e253 |
fix(mongodb): let the system-bucket cleanup escape hatch survive an unreadable operator CR
The two give-up paths sat behind the operator-CR Get, so a Get that hard-fails the way a removed CRD (NoKindMatchError) or a revoked verb (Forbidden) does returned the error and never reached the release — leaving a Backup that keeps its cleanup finalizer and requeues forever, exactly in the situation someone sets the escape-hatch annotation to get out of. Read the skip annotation before any apiserver call (as the Redis cleanup does) so it always releases, and classify NoMatchError alongside NotFound (as the Job/Redis cleanups do) so a missing CRD reads as "gone". releaseMongoDBCleanup now takes the CR name and treats every apiserver step as best-effort, so the release is never blocked by the same unreachable CR. Also correct the docs, which promised an automatic release when "the operator is down": only a terminating namespace or the annotation release. A down operator in a live namespace keeps the Backup in Terminating as a visible signal, which is what the code does. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
542f42ef67 |
fix(mongodb): make the system-bucket cleanup teardown-safe and single-source its ownership
Four issues in the system-bucket cleanup path. Teardown wedge. The cleanup requeued forever waiting for the operator to prune, and the escape-hatch annotation released the Cozystack Backup without touching the delete-backup finalizer the driver stamps on the operator CR. So a namespace teardown, where the operator's delete-backup step cannot run (it execs against a pbm agent that is going away), left the operator CR — and the namespace — stuck in Terminating with no way out but hand-patching finalizers. Cleanup now detects a terminating namespace (as Redis/Rabbitmq do) and, on that or the annotation, strips its own finalizer and releases, leaving the object for a bucket lifecycle policy. Needs patch on perconaservermongodbbackups for the finalizer strip. Ownership was read from two places written at different times: the finalizer (stamped when the operator CR is created) and a driverMetadata flag (written when the artifact is created, a later reconcile). If the app's useSystemBucket flag flipped between them they disagreed. Read ownership from the finalizer on the one CR named in driverMetadata (the one the driver minted): resolving it by name, never by scan, keeps the operator's own cron-* task-minted backups (which do carry percona.com/delete-backup at v1.22.0) out of scope, so the finalizer on that named CR is an unambiguous single marker. Drops the driverMetadata flag. Injection error was terminal. A transient server-side apply — an apiserver hiccup, or a conflict while the operator writes the same cluster — failed the BackupJob outright. Requeue with backoff instead, bounded by the existing deadline, matching the cluster/app read errors alongside it. Injection ran before the enabled check, mutating a cluster that can never back up. Gate it on backup.enabled so a disabled cluster is left untouched. Also: reframe the restore S3 backfill as the defensive guard it is (the pinned operator always echoes status.s3, so the live path normally carries coordinates), and correct the psmdbtypes doc that still claimed the driver never patches the operator CRs. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
f27a02f1b6 |
docs(mongodb): correct the inlined-storage story and surface the TLS knob
The strategy godoc still said the storage config is NOT inlined and the strategy only names the storage — which controller-gen copies verbatim into the CRD, so kubectl explain served the new s3 field and its own denial from one object. Rewrite it (and the StorageName field doc) to describe both flows, and regenerate the CRD. Render insecureSkipTLSVerify on the default MongoDB strategy: the field is plumbed end to end into the driver-injected storage entry and backed by a platform knob, but the template set only forcePathStyle, pinning it to false so a self-signed external S3 that Redis backups honour would fail MongoDB x509 verification. Pin it in the rendering test alongside the other s3 coordinates. Fix the useSystemBucket field description: angle brackets rendered as empty HTML tags in the README table and the dashboard schema, so wrap the path in backticks (as the postgres field already does) and note that schedule, retentionPolicy and PITR are ignored on this flow. Regenerate the chart schema, README and mongodb-rd. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
1d05dafb2e |
fix(mongodb): own and prune the system-bucket archive, and heal the restore read path
Two correctness gaps on the useSystemBucket flow. Retention. Gating spec.backup.tasks out of the chart also dropped the keep that psmdb task retention used, and an on-demand PerconaServerMongoDBBackup is not covered by task retention, so nothing pruned the shared cozy-backups bucket — a retention-swept Plan grew it unboundedly for every tenant at once. Own the archive the way the Redis/Rabbitmq drivers own theirs: stamp the operator's percona.com/delete-backup finalizer onto the backups minted on this flow, and on Cozystack Backup deletion delete that operator CR and wait for the finalizer to free the object before releasing. A legacy backup (tenant's own bucket) keeps the pre-existing hands-off no-op. Adds the delete verb on perconaservermongodbbackups and a skip-artifact-cleanup escape hatch for a wedged operator, matching the sibling drivers. Also drops the unused update verb from the psmdb cluster RBAC (server-side apply is a patch, not an update). Restore. resolveMongoDBBackupSource returned early off the live operator Backup CR whenever it reported a destination, copying status.s3 as-is. On this flow the operator can echo a destination with status.s3 empty (the storage was injected, not declared), and the target has no declared storage to resolve from either, so the restore stranded with no coordinates. Backfill a missing live S3 block from the snapshot the BackupJob recorded. Also correct the precondition godoc: the pbm agents start on spec.backup.enabled alone; the declared storage is required by the backup CR's resolution, not by the agents. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
47fe7c9630 |
test(mongodb): pin the restore credential swap and document the opt-out
The restore-time decision to re-point a backup's credentialsSecret at the
target cluster lived inline in reconcileMongoDBRestore with no test, so the
same-bucket guard that keeps a cross-flow restore from authenticating
against the wrong bucket could be dropped unnoticed. Extract it into
mongodbRestoreCredentialsSecret and cover all four
{legacy,system}x{legacy,system} combinations; neutralising the bucket
match now reddens the suite. The projected-secret skip is kept as well: a
target storage sharing the platform bucket but naming a different secret is
better restored with the platform-projected cozy-backups-creds than with
that secret.
Document the reverse toggle: flipping useSystemBucket true->false leaves
the driver-injected s3-storage on the live PerconaServerMongoDB under the
driver's field manager, which the chart does not own and never prunes. It
is inert once backups are disabled; the operator must delete it by hand to
return the release to the legacy flow.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
|
9545bee233 |
fix(mongodb): correct system-bucket prose, pin strategy rendering, make restore credential swap flow-aware
Several comments and the useSystemBucket field description had drifted from
what the code does.
The strategy header still said the $bucketName gate was "not because it
needs the value" and that the sibling strategies needed the bucket name
while this one only asked for ordering, but the strategy now renders
bucket: {{ $bucketName }} for the injected coordinates, so it needs the
value; the precondition comment claimed the s3-storage entry exists on
backup.enabled=true, which is only the legacy flow. The rendering-test
comment carried the same "needs no bucket coordinates" claim. The
useSystemBucket field description said injection happens "at first BackupJob
time" (it now runs on every BackupJob) and that backups scope to
<namespace>/<release> (the prefix templates the application name, not the
release), and it ran ~700 chars of implementation detail into the CRD that
backup-classes.md already carries; trimmed and corrected, then regenerated
schema/README/types/cozyrds. Same <release> slip fixed in the driver
comment and the docs.
Nothing asserted the new strategy rendering. Added the MongoDB row to the
endpoint-form suite (s3.endpointUrl, scheme preserved both ways) and field
assertions to the rendering suite — bucket, endpointUrl, region,
credentialsSecret, forcePathStyle, and prefix, which must reach the CR as a
literal go-template string so the driver can template it per-application;
a stray brace would empty it with every doc-count test still green.
The restore credential swap broke across flows. It replaced the source
credentialsSecret with whatever the target declared, comparing no buckets,
so a system-bucket backup restored into a legacy target kept the platform
bucket against the tenant's own -s3-creds, and a legacy backup restored
into a system-bucket target got cozy-backups-creds against the tenant
bucket - both AccessDenied. Skip the swap when the source already names the
platform-projected cozy-backups-creds (it outlives the source app, nothing
to repair), and only adopt a target credential whose storage is on the
same bucket as the archive. psmdbStorageS3 now reads .s3.bucket alongside
.s3.credentialsSecret; TestPsmdbTargetCredentialsSecret covers the match,
the cross-bucket refusal, and the empty-bucket no-op.
Also drop the post-apply re-Get: the typed SSA Patch decodes the server's
merged object back into the passed struct, so the precondition reads the
injected storage from it instead of a follow-up Get that hit the informer
cache and briefly reported Ready=False on the happy path.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
|
4ea4888685 |
fix(mongodb): key system-bucket injection off the app flag, not live storage
The MongoDB backup driver decided what to inject and what to record from the live cluster's storage set rather than from the app's opt-in flag. That had three consequences. The snapshot fallback fired for legacy apps too. Its gate was `snap.S3 == nil && rendered.S3 != nil`, but the coordinates moved onto the shared cozy-default strategy, so rendered.S3 is now non-nil for every MongoDB app. A legacy backup written to the tenant's own bucket got its snapshot rewritten to the platform bucket, sending restore to the wrong place. Gate the fallback on the app's backup.useSystemBucket flag instead, threaded through createMongoDBBackupArtifact, and pin the legacy case. The injected storage never caught up with the strategy. Injection was gated on the named storage being absent, so it ran once per cluster and froze the coordinates at the first backup; a later endpoint/region change or bucket re-provision was a silent no-op, and Helm could not repair it because the useSystemBucket chart renders no storages. Key injection off the app flag and re-apply on every BackupJob (the prefix is deterministic, so this never splits the archive the way CNPG's serverName would). Add UseSystemBucket to mongodbapp.MongoDBSpec so a legacy app is protected by the flag, not by whatever happens to be on the cluster. The docs said the opposite of what the driver does. backup-classes.md claimed "the MongoDB driver cannot inject its S3 target per-backup"; correct it and add an opt-in-to-the-system-bucket section that names which values stop being read and that scheduled backups + PITR must move to a Plan. Fix the endpoint table row, the tuning reference, the RBAC comment that said the driver never patches, the strategy header/inline comments, and the example comment. Also assert notExists on spec.backup.pitr in the helm test, and lift the duplicated cozy-backups-creds default into a const. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
|
c036efaa5a |
test(mongodb): cover system-bucket storage injection gate and builder
Extract the two untested contracts from applyMongoDBSystemStorage into
pure functions and unit-test both, mirroring the CNPG driver's
buildBarmanObjectStore split:
- shouldInjectMongoDBSystemStorage decides whether the driver injects the
system-bucket storage; the tests pin both branches, including the one
that leaves a legacy app's manual bucket untouched.
- buildMongoDBSystemStorageEntry builds the {type:s3,s3:{...}} entry,
defaulting credentialsSecret and omitting forcePathStyle when nil.
Also add a helm-unittest for the restore.yaml guard rejecting
bootstrap.enabled together with backup.useSystemBucket.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
|
730592f323 |
feat(mongodb): support the system-bucket backup flow (useSystemBucket)
MongoDB was the only cozy-default app with no useSystemBucket flow: its backup storage was always static on the app CR (bucket/endpoint/creds from backup.* values + a chart-emitted <release>-s3-creds Secret), so a tenant could not back up to the platform system bucket out of the box the way postgres/clickhouse/etc. do. PBM takes bucket/endpointUrl/region/prefix as static CR fields (not secret refs) and the platform bucket name/endpoint are only known at BackupJob time, so - unlike ClickHouse's runtime env secretKeyRef - the chart cannot render them. The fix mirrors the CNPG driver: the chart adds a useSystemBucket flag that keeps spec.backup.enabled (so the pbm agents run) but omits the static storages/tasks/pitr and the creds Secret; the cozy-default MongoDB strategy carries the coordinates (endpointUrl already scheme'd by the endpoint helper, credentialsSecret cozy-backups-creds); and the psmdb driver SSA-injects spec.backup.storages[s3-storage] onto the live PerconaServerMongoDB at BackupJob time under its own field manager with ForceOwnership, so a Flux re-render never reverts it. Injection is gated on the storage being absent, so a legacy app that ships its own static storage is left untouched. The backup snapshot falls back to the strategy coordinates when the operator status echo is empty, so the restore path rebuilds backupSource (credentialsSecret cozy-backups-creds is projected into the restore namespace before dispatch). RBAC grows patch/update on perconaservermongodbs for the injection. A bootstrap.enabled + useSystemBucket render guard mirrors postgres. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
|
938595afe3 |
fix(keycloak): validate the realm list wherever it is set
The list was read only inside the adminHost branch, so a value the chart would refuse — master, a scalar, a name that splits the path — sat in the values unnoticed until someone turned the admin hostname on, and the refusal then landed on the change of authentication topology rather than on the edit that caused it. Build the list on every render and let the branches use the result. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
13a4029b8b |
fix(keycloak): count the realms the route budget rejects
The over-budget message derived the realm count from the path list, which also carries the two static paths, so it named one realm more than the deployment has — the one number an operator acts on. Keep the realm list in its own template and count it there, and derive the budget from the static paths rather than restating 31. The shape guard also let a falsy scalar through, so `exposedRealms: false` reached range and failed with a Go template error instead of the message that says what to write. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
714ced9bab |
fix(keycloak): refuse more realms than a route rule can match
A Gateway API route rule takes 64 path matches and each published realm takes two of them, so past 31 realms the apiserver rejects the whole HTTPRoute and the login hostname stops answering. Fail while rendering, naming the parameter to shorten, instead of leaving the limit in a comment. The Ingress path has no such cap, so the check lives with the route. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
c96fe3d3ef |
fix(platform): refuse master as the OIDC realm name
The realm named here is where keycloak-configure creates every platform client and the cluster-admin role, and where dashboard users sign in. Naming master puts all of that inside the realm that holds the Keycloak super-admin, and publishes its login page wherever the platform realm is published — and the guard belongs where the value is set, since the Keycloak chart only reads it once a separate admin hostname is configured. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
e87232b822 |
fix(keycloak): always publish the platform realm and never master
exposedRealms replaced the published realm instead of adding to it, so a deployment naming only its back-office realm took its own login off the public route: every component's issuer URL points at the platform realm, and they would all have met a 404. Publish the platform realm unconditionally and deduplicate, which is also what the parameter's description promised. Refuse master in either position rather than warning about it in a comment — publishing it is exactly what the realm enumeration exists to prevent. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
0b8296d689 |
fix(dashboard): release the held modifier before the first character, not after some
The window this had been measuring started before the paste did. noVNC arms its timer on the keydown, and the modifier is delivered once — by that timer or by the next keyup, whichever comes first, and the V keyup is not intercepted. Both land within 100 ms of the keydown, which at the default pace is somewhere inside the opening releases. Counting from the end of those releases therefore started the clock again and let three characters through while the guest still held Ctrl, where an m is Return. The second release now waits out whatever is left of the window, measured from the keydown itself, and happens before any character goes in. The test that was written for this pinned the wrong placement: it handed the modifier over after the first character, later than noVNC can, and the fake's method was the immediate one under a second name. The fake now delivers on its own schedule, partway through the opening releases, so a test cannot choose the moment. The component's own typing pace had no test at all once every existing one passed its own value, so setting the default to zero left the suite green. One test renders without the prop and holds it. The lifecycle tests key on characters typed rather than on a count of sleeps, which is what let them drift past their code when the opening releases changed shape. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
64229b154e |
fix(keycloak): bound each published realm path to a path segment
ingress-nginx renders `pathType: Prefix` as a plain nginx prefix location instead of the element-wise match the Ingress API specifies, so a lone /realms/cozy also served /realms/cozy-admin — a realm the chart claims is unpublished. Publish each realm as an exact path plus a trailing-slash prefix, which bounds the match on every controller and on Gateway API. While here, reject a scalar in place of the realm list instead of failing inside range, and point a malformed platform realm name at the parameter that carries it rather than at exposedRealms. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
5149e67dd7 |
feat(keycloak): publish only the named realms on the login host
The narrowed login route carried a bare /realms prefix, so it also published realm master: its token endpoint accepted super-admin credentials from the internet. Enumerate the published realms instead — the platform realm by default, ingress.exposedRealms when a deployment serves more than one. Behavior changes only where ingress.adminHost is already set. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
|
cd91627c78 |
feat(marketplace): register a tapped repository's apps on connect (#4308)
## Problem Connecting a marketplace repository materialized its `PackageSource` but created no `Package`, so its `ApplicationDefinition`s never rendered and its apps never appeared in the dashboard catalog until a manual `cozypkg add`. The dashboard cannot drive that step: the package list it shows is computed from the `ApplicationDefinition`s, which do not exist yet, a chicken-and-egg that leaves a connected repository unbrowsable. ## Fix The tap materializer creates a tap-managed registration `Package` for each materialized `PackageSource` that has a `default` variant, the same way the platform ships a `Package` per built-in app. Its install-marked components (the `ApplicationDefinition` registrations) deploy on connect, so the apps become browsable; the app components carry no `install` block and are still instantiated per user resource, so registration deploys no user workload. Privileged install components are not installed without confirmation. The check is at the install site: the Package reconciler skips a privileged install component of a Package that still carries the marketplace-tap label (a tap auto-registration the operator has not confirmed) and reports `Ready=False PrivilegedNotConfirmed`, and `resolvePrivilegedNamespaces` gates the PodSecurity label on the same predicate. `cozypkg add --allow-privileged` sheds the label (an explicit ownership handover), which is the confirmation; a platform Package never carries the label. Putting the check where the install happens closes the window a materializer-side guard would race, so the earlier install-safety de-register machinery is removed. Two further best-effort layers close the paths a lingering artifact or release could take when a source revision flips a previously benign component to privileged: `cleanupOrphanedHelmReleases` treats an unconfirmed privileged component as not-desired and removes a HelmRelease left from an earlier benign revision, and the PackageSource reconciler withholds a privileged component's `ExternalArtifact` until confirmed (re-running on confirmation through a Package watch), so the helm-controller has no privileged content to upgrade to. A component flipped to privileged in a new revision can still be materialized by source-watcher before any reconciler observes the flip; closing that race fully requires decoupling the catalog registration from install, tracked in #4359. `cozypkg untap` and the dashboard disconnect remove the registration `Package` too, so the apps de-register (its registration HelmReleases and their `ApplicationDefinition`s are garbage-collected); running instances are left in place but unmanaged until re-tapped, the same outcome as `cozypkg del`. Teardown deletes the Package under a UID + ResourceVersion precondition with a re-classify-on-conflict retry, so it never clobbers a Package a concurrent `cozypkg add` handover has just confirmed. `cozypkg add` re-opens a tap auto-registration for variant selection and hands it over to the user. ## Verification - The install-site privileged gate is covered by a Package-reconciler test (a managed registration with a privileged default variant creates no privileged HelmRelease and does not raise the namespace; a confirmed Package installs) and is mutation-verified. - The two best-effort layers each have a mutation-verified test: cleanup of a HelmRelease left when a benign component flips to privileged, and withholding of a privileged component's `ExternalArtifact` while the registration is unconfirmed (a non-tap platform source is never gated). - The `cozypkg add` handover is covered by a test asserting the tap label, the source annotation and the PackageSource ownerRef are shed and the variant pinned. - The teardown delete precondition is covered by a test in which a handover lands between the read and the delete; the precondition makes the delete conflict and the retry re-classifies and spares the user's Package. Earlier end-to-end on a dev cluster (predates the install-site refactor and the layers above) with a real public demo repository `oci://docker.io/999669/cozy-gitea:v1.0.0` confirmed the connect→auto-register→instance→disconnect flow: `cozypkg tap` alone registered the app within ~10s; a `Gitea` instance reached Ready and served `/api/healthz`; `cozypkg untap` de-registered it and left the running instance. etcd healthy, `cozystack-platform` untouched. ```release-note Tapping a marketplace repository now registers its applications in the catalog on connect, without a manual `cozypkg add`. ``` |
||
|
|
92b8df076c |
fix(dashboard): a paste could still type chords on a Windows host
Windows is the platform where noVNC holds the modifier back: the first Ctrl keydown tells the guest nothing, arms AltGr detection and starts a 100 ms timer, and Ctrl reaches the guest when that timer fires. Taking the shortcut on the capture phase hides the V from noVNC, so the timer is what delivers — after the paste released a modifier the guest had not yet been told about. Everything typed between the two goes in as chords. The release is now sent a second time, once the pacing has covered that window. It happens once rather than per character, and it leaves Shift alone, because that one is held deliberately for capitals. Two tests had drifted past the code they are named for. Both dropped the session on the first sleep, which since the prelude lands before anything is pressed, so they held whatever the typing loop did: removing the session check from the loop left them green. They drop it after a character now, and a third covers the closing release, which no test reached at all. The fake learns the Windows path it could not express, and decodes function keysyms: Enter is 0xff0d, and treating it as a Unicode offset produced a negative code point, so the first test to paste a newline would have thrown instead of failing an assertion. The typing pace is injectable, so tests no longer run at the speed of a real keyboard and stop crowding their neighbours out on timeouts. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
6209a0b8d3 |
refactor(dashboard): code-split the serial console at the route
The tab pulled xterm through a dynamic import inside its own effect, which split the chunk but left the stylesheet on a fragile path: the dev server failed to fetch it once after a dependency rebuild, and the tab came up with the terminal missing. Splitting at the route instead is what the YAML editor next door already does. The imports go back to plain ones, React.lazy gives the tab its own chunk, and the stylesheet travels with it rather than through an import() of a CSS file. The entry chunk every dashboard page loads ends up smaller than before this branch: 935.91 KB at the merge base, 895.99 KB here, with the tab and xterm in a 347 KB chunk fetched when a console is opened. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |