mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-28 23:57:08 +09:00
feat/postgres-credentials
570
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Timofei Larkin
|
66a20d9a0a |
docs(postgres): describe minted passwords
The README told users to read a password from <release>-credentials, which the chart no longer renders, the createRole text named the init job as the place a planted role is refused, and the restore notes described the init job applying chart-generated passwords. They now describe the mint, the pre-upgrade check, how a restored copy gets its passwords, and why a downgrade past the conversion is unsupported. Assisted-by: LLM Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Timofei Larkin
|
e7d5724de5 |
fix(monitoring-agents): indent extra outputs like the rest
The extra outputs were indented again after the block scalar had been dedented, six columns deeper than the VictoriaLogs outputs, and fluent-bit refuses a config whose indentation is inconsistent, which stops every output, not only the added one. The test now pins the indent instead of matching any whitespace. The credentials and specschema packages join the unit-test target, so their tests run in CI, and the audit-log page names the field the pipeline writes. Assisted-by: LLM Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Timofei Larkin
|
108b51df3d |
feat(api): serve credentials with mint and revoke, and seed accounts
The Credential view lists the account Secrets of the namespace by label and skips any no live HelmRelease owns, so no kind is served until its definition carries a credentials block. A mint replaces the stored value in one conditional write keyed on the Secret's resourceVersion and returns the plaintext once; a revoke discards it. Both record an Event on the application, which needs create on events. The application registry seeds an account Secret per declared account before it writes the HelmRelease, so the chart can reference them from the first render, prunes the Secrets of removed accounts after the write, and refuses to delete an application whose release is suspended, since Flux skips the uninstall and would leave the database running with the Secrets gone. The ApplicationDefinition reconciler computes the per-account valuesFrom entries with the function the API uses, so a release that predates the block gets them on its next reconcile. Schema defaulting moves to a package both can import. A definition declaring a format or an account source that is not built leaves its kind unserved rather than served with passwords the API cannot issue. Tenant roles gain get and list on credentials at view and create on mint and revoke at admin and for the tenant ServiceAccount. Attribution comes from the kube-apiserver audit log, so the operations docs get a policy for distributions that ship none, and monitoring-agents a value for an extra fluent-bit output so a SIEM can receive the log without dropping the VictoriaLogs outputs. Assisted-by: LLM Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Andrey Kolkov
|
58b4343e5a |
ci(hack): route examples/backups/{vmi,vmdisk} edits to the vminstance suite
The two backup round-trips run the examples/backups/{vmi,vmdisk} scripts as their
harness, but select-e2e.sh mapped an examples/backups/<dir> edit to a suite only
when <dir> was itself a suite name. The dirs are vmi/vmdisk while the suite is
vminstance, so a harness-only edit selected no suite and the round-trip that runs
it never ran. Translate both dirs to vminstance, matching the vm-disk-application
entry in src_to_suites, and pin it with a bats case. The suite runs the vmi demo
with SKIP_RESTORE_TO_COPY=1, so 07-restore-to-copy.sh stays docs-only: selecting
the suite for it would pass without executing it. Both round-trips write to
the in-cluster seaweedfs-s3 endpoint, so a seaweedfs change selects the
vminstance suite as well. Also correct the controller-
owned-teardown note in docs/agents/e2e-testing.md (only Backup/RestoreJob
finalizers reach Velero; a BackupJob has none).
The two round-trips are also the only Tests that run a Velero backup and
restore, so the velero PackageSource maps to the vminstance suite; it
otherwise maps to a velero suite that does not exist, and a Velero change
ran neither round-trip.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
Andrey Kolkov
|
0c2f131192 |
feat(backups): back up Valkey with the Redis strategy
Valkey runs on the same spotahome RedisFailover operator as Redis and speaks the same replication protocol and RDB format, so it needs no strategy kind of its own. The only difference the backup Job sees is the chart's resource-name prefix: valkey-<app> instead of redis-<app> for the sentinel Service and the auth Secret. The Redis driver now accepts applicationRef kind Valkey as well as Redis and hands the template .Release.Prefix, derived from the app's kind, which cozy-default-redis uses for every name it builds. A kind with no known prefix fails the render instead of producing an empty one. cozy-default routes apps.cozystack.io/Valkey to cozy-default-redis. A restore must target an app of the same kind as the backup's source: replaying a Redis dump into Valkey, or the reverse, is not something the strategy can promise across engine releases. The shipped template reads the prefix as `.Release.Prefix | default "redis-"`: during an upgrade the previous controller, which has no Prefix in its render context and serves only Redis, renders the new CR, and a bare reference would print "<no value>" into the names. The redis:8 image already loads RDB v11, which both shipped Valkey versions (7.2, 8.1) write. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
Aleksei Sviridkin
|
7c46ffff12 |
fix(backupstrategy-controller): run backup clients on a package-owned image (#4756)
## What this PR does The Altinity (ClickHouse) and RabbitMQ backup strategy Pods ran `platform-migrations`: a 349 MB image carrying `kubectl`, `helm`, `git`, `cozyhr` and `etcd-migrate`, whose `ENTRYPOINT` is `run-migrations.sh`. Both pins were stamped from `packages/core/platform/Makefile`, so any migration change re-digested the backup clients too, and only the `command:` override kept the Pods from running platform migrations (#3420). - New `packages/system/backupstrategy-controller/images/backup-http-client/`: `alpine:3.24` (digest-pinned, the same base the tree already uses) + `curl` + `jq` + `ca-certificates`, `USER 65532:65532`, no `ENTRYPOINT`. Everything is installed at build time, so the Pods still never run `apk add` and keep working air-gapped. - New `image-backup-http-client` target in the package's own `Makefile`, chained after `image-redis-backup` like the existing ones so only one `yq -i` writes `values.yaml` at a time. It stamps `chBackupClientImage` and `rabbitmqBackupClientImage` with the same ref. - Removed the cross-package stamps from `packages/core/platform/Makefile`, the WORKAROUND/TODO comments in the chart and the RabbitMQ example, and the stale "live violation" line in `docs/agents/image-refs.md`. - `hack/image-pin-consistency.bats`: dropped the `platform-migrations`-specific test, whose subject no longer exists. The general check (no repository pinned at more than one digest, computed from `hack/promote-retag.sh --dry-run`) still covers the two keys sharing one image. **Why one image for both strategies rather than `ch-backup-client` as the issue named it.** The RabbitMQ strategy had the same reuse and the same cross-package stamp; leaving it would have kept the stamp alive. Its script needs `curl --aws-sigv4` + `ca-certificates`, a subset of the ClickHouse one, so one image serves both. The two values keys stay separate so either strategy can move to its own image later. **Initial pin.** The image does not exist in ghcr until CI builds it, so both keys are committed as `backup-http-client:v1.6.0` without a digest, the same way `redis-backup` was introduced. PR CI rebuilds touched packages and overlays the fresh refs for e2e; release preparation's `make image` stamps tag+digest. Image size: 6.9 MB vs 349 MB. Verified locally: - `docker buildx build` for `linux/amd64,linux/arm64`; in the image as uid 65532: curl 8.22 with `--aws-sigv4`, jq 1.8.2, `date`, `seq`, writes to `/tmp` (the RabbitMQ script's only write), TLS verification against the bundled CA store. - `helm unittest` (23/23), `helm template` renders both strategies with `backup-http-client` and no `platform-migrations` reference. - `hack/cozytest.sh hack/image-pin-consistency.bats` and `hack/build-matrix_test.bats` pass. Not yet run end to end on a cluster; the e2e backup round-trips will exercise it. Fixes #3420 ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map against the diff: it touches `packages/system/backupstrategy-controller` (new image, Makefile target, values comments/pins, template comments), `packages/core/platform/Makefile` (stamp removal only, no `values.yaml` key change), `hack/image-pin-consistency.bats`, `docs/agents/image-refs.md` and a comment in `examples/backups/rabbitmq`. No app package, schema, CRD, variant, asset, platform values key or `hack/package.mk` change. The website does not list first-party image repositories (searched for `platform-migrations` and `redis-backup`), so no mirror list needs the new one. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(backupstrategy-controller): the ClickHouse (Altinity) and RabbitMQ backup strategy Pods now run a dedicated `backup-http-client` image (alpine + curl + jq + ca-certificates, non-root, ~7 MB) instead of the 349 MB `platform-migrations` image. Air-gapped mirrors need the new `ghcr.io/cozystack/cozystack/backup-http-client` repository. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Altinity and RabbitMQ backups now use a dedicated backup HTTP client image, configured separately from the platform migrations image. * The client image includes the tools required for backup operations and runs with a non-root identity. * Both backup configurations receive the same digest-pinned image reference when the image is built. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
ebceef4105 |
ci(retention): sweep packages named by the published nightly tree (#4826)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The Artifact Retention workflow has failed every day since mid-August, and it never pruned anything. It lists the org's container packages with `GET /orgs/cozystack/packages?package_type=container`, and that endpoint answers `400 Invalid argument` to the app installation token the job uses. Others hit the same in actions/github-script#449 and cli/cli#9606. An old `|| true` used to hide it. This is item A19 in #3278. The per-package endpoints accept the same token: a probe run confirmed it, and the dry-run below read every package with it. So the job no longer lists the org. The new `hack/retention-packages.sh` takes the package names from the published `cozystack-packages:nightly` tree, reading refs through the shared `hack/lib/image-refs.sh`. It also adds `cozy-installer` and `cozystack-nocloud`, which the nightly publishes outside the tree. The committed tree can't be the source: it misses images whose digest only the build pins, and those still get dated nightly tags every day. Failures now turn the run red instead of passing silently: - if the name list is empty, the nightly sweep is skipped and the run fails, but abandoned promotion candidates are still handled, because their protection doesn't depend on that list - if a package's versions can't be read, that package is skipped and the run fails at the end - if a DELETE is refused, `hack/retention-delete.sh` logs an error with the HTTP status, goes on with the remaining versions and fails the run (before, this was only a warning) - if names resolve but none of the packages is linked to this repository, nothing gets swept, so the run fails too, after the promotion sweep has run (that sweep itself is skipped only when candidate protection can't be resolved) Promotion-candidate protection itself is unchanged. The contract test that pinned the old org listing now pins the new behaviour, and both new scripts have bats tests. Known gap: a component that disappears from the nightly tree drops off the list, so its old dated nightlies are never pruned. Today none of the packages outside the list has more dated nightlies than the keep window. Dry-run on this branch is green: https://github.com/cozystack/cozystack/actions/runs/38075159900. It resolves 59 packages, skips `cozystack/cozystack-ui` with the warning, and would delete 418 old nightly versions plus one abandoned promotion candidate. Deleting needs a different token. The app token lists versions, but a one-version apply run (https://github.com/cozystack/cozystack/actions/runs/37939905343) got `404 Package not found` on DELETE for a version it had just listed. The same DELETE with this repository's `GITHUB_TOKEN` and the job's `packages: write` worked in a second run (https://github.com/cozystack/cozystack/actions/runs/37941084456), with no change to package settings. So `hack/retention-delete.sh` now deletes with `GITHUB_TOKEN`, and listing stays on the app token. That delete was shown on a package linked to this repository. One of the swept packages, `cozystack/cozystack-ui`, is linked to another repository, and I didn't try deleting there. Such a package is skipped before its versions are read and named in a warning, so it can't turn every apply run red. The follow-up for it is in #3278. `RETENTION_APPLY` stays off, so the scheduled run is a green dry-run until someone turns it on. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> N/A ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ci(retention): the GHCR retention workflow takes package names from the published nightly tree instead of the org package listing, which rejects the app token, and fails the run whenever names, versions or deletions cannot be resolved. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Nightly package retention now targets packages referenced by the published nightly tree and keeps the 14 newest eligible versions per package. * Promotion candidates older than 30 days are eligible for pruning only when their digests are not protected by qualifying release pull requests or branch tips. * **Bug Fixes** * Packages linked to another repository are skipped, and sweep failures are reported without preventing other eligible packages from being processed. * **Documentation** * Clarified which nightly versions and promotion candidates qualify for retention. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
2db18a0c86 |
test(e2e): record ZFS datasets and LINSTOR volumes in cozyreport (#4657)
## What this PR does The kubernetes suites' LINSTOR cleanup check now compares dataset names per node, and prints the leftover names when it fails (#4651). This PR puts the rest of what a triager needs into `cozyreport.tgz` on every run, green or red: the size of each dataset, its `origin` and `creation`, the snapshots, the pool's pending frees, and the LINSTOR volume list. What `hack/cozyreport.sh` now collects: - `linstor/zfs-pools.txt`: the zpool behind each ZFS storage pool, per node, taken from `StorDriver/StorPoolName` in `linstor --machine-readable sp l`. - `linstor/zfs/<node>.txt`: that node's satellite runs `zpool get -p freeing,leaked,allocated,free` and `zfs list -p -t all -o name,used,avail,refer,origin,creation -r` over the node's pools. - `linstor/volumes.txt`: `linstor v l`, next to the existing `resources.txt`. Sizes are exact bytes, so the sizes of leftover datasets can be summed against the pool root's `avail`. That `avail` can differ from `storage-pools.txt` without anything leaking: on a node with only thick pools, `linstor sp l` serves the free capacity cached at the satellite's last report. A dataset LINSTOR could not destroy because snapshots depend on it is renamed to `_deleted_<name>` in the same pool, so a search for leftovers has to look for that prefix as well as `pvc-`. A pool that `zpool get` or `zfs list` cannot open does not cost the listing of the other pools. `zpool get`'s message and exit status go into the listing, and `zfs list` lists the rest. A node LINSTOR names no ZFS pool for gets a note instead of a listing, because `zfs list -r` with no pool argument lists every pool on the machine. A satellite whose `spec.nodeName` cannot be read gets a note that says so. All reads go through the bounded exec reader, like the other LINSTOR reads, and the time budget in the script header counts them. The second commit changes how that reader marks a read that wrote output and then exited non-zero without hitting the timeout. It used to append `TRUNCATED ... part way through, not because the command finished`. But `kubectl exec` passes the remote command's status through, and `zfs list` over one missing pool prints a complete listing of the others and exits 1. The marker now names the exit status and says the status can't tell an error after complete output from a cut stream. This changes the marker for every LINSTOR exec read. The timeout marker stays as it was. Tested with `hack/cozytest.sh hack/cozyreport.bats`. The new tests run the controller and satellite shells against stub `linstor`, `zpool` and `zfs`, with the real `jq`. They cover a failed and an unparsable storage-pool read, a pool row with a missing, empty or blank backing name, a non-ZFS pool, a pool name with a space, a pool that `zpool` and `zfs` cannot open, a failing `zfs`, and an unreadable node name. A test goes red if any of these break: the filter that keeps non-ZFS pools out, the filter that drops a missing, empty or blank pool name, the jq exit status, the `zpool` fallthrough, the node-name note, or the reader's new wording. Refs #4653 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note test(e2e): cozyreport records the ZFS datasets of each node's LINSTOR pools in exact bytes, and the LINSTOR volume list, on every e2e run. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Collection reports now include LINSTOR volumes, ZFS pool capacity and usage, and per-node ZFS datasets and snapshots with exact-byte details. Dataset listings are scoped to the ZFS pools LINSTOR reports for each node. * Reports include notes or warnings when node, pool, or dataset information is missing or cannot be collected. * **Documentation** * Clarified storage report contents, artifact markers, bounded reads, and how unsupported data, incomplete results, collection limits, and per-object warnings are reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
770f515105 |
docs(backupstrategy-controller): state the first-commit pin exception
docs/agents/image-refs.md forbade committing a bare tag without saying how a new image's first pin can carry a digest it does not have yet. Record the exception the backup client keys rely on: the bare tag is a placeholder the producing `make image` replaces, and an unbuilt tree pulls a tag that does not exist. Drop the securitygroup-controller example, which is pinned by digest now. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
647407545d |
fix(backupstrategy-controller): run backup clients on a package-owned image
The Altinity and RabbitMQ strategy Pods ran platform-migrations, a 349 MB image carrying kubectl, helm and etcd-migrate whose ENTRYPOINT is the migration runner, and whose pins were stamped from packages/core/platform so every migration change moved the backup clients too. Build images/backup-http-client (alpine + curl + jq + ca-certificates, non-root) from this package's Makefile and stamp both client keys from it. The cross-package stamp in packages/core/platform/Makefile and the platform-migrations-specific check in hack/image-pin-consistency.bats go away; the general one-digest-per-repository check still covers the two keys sharing one image. Fixes #3420 Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Aleksei Sviridkin
|
76a4af95c0 |
docs(monitoring): correct the release name the OIDC comment, guide and test fixture encode (#3468)
## What this PR does
The Grafana OIDC `clientId`, audience-scope name, client-secret name and
users-Job name are all derived from the Monitoring release name. In a
running cluster that release is always `monitoring-system`: the tenant
chart's outer HelmRelease `monitoring` points at `extra/monitoring`,
whose `templates/helmrelease.yaml` creates the inner HelmRelease `{{
.Release.Name }}-system` that carries `system/monitoring` and renders
OIDC. There is no release named `monitoring` that renders OIDC.
Three places encoded a release named `monitoring` (i.e.
`<ns>-monitoring`, where a cluster renders `<ns>-monitoring-system`),
per #3338:
- `packages/system/monitoring/templates/_helpers.tpl` header comment —
both example lines were wrong. The per-tenant release is
`monitoring-system`, and the platform Grafana lives in the `tenant-root`
namespace, not `cozy-monitoring`. (`cozy-monitoring` only hosts the
ExternalName services that redirect to the platform stack the root
tenant hosts in `tenant-root` — see
`packages/system/cozystack-basics/templates/monitoring-external-services.yaml`.)
- `docs/oidc-grafana.md` — placed the platform release in
`cozy-monitoring`; it lives in `tenant-root`. It also named the
per-tenant release `monitoring`; that is `monitoring-system` too, so
both instances now read `monitoring-system` and differ only by
namespace.
- `packages/system/monitoring/tests/oidc_test.yaml` — the suite pinned
`release.name: monitoring`, so its clientId / audience-scope /
client-secret / users-Job assertions pinned names no cluster renders.
Both fixtures are set to `monitoring-system` and the derived expected
values updated to what a cluster actually renders.
Nothing changes at runtime — the templates are self-consistent whatever
the release is called; this only aligns the descriptions and the test
fixture with reality.
Out of scope here (from the issue, left for a maintainer decision):
- Confirming/removing the apparently-dead `cozystack.monitoring`
PackageSource in `packages/core/platform/sources/monitoring.yaml` (issue
item 3) — removing a source is a call I can't safely make.
- Re-checking Detail B of #3259 against the corrected topology (issue
item 4).
Verification, rebased on current main: `helm unittest -f
'tests/oidc_test.yaml' packages/system/monitoring` — 39/39 pass; the
whole monitoring chart passes as well. After setting the fixtures I
updated only the assertions helm-unittest reported as newly diverging.
Disclosure: prepared with AI assistance; I reviewed the change and
verified the release-name topology against the source files cited above.
### Screenshots
N/A — no UI changes.
### Downstream repositories
Walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file. The three changed paths are a `system/monitoring`
chart `_helpers.tpl` comment, the repo-internal `docs/oidc-grafana.md`,
and a chart test fixture. None match a listed trigger: no change under
`packages/apps/` or `packages/extra/`, no
`packages/core/platform/values.yaml`, and no
variant/bundle/component/release-asset/Talos/tooling change. So no
downstream repository is mechanically affected. (The website's own OIDC
page referenced in cozystack/website#597 is a separate artifact the
reporter is already engaged with, outside this repo's trigger map.)
- [x] No downstream repository is affected by this change
### Release note
```release-note
docs(monitoring): correct the release name (`monitoring-system`) and platform namespace (`tenant-root`) encoded in the Grafana OIDC helper comment, the OIDC guide, and the chart test fixture.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Clarified that both platform and tenant Grafana instances use the
`monitoring-system` release name. The platform instance runs in
`tenant-root`, while tenant instances run in their respective tenant
namespaces.
- Updated the OIDC integration guidance to reflect the platform
namespace and tenant release naming.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
4295930989 |
feat(dashboard): show categories brought by tapped repositories in the sidebars (#4741)
## What this PR does The marketplace and console sidebars render a fixed category list, `["IaaS", "PaaS", "NaaS"]`, and drop every other `spec.dashboard.category`. A repository tapped with `cozypkg` that brings its own category is reachable only at the bottom of the full application list, and its installed applications get no console section at all. Both sidebars now follow `groupByCategory`, the order the marketplace page already uses: IaaS, PaaS, NaaS first, then any other category alphabetically. `Administration` stays in the Admin portal, as before. Unknown categories get the generic grid icon. The note in `docs/agents/contributing.md` that described the hardcoded list is removed, since it no longer holds. Tests: two new cases in `sidebar-sections.test.tsx` (a tapped `Paleocomputing` category appears after the built-in ones in both sidebars, `Administration` stays out). Both fail on the previous code. The console suite (448 tests), `pnpm typecheck` and `eslint` on the changed files pass. ### Screenshots Same cluster and tenant, a repository tapped that brings the `Paleocomputing` category. Before, marketplace: the category is only reachable at the bottom of the full list.  After, marketplace: the category gets its own filter.  Before, console: no section for the tapped applications.  After, console: installed applications of that category get their own section.  ### Downstream repositories Walked the trigger map against the diff: it touches only the console sidebar code and a note in `docs/agents/contributing.md`. No package, values schema, CRD, installer value or `ApplicationDefinition` field changes. The website page on `ApplicationDefinition` does not restrict `spec.dashboard.category` values, and `external-apps-example` uses `PaaS`, which keeps working as before. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(dashboard): categories brought by tapped repositories now appear in the marketplace and console sidebars, after IaaS, PaaS and NaaS, instead of only in the full application list. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Marketplace and Console sidebars now show application categories beyond the built-in options, including custom categories. Categories appear in the order provided by the application grouping. * The Administration category remains excluded from both sidebars. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
youdie006
|
b7708fb285 |
docs(monitoring): correct the release name the OIDC comment, guide and test fixture encode
The naming block in _helpers.tpl, the guide's opening paragraph and the oidc test fixture all describe the per-tenant Grafana as the release `monitoring`. Both instances are released as `monitoring-system` by the monitoring application and differ only by namespace: the platform one in `tenant-root`, a per-tenant one in each tenant namespace. Assisted-by: LLM Signed-off-by: youdie006 <youdie006@users.noreply.github.com> |
||
|
Andrey Kolkov
|
e4e991e837 |
fix(backups): skip the CNPG restore purge only for this restore's Cluster
The in-place CNPG restore skips deleting the live Cluster when it carries bootstrap.recovery and was created after the RestoreJob started, taking that as "this restore's own purge and re-render produced it". The timestamp does not say which restore rendered the Cluster. A Cluster deleted while the restore waits for the WAL archive is re-rendered by the still-active HelmRelease from the previous restore's values and is created after StartedAt too; the restore then skipped its purge and reported Succeeded on the previous restore's data. The restore patch already sets bootstrap.newServerName to restoredServerName(cluster, RestoreJob UID). The chart now annotates the Cluster with it as postgres.cozystack.io/restored-server-name, whether or not it renders the barman plugin that also carries the value, and the purge is skipped only when the annotation names this RestoreJob. The mark lives on the rendered Cluster rather than in RestoreJob status, so it still covers the failed status write the skip exists for. A Cluster rendered by a chart that predates the annotation keeps the creation-time check. Fixes #3322 Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Aleksei Sviridkin
|
2e5fd56954 |
fix(e2e): stop calling a failed exec read after output a cut
kubectl exec passes the remote command's status through, so a non-zero exit after output does not say whether the stream broke or the command finished with an error. zfs list is the common case: it lists every dataset it can open, skips one it cannot and exits 1, so a complete listing got a TRUNCATED marker and a note claiming it held only what was streamed before the exit. The in-file marker now names the exit status and says the status cannot separate the two, and the note beside it says the same. The timeout branch keeps its own wording. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d195a56793 |
test(e2e): record ZFS datasets and LINSTOR volumes in cozyreport
The report carries the LINSTOR resource list and the pool totals, but nothing that names the datasets holding a pool's space, so a leftover zvol, a snapshot, a clone and pool metadata cannot be told apart from the artifact alone. Each satellite now lists every dataset of the ZFS pools LINSTOR records for its node, snapshots included, with the pool's pending frees. Sizes are exact bytes with avail, so the sizes of leftover datasets can be summed against the pool root's avail; the rounded form loses a sub-GiB difference on a large pool. That avail can legitimately differ from `linstor sp l`, which serves a cached free capacity on nodes with only thick pools. The pool names come from LINSTOR rather than from a name pattern. One pool that zpool cannot open does not cost the listing of the others, and a satellite whose node name cannot be read says so instead of pointing at the pool map. The controller's volume list is recorded next to the resource list. Everything runs on every job, so a red listing can be read against a green one, and goes through the bounded exec reader like the other LINSTOR reads. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
da1f1cc387 |
ci(retention): delete with the workflow token
The app installation token lists package versions but its DELETE of a listed version is answered with 404, while the repository's GITHUB_TOKEN with packages: write deleted the same version without any change to package settings. Deletes therefore go through GITHUB_TOKEN; listing keeps the app token. That deletion was shown on a package linked to this repository. A package linked elsewhere is not attempted: it is skipped before its versions are read and named in a warning, so it cannot turn every apply run red. A failed read of the link still fails the run, and so does a sweep in which no resolved package is linked, since that sweep prunes nothing. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f1c0f6f3f6 |
ci(retention): sweep packages named by the published tree
The org container-package listing answers 400 to the app installation token, so the job has never reached its sweep. Take the names from the published nightly tree via hack/retention-packages.sh and keep calling only the per-package versions endpoints, which accept that token. An unresolved name list now fails the run: the nightly publishes into these packages daily, so an empty list cannot mean there is nothing to prune. It skips only the nightly sweep. The promotion sweep reads one fixed package and is kept safe by the protected set, which the name list does not feed, so it still runs. A versions read that fails, in either sweep, is reported with the API error and fails the run once the rest is swept, instead of aborting the step through set -e. A failed candidate read leaves no ids behind. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
be2387323b |
feat(gateway)!: reserve a passthrough listener's name by the spec alone (#4638)
## What this PR does This keeps the TLS-passthrough services on port 443 of the tenant Gateway and makes the passthrough rules hold on Cilium 1.20.2. It is stacked on #3343 and has to land with it (stack #4717). On 1.20, upstream Cilium marks an HTTPS-terminate listener and a TLS-passthrough listener on one port `Conflicted` when their hostnames overlap, and drops both from Envoy. That took `*.<apex>` and the `api`, `vm-exportproxy` and `cdi-uploadproxy` passthrough listeners down together on every tenant Gateway in `dns01` and `existingSecret` mode (#3912). #3343 now ships a patched operator that conflicts the pair only when the hostnames are equal, so the layout stays as it is on main: one Gateway, one address, everything on 443. What remains is the equal-hostname case and the cleanup that 1.20 allows. Under `http01`, a name a passthrough listener answers is now reserved by the spec alone. Before, the controller dropped the terminate listener for such a name only once a TLSRoute was attached to the passthrough listener and its backend Service resolved, which copied how Cilium builds filter chains. Without a working TLSRoute, an HTTPRoute on `api.<apex>` kept its terminate listener next to `tls-api`, and on 1.20 that equal-hostname pair takes both listeners down. Now such an HTTPRoute gets no listener and no certificate, TLSRoute or not. If all its hostnames are reserved it reads `Accepted=False` with `NoMatchingListenerHostname`. If it also claims a free hostname it stays `Accepted` and is served there. The backend lookup, its Service and ReferenceGrant watches and their RBAC rules are gone. Other changes: - `tlsPassthroughListeners` is accepted under `dns01` and `existingSecret`. The old refusal relied on 1.19.5 serving all ports from one Envoy listener; 1.20 splits ports whose SNI overlaps. - A TenantGateway reads `Ready=False` with reason `GatewayClassUnsupported` when its GatewayClass lists features in `status.supportedFeatures` but not `TLSRoute`, while it renders passthrough listeners. A class that lists nothing is not judged. Cilium 1.20.2 lists `TLSRoute`. - Comments, the README and one error message no longer describe Cilium 1.19.5 as the shipped release. - The gateway chart writes `tlsPassthroughServices` on the publishing tenant only, behind the same `gateway.enabled` switch and `publishing.exposedServices` filter as the platform TLSRoutes. Before, every tenant Gateway got the three passthrough listeners with nothing routed to them, and the reservation above would have taken those names from a child tenant's own HTTPRoutes. The uniform `allowedRoutes.kinds` on port 443 from #3087 stays. The Cilium bug behind it is fixed since 1.19.6, but several tests pin it, so removing it is a separate cleanup. ### Upgrade notes Only the publishing tenant keeps the `api`, `vm-exportproxy` and `cdi-uploadproxy` passthrough listeners. That is `tenant-root` unless `publishing.ingressName` says otherwise, and only for names in `publishing.exposedServices` while `gateway.enabled` is on. Other tenants lose these listeners on upgrade. Nothing shipped was attached to them, so a child tenant's own HTTPRoute on `api.<child-apex>` keeps its listener and certificate. Two narrow cases do change. On the publishing tenant under `http01`, an HTTPRoute on an exposed passthrough name loses its listener and certificate when the platform TLSRoute for that name forwards nowhere (the owning package is not installed, or its backends do not resolve). A TLSRoute attached by hand to a child tenant's `tls-*` listener loses its parent. To keep that listener, merge-patch `tlsPassthroughServices` into the tenant's `TenantGateway` after the upgrade. From then on the chart does not render the key there, and later upgrades leave the patch alone. `dns01` and `existingSecret` do not change: the wildcard listener still covers such a name, and the passthrough listener answers it by its exact name. Tested locally: Go tests for the controller, `go vet`, helm-unittest for `extra/gateway` and `cozystack-controller`, `make generate` and `make manifests` with no diff. The gateway e2e scenario checks that a child tenant renders no passthrough listener. It then declares `tls-api` and a passthrough listener on a native port on that tenant, checks that the tenant stays Ready with no route behind either, and checks the reservation against `tls-api`. Unit tests catch a regression in either change. In e2e, dropping the reservation turns the gateway suite red, and dropping the chart scoping does not, because CI never sets `gateway.enabled` (#4846). The controller was not run on a live cluster; the operator patch it depends on was, see #3343. ### Downstream repositories - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/issues/725 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(gateway)!: the tenant Gateway renders the api, vm-exportproxy and cdi-uploadproxy passthrough listeners only on the publishing tenant, for the names publishing.exposedServices lists while gateway.enabled is on. Under certMode http01 a name a TLS-passthrough listener answers no longer gets an HTTPS listener on 443, with or without a TLSRoute behind it. On the publishing tenant, an HTTPRoute on such a name whose TLSRoute forwards nowhere loses its listener and certificate. A TLSRoute attached by hand to a tls-* listener of another tenant loses its parent. An HTTPRoute with some reserved and some free hostnames used to be rejected whole; it is now Accepted and its free hostnames get a listener and a certificate. tlsPassthroughListeners is accepted under dns01 and existingSecret. A TenantGateway whose GatewayClass does not list TLSRoute in its supported features reads Ready=False while it renders passthrough listeners. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * TLS passthrough listeners are supported with HTTP-01, DNS-01, and existing-secret certificate modes. Edge-terminated Gateways remain unsupported. * Declaring a passthrough hostname reserves it from HTTPS termination and certificate creation, even when no route is attached. * Passthrough listeners are rendered only for exposed services on the enabled publishing Gateway. * **Bug Fixes** * Gateways that declare passthrough listeners but explicitly lack TLSRoute support are now reported as not ready. * Routes targeting a hostname served by a passthrough listener on a different Gateway are correctly withdrawn. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
ad3f15d4f3 |
feat(gateway)!: reserve a passthrough listener's name by the spec alone
The TenantGateway controller withheld the HTTPS-terminate listener for a name a passthrough listener answers only once a TLSRoute was attached there and resolved a backend, mirroring how Cilium builds its filter chains. That kept a served endpoint up on Cilium 1.19.5, but it tied every route's status to one implementation's translator and needed Service and ReferenceGrant watches to land the deferred decision. The shipped operator accepts a wildcard terminate listener beside a narrower passthrough listener on 443 and keeps conflicting every other overlap. A terminate listener for the passthrough name itself is not narrower, so on 443 the rule now protects the dataplane outright; on a native port Gateway API would serve the pair, and the rule holds as a policy that keeps a platform name with its passthrough listener. Reading it off the spec makes the outcome the same on every implementation. Under http01 an HTTPRoute claiming such a name gets no listener and no certificate whether or not a TLSRoute is attached. A route whose every hostname is reserved reads Accepted=False with NoMatchingListenerHostname; one that also claims a free hostname stays Accepted and is served there, as Gateway API accepts a route whose hostnames intersect at least one listener. dns01 and existingSecret are unchanged: their wildcard listener still covers such a name, and the passthrough listener answers it by its exact server name. The chart writes the passthrough services on the publishing tenant alone, so the names reserved by default are the ones the platform publishes there through its own TLSRoutes. The shape an upgrade changes is an HTTPRoute on such a name whose platform TLSRoute forwards nowhere, because it never attached (the package owning the service is not installed) or because none of its backends resolve: that route kept a terminate listener and now gets none. The backend resolution, its watches and the RBAC rules that served them go away. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
bdca165f4c |
feat(gateway): report a GatewayClass that does not serve TLSRoute
A GatewayClass lists the features it serves in status.supportedFeatures, and TLS passthrough is TLSRoute. On a class that leaves it out the passthrough listeners are accepted and serve nothing, so the TenantGateway now reads Ready=False/GatewayClassUnsupported naming the class instead of Ready. A class that lists no features at all is not judged: implementations predating the field leave it empty. The class writes that list after it is accepted and may change it when its controller is upgraded, so a GatewayClass event requeues the TenantGateways on it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
957397ed42 |
ci(retention): derive swept package names from the published tree
GitHub refuses to list org container packages to an app installation token, so the retention workflow cannot enumerate what to sweep. Name the packages from the published nightly packages tree instead: it is the tree the nightly mirror copied images for, so it covers every component image that receives dated nightly tags, including those digest-pinned only by the build and therefore absent from the committed tree. A tree that does not reference the packages artifact is refused rather than read as a shorter list. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
da28ea7ee8 |
ci(e2e): run nightly and release-tag e2e on the container lane
Nightly and the rc gate still booted the management nodes as QEMU guests, the substrate the merge-gating lanes left behind. On it the tenant Kubernetes suites have missed their node-join deadline on most nights since 2026-10-02, each failure spends the whole deadline, and the job reaches its timeout before the alphabetically later suites run. Over the same days those suites passed on every PR run that selected them, where the nodes are Talos containers and a tenant worker sits one virtualization level shallower. Both lanes now use the PR lanes' container substrate. Nightly's e2e no longer waits for the nocloud disk, which it does not boot; the disk is still built and published. A manual e2e-tag dispatch against a tag that predates the container lane fails early and names the ref to dispatch from instead. DRBD, the replicated StorageClass and the Cozystack Talos node image lose their last e2e coverage with this. The docs and comments that pointed at nightly and the rc gate as that coverage now say so. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
387760b94e |
test(tests): run unit suites with Bats and retain POSIX coverage (#3849)
## What this PR does This runs the unit suite with Bats and keeps a separate `dash` pass for tests that source production shell helpers. Unit files are discovered from the filesystem, and each loads the audited helper that restores `set -u` in test bodies without changing subprocess shell options. CI pins Bats 1.14.0, keeps JUnit reports, and runs Bats for docs-only changes too. Code changes also run Helm, Go, controller and POSIX compatibility tests. Per-file Make targets and the `--keep-going` sub-make are removed because one `bats -j` run goes through all files anyway, so a red file doesn't stop the others, and `hack/bats-unit-keep-going.bats` checks this with real bats. `-k` stays in the workflow because without it a red Bats file keeps `test-controllers` from starting. Local tests require bats-core 1.5 or newer. GNU parallel is optional, and the complete local hook runs with `pre-commit run bats-unit-tests --hook-stage manual --all-files`. Live-cluster suites keep their existing runner. ### Validation `make unit-tests test-controllers -j4 -k --output-sync=target BATS_JOBS=4 BATS_POSIX_SHELL=dash` passed, including every discovered Bats file and the POSIX pass. Mutation checks reject omitted files, disabled or exported nounset, missing CI steps and missing or excessive POSIX coverage. ### Downstream repositories The developer guide update is tracked separately in [cozystack/website](https://github.com/cozystack/website/issues/717). CCP has no caller of the changed targets. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - documentation follow-up: [issue](https://github.com/cozystack/website/issues/717) - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Testing & Reliability** * Bats unit tests run as a dedicated suite, with optional parallel execution and JUnit reports. * Pull request checks run Bats tests for documentation-only changes, and unit and controller tests for code changes. * POSIX-shell compatibility tests are included in the standard unit-test run and continue after individual failures. * Test reports are uploaded even when checks fail. * Added a manual local test hook for running Bats and POSIX compatibility tests. * Shared strict setup is applied across Bats unit tests. * **Documentation** * Updated testing guidance with current commands, runner compatibility, and CI coverage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
8c1a78dc9b |
feat(operator): default system namespace memory via LimitRange (#3494)
## What this PR does
System components now get a default container memory limit, so their pod
cgroups carry a `memory.max` and drop out of the Talos userspace OOM
handler's victim set. `cozystack-operator` maintains a LimitRange named
`cozystack-system-defaults` in every system namespace it reconciles.
The Talos handler (v1.12+) scores cgroups with `memory_max.hasValue() ?
0.0 : {Besteffort: 1.0, Burstable: 0.5, ...}[class] * memory_current`
and drops everything that scores zero. Only a memory limit sets
`memory.max`, so only a limit takes a pod out of the candidate set. A
request just moves a pod from BestEffort to Burstable, which under the
default `strictCgroupClassOrdering: true` still means killed second.
`PriorityClass` does nothing here, because the handler reads cgroupfs
and never talks to the apiserver. Victim selection is also unrelated to
the trigger, so a tenant workload thrashing against its own limit can
drive node memory pressure while a system component with no limit is
what gets killed.
An audit of a running cluster found around 70 workloads across ~30
packages with at least one container missing a memory limit. Per-chart
resources would not cover that set, and would not keep covering it as
packages are added. So the LimitRange is written by
`reconcileNamespaces`, which already creates these namespaces and
already tells system from tenant. It also covers init containers, which
#3495 can't reach: one of them without a limit leaves the pod cgroup
without `memory.max`, whatever the main containers declare.
Tenant namespaces are skipped. Where the tenant chart's own
`tenant-range-limits` exists (only when the operator configured
`resourceQuotas`), a second LimitRange next to it would leave the result
to the order LimitRanger iterates them. Where it doesn't exist, a tenant
namespace is not the place for the platform to impose a default.
### Why the default is 32Gi
The handler grants immunity for `memory.max` existing at all, whatever
its value. So the ceiling doesn't need to be fitted, and 32Gi is high
enough that nothing plausibly reaches it. A ceiling low enough to bind
would turn a rare pressure-driven kill into a deterministic one for any
component whose real usage nobody measured. This repo already has one
such case, recorded in the `vertical-pod-autoscaler` values, where a
110Mi limit OOM-killed the updater and failed HelmReleases with it.
Scheduling is unaffected, since pods are scheduled against requests and
the paired default request is 32Mi. An operator who wants the ceiling to
bind can lower it, per component in that component's chart or fleet-wide
with the knob, once real usage is measured.
### When the default is withheld
A default below a container's own request makes that pod impossible to
admit, so the operator checks before it applies. It scans Deployments,
StatefulSets, DaemonSets, CronJobs, Jobs and Pods in the namespace, init
containers included, for a container that requests more than the
configured limit and declares no limit of its own. If it finds one, it
withdraws the managed LimitRange and records a hold in the
`operator.cozystack.io/system-memory-state` annotation on the Namespace.
The hold does not clear by itself when the blocker disappears, because
an empty scan looks the same as a pod between deletion and recreation.
Lowering a previously applied limit also creates a hold, even with no
visible blocker, because LimitRanger writes the defaulted limit onto
admitted pods and the scan can no longer tell which containers declare
one. A cluster administrator releases the hold by copying the
`<hold>:<target>` token from the state into the
`operator.cozystack.io/system-memory-ack` annotation. The token is used
once and only for that hold and target. If a blocker is still visible,
the token is rejected and the hold stays. Package reconciles are
scheduled at least every five minutes, so an acknowledgement takes
effect on a quiet cluster too. The reason and the token also appear in
the operator log under `withholding the default container memory limit`.
### Namespace another LimitRange already governs
If a system namespace already has a LimitRange that says anything about
memory, at the Container or the Pod scope, the operator writes nothing
there and withdraws what it wrote before. A second default leaves the
effective value to iteration order. A foreign `max` below this default,
a `min` above the default request or a `maxLimitRequestRatio` rejects
pods outright, which takes out a whole namespace. At the Pod scope the
collision can't be decided in advance, because it depends on how many
containers a future pod has. A LimitRange that bounds only cpu or
storage coexists normally. A same-named LimitRange without the
operator's managed-by label is treated as foreign and left alone.
This matters most in `kube-system`, which is in scope because
`cozystack-scheduler` installs there, and where a distribution's or an
administrator's own LimitRange is likeliest to sit.
### Disabling
Setting `cozystackOperator.systemNamespaceMemoryLimit` to `0` disables
the feature. On the next reconcile the operator removes every LimitRange
it owns cluster-wide, found by the managed-by label. While the feature
is on, it also removes its LimitRange from namespaces no active Package
targets any more. The safety state on the Namespace stays in both cases.
### Docs
`docs/operations/system-memory-limits.md` landed in #3496 ahead of this
code, so main describes a `4Gi` default and installer values that do
nothing yet. The page is corrected to the 32Gi default and now describes
withholding, the acknowledgement, and the foreign LimitRange rule.
### Not in this PR
linstor policy. #3824 extends the controller's existing exemption from
limits to the satellite, and a linstor limit here would contradict it,
so linstor stays with that PR.
The scan reads six workload kinds through `APIReader`, uncached, on
every reconcile of every system namespace. That cost is tracked in
#3977, since the system namespace set isn't known when the cache is
built and the right fix depends on a reconcile-rate measurement nobody
has taken.
### Verification
`go vet` and `go test` pass for `cmd/cozystack-operator` and
`internal/operator`, and `helm unittest` passes on the installer chart
for the knobs unset, set, and the limit at `0`. The reconciler tests
cover withholding on an oversized request, the hold surviving the
blocker's removal, the hold on a lowered limit, acknowledgement and
token rejection, every memory field of a foreign LimitRange at both
scopes, ownership by label, the disabled sweep, and a deleting Package
not creating policy.
The withhold design has not been run on a live cluster yet. An earlier
design of this PR was, and that run does not carry over.
### Screenshots
Not a UI change.
### Downstream repositories
I walked the trigger map against the diff.
`packages/core/installer/values.yaml` gains two keys, but the
`ansible-cozystack` trigger is for a required key or a rename of one the
role passes, and these are new, optional and default to `""`, so the
role is unaffected. Nothing else in the map is reached by the Go
changes, the operator template or the page.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
feat(operator): system namespaces now get a default container memory limit through a LimitRange maintained by cozystack-operator, which keeps system components out of the Talos userspace OOM handler's victim set. Tune it with cozystackOperator.systemNamespaceMemoryLimit (default 32Gi) and cozystackOperator.systemNamespaceMemoryRequest (default 32Mi); set the limit to 0 to disable it. The default is far above any real working set on purpose, because the handler grants immunity for a memory.max existing at all, not for its value. A namespace with a container that requests more than the limit and declares no limit of its own gets no default, and stays without one until a cluster administrator acknowledges the hold recorded on the Namespace (see docs/operations/system-memory-limits.md); lowering a previously applied limit needs the same acknowledgement. A namespace where another LimitRange already sets anything about memory is left alone. A container that declares its own memory limit is never changed.
```
|
||
|
Aleksei Sviridkin
|
3e078b79a9 |
ci(e2e): refuse a new fork image before the build starts (#4767)
## What this PR does A fork PR that adds a first-party image cannot pass the fork e2e lane, and today it finds that out only after the whole build. `e2e-fork.yaml` pushes the fork's image archives only under names the base tree already exports, so a new image is refused at the push with `image '<name>' is not in the base-tree allowlist`. Nothing the contributor does on the PR changes that. This PR keeps the push allowlist as strict as it is and moves the failure to the start of the run. On fork PRs, `plan` in `pull-requests.yaml` now runs `hack/fork-new-images.sh`. The script derives the image names of the PR tree and of its base parent with the same `make -n` derivation the push step uses. If the PR tree exports a name the base does not, it fails before any build, names the images and says what has to happen: a maintainer lands the image on the base branch first, or the change runs from a branch in this repository. The check runs from the fork's own tree, so a fork can remove it. That is fine, because it only saves an honest contributor a wasted build. The push allowlist in `e2e-fork.yaml` is still the boundary. That workflow cannot call the script, since it runs from the default branch over an untrusted checkout. So `hack/fork-new-images.bats` pins the two derivations to each other, and also covers the script's behaviour and the fork-only gating of the step. I replayed the script against the merge commit of #4291 and its base parent. It fails with `cluster-api-provider-kubevirt`, the same name the push step refused there. The workflow_run lane itself cannot be exercised locally. This does not lift the limitation itself: a fork still cannot get a green run for a new image. Part of #4329. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note ci(e2e): a fork PR that adds a new first-party image now fails in the plan job, naming the image, instead of after the whole build at the image push. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Fork pull requests that add image names not present in the base branch are now rejected during planning, with the new names reported before builds begin. * **Documentation** * Updated guidance describes the early check and clarifies how it relates to the image allowlist used during publishing. * **Tests** * Added coverage for new, unchanged, reduced, and empty image sets, along with checks that the pull request workflow applies the check only to fork submissions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
cc942fed68 |
test(tests): run unit suites with Bats and retain POSIX coverage
The shell translator rewrites test bodies and loses Bats reporting. The unit lane needs complete TAP results and JUnit reports without changing how live-cluster suites run. Sourced production helpers still need a dash pass, and unit test bodies need nounset without exporting shell options into subprocesses. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
b1c84e9c06 |
feat(platform, tests): select E2E packages and declare controller dependencies (#4022)
## What this PR does E2E installs the whole platform on a fresh cluster for every PR, whatever the PR touches, so install time and flake surface go to packages the selected suites never use. This PR makes the E2E install take only what the selected suites need. `hack/select-install.sh --disabled <suites>` generates `bundles.disabledPackages` for fresh E2E installs using an ordinary platform variant. It keeps suite owners, runtime requirements and the install baseline, then follows their forward `PackageSource` dependencies. The baseline follows the unconditional waits in `hack/e2e-install-cozystack.bats` and stays seeded directly even where the graph reaches the same packages. It also keeps platform services that no suite owns but the rest of the platform uses: `multus`, `cozystack-scheduler`, `linstor-scheduler`, `flux-plunger`, `cozy-proxy` and `ouroboros` (the Kubernetes suites assert its hairpin NAT). Backup suites also keep `backupstrategy-controller` and its dependencies, and the gateway suite keeps `kubeovn-webhook`. The E2E job now selects the suites before it installs, and passes the complement to the platform Package as `disabledPackages`. The `full-e2e` label and an empty selection keep the full install, and the nightly and tag lanes pass no list. The platform sources also change: `mongodb-application` now depends on `mongodb-operator`, and `seaweedfs-application` on `objectstorage-controller`. These dependencies apply to regular platform installs and upgrades too. The new `mongodb-operator` edge also reaches reverse test selection: a change under `packages/system/mongodb-operator` now selects the `mongodb` suite instead of falling back to the full suite. Backup APIs and the gateway webhook remain suite-specific install requirements, so they add no edges. The disable list governs fresh installs because existing Packages carry `helm.sh/resource-policy: keep`. Optional packages still need `bundles.enabledPackages`. Packages that no suite owns and the baseline does not keep stay uninstalled on purpose, also when test selection answers with every suite (monitoring exporters like `goldpinger` and `kubeovn-plunger`, apps without a suite, provider-specific operators). Nothing in the run looks at them, so installing them only adds a readiness wait, and a change to such a package gets no install check from this gate. ### Measured One run per variant on a 32 vCPU / 128 GB VM, container lane, the CI artifacts of this PR, `postgres` suite selected, 55 packages disabled: - HelmReleases 111 to 77, pods 219 to 197 - pod requests 14.8 CPU / 37.7 GiB to 13.5 CPU / 35.7 GiB - host memory peak during install 33.4 GB to 30.2 GB - `install-cozystack` 26m33s in both runs - `postgres` suite 25m50s to 25m17s, passes on the reduced install Install time did not change. The removed packages are mostly operators and apps without instances, and what the install waits on (SeaweedFS, LINSTOR, CDI, the Cluster API providers, Keycloak) stays in the baseline. With one run each, differences of a minute or two are noise. The selection, the install closure and the workflow wiring are covered by bats tests (`hack/select-install_test.bats`, `hack/select-e2e_test.bats`, `hack/platform-packages_test.bats`) and by a helm-unittest suite in the platform chart. ### Screenshots Not a UI change. ### Downstream repositories The new dependencies ship in the platform chart. No coordinated changes are needed in downstream repositories. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(platform, tests): E2E installs only the packages the selected suites need. MongoDB now depends on its operator, and SeaweedFS on the object storage controller. ``` |
||
|
Aleksei Sviridkin
|
798094e725 |
feat(release): add backport-audit, a pre-release backport triage tool (#3527)
## What this PR does
This adds `cmd/backport-audit`, a maintainer tool that checks, before a
release or an rc is cut, whether everything labelled for backport
actually landed on the release branch. It prints the URLs of the PRs
that did not and exits non-zero, so the cut can gate on it.
`docs/release.md` and `docs/agents/releasing.md` now send the patch-cut
check through it.
The docs it replaces inventoried backports with `gh pr list --search
"is:merged label:kind/backport"`. That lists every PR ever labelled, not
what reached the branch. Answering the real question by hand means
reading the branch history per PR, and that is how backports go missing
quietly.
```console
$ go build ./cmd/backport-audit
$ ./backport-audit release-1.4
=== release-1.4 === 30 candidate PRs: MISSING=5 dropped=4 backported=21
MISSING -- labelled for this line, no trace of it here (5):
https://github.com/cozystack/cozystack/pull/2704
#2704 fix(info): use root-host for Keycloak OIDC issuer URL in tenant kubeconfig
label=kind/backport author=myasnikovdaniil merged=2026-05-26 -- no backport PR, nothing on branch
```
A `kind/backport` label does not name a branch. `backport.yaml` resolves
the target when the PR merges, so the same label means `release-1.5` on
a PR merged in June and `release-1.6` on one merged in August. The audit
reproduces that rule instead of trusting the label text. The rule
changed once, on 2026-08-03, when
|
||
|
Myasnikov Daniil
|
5b585d6eda |
fix(operator): persist holds for unsafe memory defaults
An admitted Pod loses the distinction between explicit and defaulted limits. Its disappearance cannot prove a lower default is safe for its replacement. Keep that decision on the Namespace until an administrator acknowledges the particular target, without waiving current blockers or foreign policy ownership. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
30ecc41070 |
fix(operator): close memory policy safety gaps
Preserve foreign same-name objects and keep create-time SSA non-force. Only ignore failed pods controlled by terminal Jobs. Make the failure-containment test inject Apply errors for real. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
6a521f0c68 |
fix(operator): sweep memory defaults when disabled
List LimitRanges by the ownership label once per Package reconcile when the feature is disabled, and remove managed objects even in namespaces outside the current Package target set. Skip the per-namespace path while disabled and keep cleanup failures non-fatal to namespace reconciliation. Update the flag help, installer guidance, and operations documentation to describe the global cleanup boundary. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
f6dcb69b61 |
refactor(operator): withhold instead of raising, and drop the grace period
The raise-and-grace machinery cannot work: LimitRanger writes the raised limit onto the very container that justified the raise, so at that container's next admission it carries a limit, the scan skips containers that have one, and the raise has erased its own evidence. The ceiling then drops on schedule and the workload is rejected permanently. Any such machinery approximates admission logic ahead of admission, and that is incomplete by construction: a LimitRange default is one number for a whole namespace and cannot mean "the configured limit, or this container's request, whichever is larger". Only per-pod conditional defaulting expresses that, which is a mutating webhook and not this. The justification for the machinery had also expired. It was designed when the default was 4Gi, where withholding was self-perpetuating for VPA-managed namespaces. At 32Gi nothing in the tree approaches the ceiling -- the largest static memory request is 2Gi -- and VPA sets requests and limits together, so those containers carry their own limit and were never blockers. So a namespace holding a container that requests more than the configured limit with no limit of its own is now left with no default at all, and anything written there earlier is withdrawn. It fails in the safe direction, which is the whole argument: a namespace with no memory.max is a namespace behaving as it did in every release before this, while a default below a request is a namespace whose workload cannot be admitted. It needs no stored state and no timer, because withholding is not a decision that has to survive between reconciles -- once the oversized request is gone the default is applied again on the next pass. Gone with it: the grace period, the clear-since annotation and its clamp, the injected clock, containerMemoryDefault, raisedCeiling, and fourteen tests. The scan stays, because deciding whether to apply or withhold is what it was always for. The foreign-LimitRange ownership guard, the disable path and the 32Gi default are untouched. The residual sharp edge is stated rather than machinery: a workload whose only evidence is a live Pod, built from a custom resource with no template this scan can read, can still be missed if the scan runs while that Pod is absent, and applying the default then rejects its recreation. At 32Gi that is unreachable in this tree. It becomes reachable if an operator lowers the knob below such a workload's request, which the flag help, values.yaml and the operations page now say plainly. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
81c12bc953 |
fix(operator): withhold on any foreign memory policy, not two fields
The coexistence guard treated a namespace as already-governed only when another LimitRange carried a container memory default or max, and said so in its comment. That is not the set of fields that collide with what this reconciler writes. A container min above the default request rejects the pod, and so does a maxLimitRequestRatio, since a 32Gi default against a 32Mi default request is a ratio of 1024:1 and no cap allows it. So the namespace-wide admission failure the guard exists to prevent stayed reachable through fields the guard did not read, and kube-system is where it is likeliest, this feature having deliberately brought it in scope. Enumerating the colliding fields does not fix it, because at the Pod scope the collision is not decidable here. Whether a per-container default of 32Gi breaches a pod-wide max depends on how many containers the pod has, and the pods in question do not exist yet: any arithmetic about them is a guess, and a wrong guess costs a namespace its entire pod admission. So the rule is ownership rather than arithmetic. If anything else in the namespace has an opinion about memory, at either scope and in any field, the namespace is not this reconciler's to default. Narrow still on the resource, because nothing here writes cpu or storage and a LimitRange bounding those must keep coexisting. This is the rule the tenant-namespace exclusion already rests on. The pod scan no longer skips Failed pods, only Succeeded. A pod that ran to completion is never recreated from its spec, but an eviction or a node crash leaves a Failed carcass holding the same oversized request that its controller will recreate, and that carcass is frequently the only surviving evidence the namespace needs a raised ceiling, since the shape the scan looks for cannot be admitted once the namespace has settled. Dropping it lowered the ceiling under a workload that was coming back, which is what the grace period exists to prevent. A grace stamp dated ahead of the clock is clamped to now. Unclamped it held the ceiling for the skew plus the grace period, silently, so an hour of skew bought ninety minutes instead of thirty. Clamping never shortens the wait, it only stops it being extended. The two flag defaults are named constants the flag registration and its test both read. The test previously duplicated the literal and still asserted 4Gi after the default moved to 32Gi, so it misinformed a reader about the shipped value and pinned nothing. The disable claim is narrowed to what the code guarantees. "Removes any previously created one" was universal, but the reconciler only runs for namespaces an active Package still targets, so a namespace whose Package was removed keeps its LimitRange. The flag help, values.yaml and the operations page now say that, and name the managed-by label for finding leftovers by hand. The test asserting a Pod-scoped memory max should be ignored is gone, since under the ownership rule a Pod-scoped memory bound withholds. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
59555c895e |
fix(hack): retain controllers and APIs for selective installs
Tenant HelmReleases need a shard reconciler, and the root SeaweedFS release creates COSI resources even when no backup suite is selected. Without those controllers, a fresh reduced install cannot become ready. The Kubernetes suites create worker pools through KubernetesNodes, and MongoDB renders an operator-owned CR. Selecting only the cluster API or application chart drops resources those suites require. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
df889a6419 |
docs(platform): state when the ceiling drop actually lands
Two imprecisions in the page written alongside the grace period. It said a lowered limit reaches previously-raised namespaces "within" the grace period, which is backwards: the grace period is a floor, not a ceiling, and since reconciles are event-driven the drop lands on the first one after it elapses. And the withdraw path was described only as leaving the namespace alone, without saying that it hands the namespace to the administrator's policy in full, including any admission rejection that policy implies -- which is theirs, and would have happened without Cozystack, but is worth naming where an operator is being told the operator stopped writing. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
769bce59a2 |
docs(platform): say that lowering the knob waits out the grace period too
The grace period was documented as a property of a raised ceiling coming back down, which is where it started, but it applies to any namespace defaulting more than the configured limit -- and a namespace sitting at a previously higher configured value is indistinguishable, on the object, from one the operator raised. So lowering the knob reaches those namespaces within the grace period rather than on the next reconcile, which is worth stating before an operator concludes the setting did not take. The page now also shows how to read the countdown off the objects. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
fe792369e5 |
docs(platform): bring the memory-limit page in line with the operator
docs/operations/system-memory-limits.md merged ahead of the code it describes, so main documents two installer knobs and a 4Gi default that exist nowhere outside this branch. An operator following the shipped page today sets a value that silently does nothing. The page is corrected to the default this branch ships and gains the three behaviours it had no account of at all: that a namespace holding a request above the ceiling is raised rather than skipped, that a raised ceiling only comes back down after the scan has been continuously empty for a grace period and why that delay has to exist, and that a namespace another LimitRange already governs is left alone. Each names the log line to grep for, since none of the three is visible in the object. The ceiling rationale is also stated more honestly: the scan reads declared requests and never a container's real resident set, which is the reason the default sits far above any plausible working set rather than close to one. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
d3df698c43 |
docs(release): gate maintenance cuts on backport delivery
Listing labelled pull requests cannot establish whether their changes landed on a maintenance branch. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-by: LLM |
||
|
Aleksei Sviridkin
|
520af57905 |
chore: drop private cluster, host and personal names from the tree
Private infrastructure names, a real hostname, a commented-out internal mirror block, an internal product name, and a reviewer's name in a comment do not belong in a public repository; drop them and replace what needs a stand-in with neutral placeholders. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f5325425d6 |
fix(vm-instance): supply cloud-init network-data to netplan guests (#4306)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
A guest whose interfaces are rendered by netplan ignores two things the
platform already puts on the wire for it, and this adds an opt-in
`netplanDhcp` value that supplies the cloud-init network-data making it
ask for both.
`virt-launcher` starts a DHCP server on **every** bridge-bound
interface, secondary ones included — `newDHCPConfigurator` in
`pkg/network/setup/launcher/configurator.go` gates only on `iface.Bridge
!= nil`, and `BridgeConfigGenerator.Generate` serves the address the CNI
put on the pod-side interface along with that link's MTU. The chart
renders `bridge: {}` for every network, so a VM joined to a VPC subnet
is offered its reserved address and its MTU. cloud-init's fallback
network configuration covers a single interface, so on Debian and Ubuntu
nothing ever sends a DHCPDISCOVER on the second NIC and the interface
stays inert, while AlmaLinux (NetworkManager auto-profile) and Windows
come up. That asymmetry is the tell: with no DHCP server at all, none of
the three would work.
The same DHCP server advertises the launcher pod's search domains as
option 119, and `UseDomains=` defaults to `no` in systemd-networkd, so a
netplan-managed link keeps none of them. A cluster domain under `.local`
then reaches the mDNS exclusion in resolved and every lookup answers
`REFUSED`, which is #2085. `dhcp4-overrides: use-domains: true` is
enough on its own: the domains on the wire already carry whatever
cluster domain the cluster actually uses, so nothing needs to be
hardcoded here and #2372 does not collide with this.
### This is a regression, not a missing feature
`packages/apps/virtual-machine` carried exactly this, added in
|
||
|
IvanHunters
|
f05718b2de |
feat(redis)!: refuse external access without TLS at admission (#4639)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does An admission policy now refuses a Redis release that would expose a plaintext endpoint outside the cluster. Redis never turns TLS on by itself, so `external: true` without `tls.enabled: true` is exactly that case. The error names the release and a remedy that works. The rule has three parts, because TLS is fixed when a Redis instance is created: the RedisFailover schema refuses a change, while the Redis API accepts one and the HelmRelease then fails. A spec's `tls.enabled` is therefore a request, not proof of what the instance runs. - On create, `external: true` needs `tls.enabled: true`. - On update, turning `external` on is refused whatever `tls.enabled` says. - While `external` stays on, `tls.enabled` cannot change. Before the first reconcile nothing else stops TLS being dropped from a release admitted as external with TLS. Withdrawing `external` in the same edit lifts the freeze, and the callers trusted by `cozystack-tenant-host-policy` are exempt, so an instance whose TLS was flipped earlier can still be unblocked. Edits to a release that was already external pass, so releases exposed in plaintext before this policy keep working. The guarantee covers the Redis app. A generic publisher such as `tcp-balancer` can still forward to a plaintext Redis, by design. This diverges from the Security section of the `external-database-exposure` proposal, which refuses only an explicit `tls.enabled: false` and covers every engine. That rule was written before redis had TLS and matches only postgres. Postgres is left out here on purpose: CloudNativePG keeps `ssl=on` and its default pg_hba accepts plaintext from any client that asks, whatever the chart's `tls.enabled` says, so an admission rule on that field closes nothing. That gap is #4619. The proposal update and a decision record are cozystack/community#91. The policy follows `cozystack-tenant-host-policy`: a ValidatingAdmissionPolicy in `cozystack-basics`, enforced inside `cozystack-api` for the aggregated kind. Its decisions are tested in Go by compiling the shipped template with the apiserver's own CEL compiler and running Redis objects through it, including the bypass sequences found in review. A structural chart test can't show what the policy admits. The redis README, the `external` field description and the threat model describe the new rule. Related: #4622 (valkey `external` is always plaintext), #4624 (the redis `tls.enabled` description implies a live switch). ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: I walked the trigger map against the diff and nothing in it fires, but I left the boxes empty because cozystack/community is reached anyway: the `external-database-exposure` proposal still states the old admission rule. Its update and a decision record are cozystack/community#91. ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(redis)!: a Redis release can be exposed outside the cluster only when it is created with `tls.enabled: true`. Creating one with `external: true` and no TLS, turning `external` on for an existing instance, or changing `tls.enabled` while `external` is on is refused at admission. Instances already external without TLS keep working and stay editable. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Redis admission checks now require TLS when external access is enabled at instance creation. They block enabling external access on an existing instance and changing TLS while external access remains enabled. Disabling external access in the same update allows TLS changes. * Existing instances that already had external access without TLS before the policy was introduced remain editable. * **Documentation** * Updated Redis configuration and security guidance to explain these restrictions and TLS migration behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
80eb2e5037 |
ci(e2e): refuse a new fork image before the build starts
The fork e2e lane pushes a fork's image archives only under names the base tree already exports, which keeps a fork from minting repositories under the CI registry. An image the PR itself adds is therefore refused, but only at the push, after the whole build has run, and nothing the contributor does on the PR can clear it. Compare the image names of the PR tree and its base in the plan job of fork runs, with the same make -n derivation the push allowlist uses, and fail there naming the new images and what has to happen first. The push allowlist is unchanged and remains the boundary: this check runs from the fork's own tree, so it only saves an honest contributor the wasted build. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Mattia Eleuteri
|
39bdc9c404 |
fix(vm-instance): make the netplan network-data opt-in
Keying the network-data on instanceProfile reached every VM that never set the field, because the profile defaults to ubuntu whatever image the disk holds. The sysconfig, NetworkManager and eni renderers turn each match stanza into an interface literally named after the netplan ID, so a non-netplan guest left on the default profile would lose the DHCP it gets today. A VM with neither cloudInit nor sshKeys would also gain a cloud-init disk on upgrade, and KubeVirt's NoCloud instance-id is new to that guest, so its next boot would regenerate host keys as a first boot. The configuration also shadows a static NIC set up through netplan: netplan emits one networkd file per ID, networkd applies only the first file in lexical order that matches a link, and 10-netplan-cozy-en sorts before the file for enp2s0. The external-VLAN recipe claimed otherwise. An explicit netplanDhcp value, off by default, leaves every existing VM rendering exactly as before and lets the documentation say when not to turn it on. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
Mattia Eleuteri
|
b5181b0b9b |
fix(vm-instance): supply cloud-init network-data to netplan guests
A netplan guest configures only the interface cloud-init's fallback configuration covers, and keeps none of the search domains DHCP offers it, so two things the platform already puts on the wire never reach it. virt-launcher runs a DHCP server on every bridge-bound interface, secondary ones included, yet a VM joined to a VPC subnet leaves that interface inert for want of a client asking. The same server advertises the launcher pod's search domains as option 119, which systemd-networkd discards because UseDomains= defaults to no, and a cluster domain under .local then falls through to the mDNS exclusion in resolved and answers REFUSED. The virtual-machine chart carried a network-data template for the first of those, gated on systemDisk.image. Splitting that chart into vm-disk and vm-instance removed the template along with the field it read, and neither came back. Matching en* and eth* rather than naming interfaces by index keeps the configuration correct whatever the guest enumerates its NICs as, which is where the previous template was fragile. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <mattia@hidora.io> |
||
|
Alexey Artamonov
|
b3103af693 |
docs(observability): add a distributed tracing guide
Tracing reaches a workload only if it is instrumented and pointed at the collector, and the managed engines push no OTLP spans of their own, so operators and tenants need one place that says what the platform provides and what is left to the application: the in-namespace OTLP endpoints and the standard OTEL_* variables, sampling and redaction, and what each engine can and cannot emit. Shared-central tracing has more to it than the Tenant field description can carry, since that text is shown as field help in the dashboard: hosting the store in tenant-root, the shared disk and the per-tenant write rate with its limits, the write-only and read-only logins and the trust boundary, the windows on turning it on and off, and the order to stop hosting in. The guide holds that detail. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Aleksei Sviridkin
|
415430bcc7 |
fix(e2e): fail on a failed release of cilium and csi as on the parent, and report addon remediation (#3594)
## What this PR does The history guard in the tenant Kubernetes e2e suite checked only the parent `kubernetes-<test>` HelmRelease. A failed action on one of the addon releases it installs left nothing the suite looked at, so a run showed only the downstream symptom, usually the node-join deadline, and never named the release behind it. The guard now reads every addon release of the parent. It selects them from the cluster by name prefix, so a release added to the chart later is covered too. The rule: - A `failed` or `uninstalled` Snapshot fails the run on any release whose install strategy is `RetryOnFailure`. That is the parent (the aggregated apiserver stamps it on every Application HelmRelease) and the tenant `cilium` and `csi`. - The same footprint on the other addons, which run the default strategy with `retries: -1`, is printed as a note and the run stays green. For the CRD addons that configuration is tracked in #3555. A transient `cilium` or `csi` failure that later recovers on a retry still turns the run red, by design, as long as its Snapshot still reads `failed`. One recovery leaves no such Snapshot: when an install fails and the very next attempt succeeds, Helm marks the failed install `superseded`, because there is no deployed release to supersede, and the history reads like an ordinary upgrade. The guard cannot see that case from the statuses alone. The code and the docs now say so, a test pins it, and detection is left to a follow-up. An earlier revision of this PR tried to tell a teardown apart from a retried failure and made only `uninstalled` fatal. That split does not hold. In helm-controller v1.5.0, the version the platform installs, a release with an active retry strategy never remediates. `atomic_release.go` returns an upgrade for a failed release before it reaches the remediation switch, and the two other paths that uninstall (a changed release target, and deletion) both clear history afterwards. So on these releases `uninstalled` never shows up, and `failed` is the only signal. The parent keeps the rule it already had on main, and `cilium` and `csi` now follow it too. The failure paths get the same reading. The guard runs after the assertions, while the deadlines a failed addon breaks end the script earlier, so the same classification is also printed, without a verdict, from the node-join collector and from the exit handler. That handler is now armed at the tenant CR apply, so the management-cluster waits before the tenant answers are covered too. Smaller changes that came with this: - Every guard read is bounded, and a failed read is never folded into "no history". - The parent must read back a non-empty strategy. An empty value there means the jsonpath stopped resolving, and without this check every footprint would quietly turn into a note. - The history read uses `HELMRELEASE_HISTORY_JSONPATH`, and the upstream-type pin in `internal/fluxcontract` now accepts the bounded wrapper the read goes through. `docs/agents/e2e-testing.md` §6 and checklist item 8 describe the new rule. On related issues: this adds the `local` that #3774 reports missing in `helmrelease_has_remediation_cycle`, but not the sweep over the other `_lib/` helpers that the issue also asks for. For #3773 it rewrites the guard comment the issue quotes. The waits without a backstop are not touched. Closes #3553. ### Screenshots Not a UI change. ### Downstream repositories I walked the trigger map against the diff. It touches `hack/e2e-chainsaw/_lib/`, `hack/*.bats`, one Go test under `internal/fluxcontract`, two spend-order comments in the kubernetes suites, and `docs/agents/e2e-testing.md`. No package, schema, CRD, make target or user-facing document changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(e2e): the HelmRelease history guard now also covers the tenant addon releases. A failed or uninstalled Snapshot fails the run on the parent, cilium and csi, which retry in place; other addons get a note. The same classification is printed on the failure paths where the guard itself cannot run. ``` |
||
|
IvanHunters
|
40638429bd |
fix(release): collect refs stored as registry/image/tag/digest keys (#4697)
## What this PR does The shared image-ref enumeration skipped refs stored the way upstream ingress-nginx stores them: `registry`, `image`, `tag` and `digest` as separate keys, with the repository in `image` and a digest that has no `@`. Two refs in the tree use this layout, the ingress-nginx controller and its default backend, both in `packages/system/ingress-nginx/values.yaml`. Every tool that reads refs through `hack/lib/image-refs.sh` missed both of them: the promote retag, the nightly mirror, the multi-arch gate, the stitch step and the promoted-packages verifier. I added a sixth YAML shape that joins `<registry>/<image>@<digest>`. It fires only on a complete `sha256:` digest, so upstream's unpinned default (`digest: ""`) and image maps without a digest emit nothing. It checks that `digest` is a string before `test()`, same as shape 3 does. The library header and `docs/agents/image-refs.md` now say "the YAML sub-shapes listed" instead of a number, so the next shape doesn't make the count wrong again. The real tree now gives 93 refs instead of 91, and the two new ones are exactly the ingress-nginx pair. The retag and mirror output does not change today, because both refs sit outside the owned registry and the ownership filters drop them. Both digests are multi-arch indexes, so `hack/verify-multiarch.sh --refs-file` with just those two passes and the release gate stays green. To show the consumers really pick the shape up, I ran both dry-runs on a copy of the tree with the controller moved under the build registry, the host being the whole `registry` value. In that copy the controller is the only build-registry ref. Before: ```text $ REGISTRY=iad.ocir.io/idyksih5sir9/cozystack hack/promote-retag.sh 1.99.0 --dry-run No cozystack-owned digest-pinned image refs found under iad.ocir.io/idyksih5sir9/cozystack/ — is this the rc's baked tree? exit 1 $ hack/nightly-mirror.sh 1.99.0 <tree>/packages --dry-run No cozystack-owned digest-pinned image refs found under iad.ocir.io/idyksih5sir9/cozystack/ in '<tree>/packages' — is this the baked main tree? exit 1 ``` After: ```text $ REGISTRY=iad.ocir.io/idyksih5sir9/cozystack hack/promote-retag.sh 1.99.0 --dry-run ▸ iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller sha256:a8ddcee59b63f6cbf93f4d3c9ffa1d1dda3c1d39497748d4484655356c68b3f6 DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller@sha256:a8ddcee5… docker://iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller:1.99.0 exit 0 $ hack/nightly-mirror.sh 1.99.0 <tree>/packages --dry-run ▸ iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller -> ghcr.io/cozystack/cozystack/ingress-nginx-controller sha256:a8ddcee5… DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/…/ingress-nginx-controller@sha256:a8ddcee5… docker://ghcr.io/cozystack/cozystack/ingress-nginx-controller:1.99.0 DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/…/ingress-nginx-controller@sha256:a8ddcee5… docker://ghcr.io/cozystack/cozystack/ingress-nginx-controller:nightly exit 0 ``` The red test is a new case in `collect_refs_from_file understands every YAML shape`. It has a fixture in the ingress-nginx layout, with a non-string digest in front of it, and three near-misses that must emit nothing: no digest, an empty digest, and a digest without `image`. On main the positive case fails with `got: <nothing>`. Removing the rule, the registry join, the full-digest check, or the type guard on `image` or on `digest` turns that test red, and only that one. The bats suites of all library consumers pass with both the old and the new library. Two gaps remain in the tests and I left them out on purpose. No fixture covers the new rule without a `registry` key, and none pins its type guard on `registry`. Shapes 2 and 3 have the same unpinned `registry` guard on main. The overlay script has no such gap. `hack/overlay-main-images.sh` does not use the library, and its line classifier already accepts `image:`, `registry:`, `tag:` and `digest:` lines. Closes #4518 ### Screenshots Not a UI change. ### Downstream repositories I walked the trigger map against the diff. None of the changed files are in it. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(release): the image-ref enumeration used by promotion, the nightly mirror and the multi-arch gate now reads refs stored as separate registry/image/tag/digest keys, the layout of the ingress-nginx controller and default backend. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Image references can now be read from configurations that store the image name and SHA-256 digest separately, with an optional registry. * **Bug Fixes** * Incomplete or invalid image and digest values are ignored rather than producing malformed references. * **Documentation** * Documented the additional image storage format. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
9196b9798e |
fix(e2e): bound the dataplane capture collector by wall clock (#4640)
## What this PR does The e2e data-plane collector limited itself by counting pods and LoadBalancers, but both of its callers stop it by the clock. `hack/cozytest.sh` gives it 600s and the Chainsaw catch gives it 300s. One affected pod could already run past the smaller limit, and the LoadBalancer capture, which runs last, was the part that got lost. I moved the collector to a wall-clock budget and fixed the other issues filed against it. - Every kubectl call now goes through `dp_run`. It cuts the call's timeout to what is left of `COZY_DATAPLANE_BUDGET` and starts nothing once the budget is spent. The default is 270s, and `cozytest.sh` passes 570s. The pod section stops 90s early, so the LoadBalancer section always gets time to start. A call the deadline cut says so, a call cut by its own bound names that bound, and a refused call says it was not started and names its section; exec blocks carry that line inside the block. The node capture and the LoadBalancer probe used to blame an unanswered lookup or a missing cni-server when the budget stopped a call. Now they name the budget (#3671). - A static test fails on any kubectl call that bypasses `dp_run` and lists the lines. It matches by command position, so a call inside a double-quoted `$(...)` is still checked (#3694). - Per-pod and per-LoadBalancer capture files join namespace and name with `_`, which neither can contain. A second run into the same directory appends, with the run separator `capture-notes.txt` already uses (#3679). - `capture_node` and `capture_lb_node` no longer write their callers' loop variables. A static test fails when a function assigns a name that a `read -r` loop or another scope owns (#3675). - The tests that shrank every timeout to one second now cut only the read under test. A `timeout` shim decides by argv, so a slow machine can't cut a read the test needs answered (#3685). - `cilium-dbg bpf lb list` runs unfiltered in every affected node's capture, not only for a failing LoadBalancer (#3689). The call-site comment in `.chainsaw.yaml` and the collector paragraph in `docs/agents/e2e-testing.md` now match the code. Every behaviour change has a test in `hack/capture-dataplane.bats` that failed on the code before it. Two parts of the issues live elsewhere. The runner stopping a bats file at its first failing test is #4620, and a conntrack read filtered by service VIP is #4623. From #4629 I fixed the items on lines this PR already changes: the node capture now gets the run separator, a failed speaker log read writes a note, the endpoint tcpdump says when it falls back to the geneve interface, and the per-read timeout knobs are validated (a zero there would otherwise look like a spent budget). The rest of #4629 stays open: `ovn-lflows.txt` and `lb-speaker-logs.txt` are still replaced on each run, and two comments still describe older behaviour. - Closes #3694 - Closes #3671 - Closes #3679 - Closes #3675 - Closes #3685 - Closes #3689 - Refs #4629 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(e2e): bound the failure-path data-plane collector by a wall-clock budget, keep its per-pod and per-LoadBalancer captures from overwriting each other, and capture the cilium service map for ClusterIP failures. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Data-plane diagnostics now follow a configurable time budget, reserving time for LoadBalancer captures and stopping operations when the budget is spent. * Capture reports distinguish failed, successful, unknown, and budget-limited checks. LoadBalancers with successful probes are skipped for heavy capture. * Repeated captures are preserved together, and limits help control LoadBalancer diagnostic collection. * Diagnostics include additional network details from relevant nodes during probe replay. * **Documentation** * Updated guidance covers capture time limits and how unanswered EndpointSlice reads are reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
858ca4d0a3 |
test(e2e): cover monitoring OIDC modes with a Chainsaw suite (#4678)
## What this PR does Monitoring OIDC gets live-cluster e2e coverage again. It had none since the two pre-Chainsaw BATS suites were removed without a port. This adds a Chainsaw suite in `hack/e2e-chainsaw/monitoring/` that ports what they asserted, plus the in-place switch from an inline config to a `secretRef` that nothing covered. The suite takes one Monitoring release in `tenant-test` through three modes in a row: System, CustomConfig with an inline config, and CustomConfig with a `secretRef`. It creates that release itself and never reads the tenant-root monitoring instance, so it keeps working if #4023 or #4479 turns root monitoring off. What each phase checks: - System mode: cozystack-api admits `spec.oidc` and both HelmReleases go Ready. The Grafana CR points `auth.generic_oauth` at the cozy realm, has no `role_attribute_path`, and has `skip_org_role_sync` and `oauth_allow_insecure_email_lookup` forced. The KeycloakClient is confidential and references the generated Secret, which holds 32 alphanumeric characters. The audience scope is bound to this release's clientId. There is no chart-owned `KeycloakRealmGroup`. The users-Job carries the users and the mode, and `grafana-deployment` reads the client secret from the Secret. - Switch to inline: the Grafana CR takes the tenant's map, keeps the forced keys and drops every System-only key. The users-Job reruns with the new list. The KeycloakClient, the audience scope and the client Secret are pruned, and the Deployment stops referencing that Secret. - Switch to `secretRef`: the inline `auth.generic_oauth` block is gone from the Grafana CR. The operator Secret is mounted at `/etc/grafana/oidc` on both the CR and the Deployment. This checks the mount only and does not prove Grafana loads the file, see #4679. - After the install and after each switch, neither release may show a remediation cycle. That means no `failed` or `uninstalled` snapshot, and the latest revision must be exactly 1, then 2, then 3. The revision check is there because helm-controller truncates history after the previous deployed snapshot. An upgrade that failed, rolled back and then passed on retry leaves no failed entry, only extra revisions. I did not assert `GF_PATHS_CUSTOM_INI`. I could not find anything in Grafana that reads it (#4679), and pinning it would make possibly dead wiring part of the contract. The coverage split with the render tests is this. `packages/system/monitoring/tests/oidc_test.yaml` keeps the fresh-install render of each mode, including the CustomConfig negatives (no Keycloak objects, no client Secret). This suite covers what a render cannot see: admission, the Deployment the Grafana operator builds, and what a switch between modes prunes or leaves behind. The Keycloak objects are checked as written, so creating the client in Keycloak is not asserted. Removing it is, because the operator's finalizer holds the KeycloakClient until the client is gone from Keycloak. Before the first apply the suite also waits until the tenant's `cozystack-values` carries `oidc-enabled: "true"`. The outer release copies `_cluster` into the inner release once, so a Monitoring applied right after the tenant is created can render from stale values and fail the System-mode guard for the full 15m. That is what the first CI run on this PR hit. The Keycloak CRDs are required, not probed for. The e2e install always enables platform OIDC, so a missing CRD is a broken install. The removed suites skipped in that case, and their probe could not tell an absent API group from a present one. The selector picks the suite up by name from `cozystack.monitoring-application` and `cozystack.monitoring`. A change to `system/monitoring`, `extra/monitoring`, `monitoring-rd` or `grafana-operator` now runs only this suite instead of the full run. The tenant-root Grafana bring-up stays covered by the install BATS while it exists. One selector unit test relied on `cozystack.monitoring` reaching no suite, so it now renames that source in its copy of the graph. The second commit keeps Keycloak changes on every suite that drives a Keycloak client. Before this PR a change to `system/keycloak` or `system/keycloak-operator` escalated to the full run. With the monitoring suite it would select monitoring alone, because `monitoring-application` is the only source that depends on `keycloak-operator`, and `kubernetes-latest` with its OIDC lifecycle would drop out. `src_to_suites()` now maps both to `kubernetes-latest monitoring`. The mapping has to be explicit because `kubernetes-application` has no `dependsOn` on `keycloak-operator` (#4680). A `postgres-operator` change now also reaches `kubernetes-latest`, through Keycloak's database. The selector comment and the e2e doc no longer use `system/postgres-operator` as the example of a path with a source that reaches no suite. Cost: one monitoring stack bring-up, two in-place upgrades and teardown. I expect 10 to 20 minutes on every full run. The selector change makes the CI run on this PR a full run, so the suite's own timing is in the chainsaw report. A red run can spend up to 16m on each of four waits, since each one sits a minute above the 15m timeout of the release it waits on. The suite cannot run locally, so I checked it this way: - `chainsaw lint test` at v0.2.15, the sandbox version, passes. - I ran the suite's own steps against a local envtest apiserver. Chainsaw applies the real manifests, and a stand-in for helm and the operators renders the chart from the stored Monitoring spec. With the chart intact all nine steps pass. Each of 30 single breakages of the chart or of the simulated controllers turns the expected step red. - `hack/select-e2e_test.bats` and `hack/select-install_test.bats` pass. The new Keycloak selection case fails without the mapping and when `kubernetes-latest` is dropped from it. A browser login through Keycloak is out of scope, same as in the removed suites. Left for later: - KeycloakClient and scope `status` after the operator creates them in Keycloak is not checked. - `hack/e2e-chainsaw/README.md` lists suites by hand. I added `monitoring`, but `backup`, `kafka-metadata` and `kubernetes-proxmox` were already missing. - The `operator diff selects all dependent app suites` selector test could also check `monitoring`. - The comment on the Keycloak mapping says both suites wait for the finalizer to remove the client. The monitoring suite does wait. `kubernetes-latest` only reads once after its upgrade. - The ready checks compare `observedGeneration` with `metadata.generation` through `contains()`. Both are fields of the same object, not a number literal, so the v0.2.15 literal gotcha should not apply, but no live run has passed this suite yet. - The inline fixture carries no `enabled: "true"`, so that phase checks a disabled OAuth block. The asserts do not depend on it. Closes #4051 Closes #3785 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added end-to-end coverage for Monitoring installations using system OIDC, inline custom configuration, and Secret-based configuration. The tests verify readiness, rendered resources, configuration cleanup, and upgrade history. * Updated test selection so Keycloak-related changes also run the Monitoring suite. * **Documentation** * Added Monitoring to the documented Chainsaw suite list and clarified how changed paths determine test coverage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
