mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 22:13:18 +09:00
7ea71042b73bc0626cd8835fc9f68cc96f03faf2
7211
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
7ea71042b7 |
ci(e2e): add a daily isp-hosted install lane on kind (#4712)
## What this PR does
This adds a daily CI lane that installs the `isp-hosted` platform
variant through the `hosted` installer variant. No lane ran either of
them before. The PR, nightly and tag lanes all install `isp-full` on
Talos, where Cozystack brings its own CNI and storage, and that is how
#3376 shipped unnoticed.
What `isp-hosted` changes is what it leaves to the host. The system
bundle swaps kube-ovn and Cilium for a `noop` networking Package and
drops multus, LINSTOR, MetalLB, cozy-proxy, snapshot-controller and the
rest of the shared data-plane set. The iaas bundle refuses to render.
Migrations are not a difference, `isp-full` enables them too.
So the lane runs on kind. kindnet, kube-proxy and local-path come from
the cluster, and there is no Cilium, no LINSTOR and no `replicated` or
`local` StorageClass. It installs the latest GHCR nightly instead of
building anything, at 05:00 UTC after the nightly mirror publishes, and
on `workflow_dispatch`. There is no pull request trigger, because a run
would install the nightly images and not the PR's.
A run passes when every HelmRelease is Ready, `cozystack-platform` and
`tenant-root` included. It also checks that the platform emitted the
`noop` networking Package and none of LINSTOR, MetalLB, multus or
KubeVirt, and that the cluster has no Cilium CRD and no `replicated` or
`local` class.
The lane reuses the e2e sandbox container for tooling and
`collect-report`, attached to the kind network.
`hack/isp-hosted-lane_test.bats` pins the wiring: the Package selects
`isp-hosted`, the install uses `variant=hosted`, the kind DNS domain
matches `networking.clusterDomain`, and kind is pinned by checksum and
digest. Six of its seven tests were red before the implementation
existed.
`hack/sandbox-runner-headroom.bats` gets a named exemption. That check
treats any workflow touching `packages/core/testing` as a lane that
boots the QEMU guests, and this one boots none. The exemption is checked
after the job match, and removing it turns the test red again.
The pins come from the kind v0.32.0 release: the binary sha256
`50030de2…` from its `kind-linux-amd64.sha256sum`, and
`kindest/node:v1.33.12@sha256:3f5c8443…` from its release notes.
v1.33.12 is the Kubernetes version the Talos lanes use, so the variant
is the only thing that differs. All three are plain `env` values in the
workflow, where a Renovate custom manager can match them.
### Real runs
The first real run on this branch,
[36961209755](https://github.com/cozystack/cozystack/actions/runs/36961209755),
went red on `tenant-root`:
```text
tenant-root False Helm install failed for release tenant-root/tenant-root with chart tenant@0.0.0+629b61300902: unable to build kubernetes objects from release manifest: resource mapping not found for name: "allow-external-communication" namespace: "tenant-root" from "": no matches for kind "CiliumNetworkPolicy" in version "cilium.io/v2"
```
This is #3387 reproduced live: `packages/apps/tenant` renders Cilium
policies on a cluster without Cilium. The other releases went Ready,
except `cozystack-basics`, which was still in an upgrade at the last
listing. The fix landed in #4710.
The run after it,
[37089699259](https://github.com/cozystack/cozystack/actions/runs/37089699259),
is green on the first attempt. It installed nightly
`0.0.0-nightly.20261003.37089458387.1`, built from
|
||
|
IvanHunters
|
14eab3cfe3 |
fix(backups): restrict the kafka-metadata CLI Pod and correct demo personas (#4695)
## What this PR does
This fixes two problems in the Kafka and NATS backup demos under
`examples/backups/`.
The Kafka metadata demo ran its CLI Pod without any securityContext, so
a namespace that enforces the restricted Pod Security level would reject
it. The Pod now gets the same restricted context as the CLI Pod in the
topic-data demo: `runAsNonRoot`, the `RuntimeDefault` seccomp profile,
no privilege escalation, all capabilities dropped. The new bats case
reads the overrides passed to `kubectl run` and fails if any of those
fields is missing or the patch is not sent as `--override-type=json`. It
is red on main, and it goes red on each of seven single-field mutations
of the patch while the other five cases stay green.
One known limit: `runAsNonRoot` needs an image that runs as a non-root
user. The default Strimzi image runs as UID 1001 and is fine. If
`KAFKA_IMAGE` is overridden with an image that runs as root, the kubelet
refuses to start the container, and the helper fails only when its 5
minute `kubectl wait` times out.
The kafka and nats walkthroughs told a tenant to run scripts that the
tenant roles forbid. Steps 03, 04, 06, 07 and `cleanup.sh` were in the
tenant column, and the scenario docs and several script comments had the
tenant create the backup Secret. Those rows now say admin and name the
part a tenant admin can do on its own, and the scenario docs say the
same. After this change the persona column in both tables matches the
tenant roles. kafka-metadata has no persona table, so I didn't change
anything there.
These are the tenant rules on pods and backup objects, plus one example
of the per-app secret rules. Each app chart's dashboard Role allows
`get`, `list` and `watch` on a few named Secrets and no other verb on
secrets. Nothing aggregated into the tenant roles grants `pods/exec`,
`pods/log`, `helmreleases`, `objectstorage.k8s.io`, `statefulsets` or
`kafkas.kafka.strimzi.io`.
```yaml
# packages/system/cozystack-basics/templates/clusterroles.yaml:100-112 (cozy:tenant:view:base, every tenant level)
- apiGroups:
- ""
resources:
- pods
- services
- persistentvolumes
- endpoints
- events
- resourcequotas
verbs:
- get
- list
- watch
# packages/system/cozystack-basics/templates/clusterroles.yaml:238-246 (cozy:tenant:admin:base)
- apiGroups: [""]
resources:
- pods
- services
- persistentvolumes
- endpoints
- events
verbs:
- delete
# packages/system/backup-controller/templates/tenant-clusterroles.yaml:35-44 (cozy:backups:admin)
- apiGroups: ["backups.cozystack.io"]
resources:
- plans
- backupjobs
- restorejobs
verbs:
- create
- update
- patch
- delete
# packages/apps/bucket/templates/dashboard-resourcemap.yaml:6-14 (bound at the tenant "use" level)
- apiGroups:
- ""
resources:
- secrets
resourceNames:
{{- range $name, $user := .Values.users }}
- {{ $.Release.Name }}-{{ $name }}-credentials
{{- end }}
verbs: ["get", "list", "watch"]
```
How each changed row runs into those rules:
| Row | What the script needs | Why a tenant can't |
| --- | --- | --- |
| 03 (kafka, nats) | waits on the `HelmRelease` and COSI objects, `get`
on the `bucket-<name>-backup` Secret, kafka also reads the CA Secret in
`tenant-root` | no `helmreleases` or `objectstorage.k8s.io` rule; the
bucket Role covers only `<release>-<user>-credentials` |
| 04 (kafka, nats) | waits on the `HelmRelease` and the Strimzi `Kafka`
or the `StatefulSet`, `create` on `secrets`, `create` on `pods`,
`pods/exec` | pods are `get`/`list`/`watch`/`delete` only, and nothing
grants `create` on secrets |
| 06 (kafka, nats) | `create` on `pods` and `pods/exec` for the CLI Pod,
kafka also `get` on `pods/log` for the refused restore | same pod rules,
and nothing grants `pods/log` |
| 07 (kafka, nats) | same as 04 | same as 04 |
| `cleanup.sh` (kafka, nats) | `delete` on `secrets`, on `backups`, and
on the cluster-scoped `BackupClass` and strategy | `cozy:backups:admin`
deletes only plans, backupjobs and restorejobs, and no tenant rule
covers secrets or cluster-scoped objects |
Step 05 stays a tenant step. Its `BackupJob` comes from
`cozy:backups:admin`, so in practice that means a tenant admin. The nats
demo's CLI Pod has the same missing securityContext as the
kafka-metadata one, and I left it for a separate issue.
Closes #4625
Closes #4636
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
Not a UI change.
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix(backups): the kafka-metadata backup demo now runs its CLI Pod with the restricted Pod Security context, and the kafka and nats backup demos now list steps 03, 04, 06, 07 and cleanup as admin steps, because they need Pod, exec, Secret and cluster-scoped permissions that tenant roles do not grant.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Security**
* Kafka metadata backup CLI Pods are recreated with restricted security
settings when they are not running or lack the required non-root
configuration.
* **Documentation**
* Kafka and NATS guides clarify which setup, restore, and cleanup steps
require administrator access and which job actions tenants can perform.
* Updated examples clarify that the demo provides required storage
Secrets and certificates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
IvanHunters
|
1621c471f2 |
ci(dashboard): run eslint on the console and clear its errors (#4692)
## What this PR does
The console now passes its own eslint config, and CI runs it. On main
`pnpm lint` exits 1 with 54 problems, 51 of them errors, and
`ui-test.yaml` never ran it. On this branch it exits 0 with no errors,
and a Lint step sits between Typecheck and Test.
How each rule was handled:
- `no-explicit-any` (38 errors): replaced with real types. These commits
build a vite bundle byte-identical to main's.
- `react-refresh/only-export-components` (6 errors): the hooks, context
objects, `resolveSource` and the RJSF registry moved out of component
files into their own modules. `useTenantContext` stays in
`tenant-context.tsx`, so its importers and the tests that mock that
module are untouched.
- `no-unused-vars` (1 error): dropped the unused `ad` binding in
`OverviewTab`.
- `react-hooks/set-state-in-effect` (6 errors): relaxed per line with a
reason, not in the config.
The last rule is the only relaxation. Each of these effects re-syncs
local state to something that changes outside the component. The rule
wants the state derived during render, which changes when the value is
committed or what the first render shows, so it would be a behaviour
change. Per file:
- `components/ResourceQuotasField.tsx` re-syncs the inputs after the
form value is reset from outside.
- `components/SourceWidget.tsx` re-syncs the selected option when the
value loads asynchronously in edit forms.
- `components/command-palette/command-palette.tsx` resets query and
level whenever the palette closes.
- `components/command-palette/use-keyboard-nav.ts` moves the highlight
back to the top when the item list changes.
- `hooks/use-is-mac.ts` would return `true` on the first render on macOS
with a lazy initial state. Today it returns `false`.
- `lib/tenant-provider.tsx` falls back to an existing tenant once the
tenant list has loaded.
The eslint config itself is unchanged, and `no-explicit-any` stays on
everywhere.
`pnpm typecheck` and `pnpm test` (442 tests) pass at every commit. After
the file moves the bundle has the same chunk set, and only the entry
chunk differs, by reordered and renamed minified functions.
Three `react-hooks/exhaustive-deps` warnings from main are left as they
are, because fixing them changes when the effects run. One is in
`SourceWidget.tsx`, where the effect reads `options` but depends only on
`value`. The other two come from `ads = adList?.items || []` in
`use-command-items.tsx`, which gives two `useMemo` hooks a new
dependency on every render.
Things I did not do here:
- The step has no `--max-warnings`, so a new warning will not fail CI.
Once the three warnings are fixed, `--max-warnings 0` can go in.
- `SourceField.tsx` and `SourceWidget.tsx` each declare the same small
`SourceOptionSchema` interface. I kept them local since the two widgets
share nothing else.
- The job name stays `Typecheck and test`, so a required check with that
name keeps matching.
- `pnpm -r lint` only reaches `apps/console`.
`packages/{k8s-client,ui,types}` have no `lint` script, same as on main.
Closes #3911
Closes #4566
### Screenshots
No visual change: types, module layout and CI only.
### Downstream repositories
The diff touches only the console workspace and `ui-test.yaml`. The
console has no downstream copy since `cozystack-ui` was archived, and no
trigger map entry matches.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
ci(dashboard): the console's eslint now runs in the UI test job, and the console passes it with no errors.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Tenant selection now uses available tenant namespaces, restores a
saved selection when possible, and falls back to the root tenant or the
first available tenant.
* Tenant options are sorted, and the selected tenant is kept in sync
with the available choices.
* **Chores**
* Automated checks now include linting alongside type checks and unit
tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
IvanHunters
|
9239555e6a |
ci(workflows): pin flux, yq and helm-unittest installs (#4645)
## What this PR does Workflows no longer run the flux install script as root. Every step that needed the flux CLI downloaded `https://fluxcd.io/install.sh` and ran it with `sudo`, so whatever that host served at the moment ran on runners that later get registry credentials. PR #4442 pinned the version, but the script was still fetched and executed. All 12 of those steps now use `fluxcd/flux2/action`, pinned to the v2.8.6 commit with `version: "2.8.6"`. The action downloads the release tarball, checks it against that release's checksums file, and adds the binary to `PATH`. Two workflows still took yq from the floating latest release, nightly and the fork e2e lane. They now use the same install the promotion steps already have: yq v4.53.3 checked against a committed SHA-256. I verified that hash against the release's `checksums` file and against the binary itself. The helm-unittest plugin was installed at whatever version was newest. The plugin embeds its own Helm, so a plugin release could change chart test results on a PR that touched no chart. It's pinned to v1.1.2 now, the version CI already runs. A runner that carries a different version gets it replaced. A new contract test scans every workflow for these three tools, because an unpinned install usually comes back by copying an older step. On main it fails on the install script, the missing setup action, the latest-release yq and the unpinned plugin. I ran one mutation per guard, and each one failed only the test meant to catch it. The existing contract tests that checked `FLUX_VERSION` inside the old steps now check for the action step. The e2e sandbox image still runs `install.sh` in its Dockerfile. That's an image build, not a workflow step, so I left it for a separate change. Closes #3619 Closes #3640 Closes #4510 ### Screenshots Not applicable, no UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only workflow tool-install steps and `hack/*.bats` contract tests. Nothing in the trigger map reaches them. ### Release note ```release-note ci(workflows): install flux through the commit-pinned upstream setup action instead of running fluxcd.io/install.sh as root, pin yq with checksum verification in the nightly and fork e2e workflows, and pin the helm-unittest plugin version. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Reliability** * CI workflows now use a consistent, pinned Flux version. The yq and Helm plugin setup also verifies or enforces specific versions. * **Tests** * Added checks to ensure workflow tool versions are pinned and consistent, and that downloaded yq releases are checksum-verified. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
40638429bd |
fix(release): collect refs stored as registry/image/tag/digest keys (#4697)
## What this PR does The shared image-ref enumeration skipped refs stored the way upstream ingress-nginx stores them: `registry`, `image`, `tag` and `digest` as separate keys, with the repository in `image` and a digest that has no `@`. Two refs in the tree use this layout, the ingress-nginx controller and its default backend, both in `packages/system/ingress-nginx/values.yaml`. Every tool that reads refs through `hack/lib/image-refs.sh` missed both of them: the promote retag, the nightly mirror, the multi-arch gate, the stitch step and the promoted-packages verifier. I added a sixth YAML shape that joins `<registry>/<image>@<digest>`. It fires only on a complete `sha256:` digest, so upstream's unpinned default (`digest: ""`) and image maps without a digest emit nothing. It checks that `digest` is a string before `test()`, same as shape 3 does. The library header and `docs/agents/image-refs.md` now say "the YAML sub-shapes listed" instead of a number, so the next shape doesn't make the count wrong again. The real tree now gives 93 refs instead of 91, and the two new ones are exactly the ingress-nginx pair. The retag and mirror output does not change today, because both refs sit outside the owned registry and the ownership filters drop them. Both digests are multi-arch indexes, so `hack/verify-multiarch.sh --refs-file` with just those two passes and the release gate stays green. To show the consumers really pick the shape up, I ran both dry-runs on a copy of the tree with the controller moved under the build registry, the host being the whole `registry` value. In that copy the controller is the only build-registry ref. Before: ```text $ REGISTRY=iad.ocir.io/idyksih5sir9/cozystack hack/promote-retag.sh 1.99.0 --dry-run No cozystack-owned digest-pinned image refs found under iad.ocir.io/idyksih5sir9/cozystack/ — is this the rc's baked tree? exit 1 $ hack/nightly-mirror.sh 1.99.0 <tree>/packages --dry-run No cozystack-owned digest-pinned image refs found under iad.ocir.io/idyksih5sir9/cozystack/ in '<tree>/packages' — is this the baked main tree? exit 1 ``` After: ```text $ REGISTRY=iad.ocir.io/idyksih5sir9/cozystack hack/promote-retag.sh 1.99.0 --dry-run ▸ iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller sha256:a8ddcee59b63f6cbf93f4d3c9ffa1d1dda3c1d39497748d4484655356c68b3f6 DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller@sha256:a8ddcee5… docker://iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller:1.99.0 exit 0 $ hack/nightly-mirror.sh 1.99.0 <tree>/packages --dry-run ▸ iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller -> ghcr.io/cozystack/cozystack/ingress-nginx-controller sha256:a8ddcee5… DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/…/ingress-nginx-controller@sha256:a8ddcee5… docker://ghcr.io/cozystack/cozystack/ingress-nginx-controller:1.99.0 DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/…/ingress-nginx-controller@sha256:a8ddcee5… docker://ghcr.io/cozystack/cozystack/ingress-nginx-controller:nightly exit 0 ``` The red test is a new case in `collect_refs_from_file understands every YAML shape`. It has a fixture in the ingress-nginx layout, with a non-string digest in front of it, and three near-misses that must emit nothing: no digest, an empty digest, and a digest without `image`. On main the positive case fails with `got: <nothing>`. Removing the rule, the registry join, the full-digest check, or the type guard on `image` or on `digest` turns that test red, and only that one. The bats suites of all library consumers pass with both the old and the new library. Two gaps remain in the tests and I left them out on purpose. No fixture covers the new rule without a `registry` key, and none pins its type guard on `registry`. Shapes 2 and 3 have the same unpinned `registry` guard on main. The overlay script has no such gap. `hack/overlay-main-images.sh` does not use the library, and its line classifier already accepts `image:`, `registry:`, `tag:` and `digest:` lines. Closes #4518 ### Screenshots Not a UI change. ### Downstream repositories I walked the trigger map against the diff. None of the changed files are in it. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(release): the image-ref enumeration used by promotion, the nightly mirror and the multi-arch gate now reads refs stored as separate registry/image/tag/digest keys, the layout of the ingress-nginx controller and default backend. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Image references can now be read from configurations that store the image name and SHA-256 digest separately, with an optional registry. * **Bug Fixes** * Incomplete or invalid image and digest values are ignored rather than producing malformed references. * **Documentation** * Documented the additional image storage format. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
9196b9798e |
fix(e2e): bound the dataplane capture collector by wall clock (#4640)
## What this PR does The e2e data-plane collector limited itself by counting pods and LoadBalancers, but both of its callers stop it by the clock. `hack/cozytest.sh` gives it 600s and the Chainsaw catch gives it 300s. One affected pod could already run past the smaller limit, and the LoadBalancer capture, which runs last, was the part that got lost. I moved the collector to a wall-clock budget and fixed the other issues filed against it. - Every kubectl call now goes through `dp_run`. It cuts the call's timeout to what is left of `COZY_DATAPLANE_BUDGET` and starts nothing once the budget is spent. The default is 270s, and `cozytest.sh` passes 570s. The pod section stops 90s early, so the LoadBalancer section always gets time to start. A call the deadline cut says so, a call cut by its own bound names that bound, and a refused call says it was not started and names its section; exec blocks carry that line inside the block. The node capture and the LoadBalancer probe used to blame an unanswered lookup or a missing cni-server when the budget stopped a call. Now they name the budget (#3671). - A static test fails on any kubectl call that bypasses `dp_run` and lists the lines. It matches by command position, so a call inside a double-quoted `$(...)` is still checked (#3694). - Per-pod and per-LoadBalancer capture files join namespace and name with `_`, which neither can contain. A second run into the same directory appends, with the run separator `capture-notes.txt` already uses (#3679). - `capture_node` and `capture_lb_node` no longer write their callers' loop variables. A static test fails when a function assigns a name that a `read -r` loop or another scope owns (#3675). - The tests that shrank every timeout to one second now cut only the read under test. A `timeout` shim decides by argv, so a slow machine can't cut a read the test needs answered (#3685). - `cilium-dbg bpf lb list` runs unfiltered in every affected node's capture, not only for a failing LoadBalancer (#3689). The call-site comment in `.chainsaw.yaml` and the collector paragraph in `docs/agents/e2e-testing.md` now match the code. Every behaviour change has a test in `hack/capture-dataplane.bats` that failed on the code before it. Two parts of the issues live elsewhere. The runner stopping a bats file at its first failing test is #4620, and a conntrack read filtered by service VIP is #4623. From #4629 I fixed the items on lines this PR already changes: the node capture now gets the run separator, a failed speaker log read writes a note, the endpoint tcpdump says when it falls back to the geneve interface, and the per-read timeout knobs are validated (a zero there would otherwise look like a spent budget). The rest of #4629 stays open: `ovn-lflows.txt` and `lb-speaker-logs.txt` are still replaced on each run, and two comments still describe older behaviour. - Closes #3694 - Closes #3671 - Closes #3679 - Closes #3675 - Closes #3685 - Closes #3689 - Refs #4629 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(e2e): bound the failure-path data-plane collector by a wall-clock budget, keep its per-pod and per-LoadBalancer captures from overwriting each other, and capture the cilium service map for ClusterIP failures. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Data-plane diagnostics now follow a configurable time budget, reserving time for LoadBalancer captures and stopping operations when the budget is spent. * Capture reports distinguish failed, successful, unknown, and budget-limited checks. LoadBalancers with successful probes are skipped for heavy capture. * Repeated captures are preserved together, and limits help control LoadBalancer diagnostic collection. * Diagnostics include additional network details from relevant nodes during probe replay. * **Documentation** * Updated guidance covers capture time limits and how unanswered EndpointSlice reads are reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
f209c477e2 |
test(e2e): read the opensearch release history through the pinned name (#4689)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
The opensearch suite now reads HelmRelease history through the shared
jsonpath, and a unit test fails if any Chainsaw suite goes back to a
copied literal. Before this, the suite carried its own copy of the
expression, and the upstream contract test in `internal/fluxcontract`
only checks `HELMRELEASE_HISTORY_JSONPATH`. I checked the copy against
helm-controller `api` v1.5.1, the version go.mod pins: the history field
is `history` and each snapshot has `status`. So the read works today, it
was just not pinned.
The suite passes `-o "jsonpath=${HELMRELEASE_HISTORY_JSONPATH}"` now,
which gives kubectl exactly the same argument as before. The new test
reads every suite under `hack/e2e-chainsaw/`. A suite that calls
`helmrelease_has_remediation_cycle` must fill the local it hands the
guard from the shared name, and no suite may assign that name itself. It
reuses the matchers that already pin `run-kubernetes.sh`. The test is
red on main and green with the fix.
One known limit. The second-assignment check counts over the whole file,
which fits `run-kubernetes.sh` because it runs as one shell. A Chainsaw
suite runs each `script` step in its own shell, so an unrelated
`statuses=` in another opensearch step would turn the test red with
nothing broken. That fails toward red, never toward a silent pass, so I
left it.
Closes #4550
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
test(e2e): the opensearch suite reads HelmRelease history through the shared jsonpath, which is now pinned against the upstream Flux type for every Chainsaw suite that runs the remediation guard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Expanded validation across Chainsaw end-to-end suites to check
consistent HelmRelease history handling in remediation checks, including
the order and isolation of script steps.
* The OpenSearch suite now uses the shared history lookup. Empty-history
checks and remediation-cycle failure handling are unchanged, and there
is no change to end-user behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
IvanHunters
|
b4fbaf00a2 |
test(e2e): keep gateway and redis-tls op ceilings above their own waits (#4643)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Two Chainsaw script ops had a timeout smaller than the waits inside them. When the op ceiling fires first, Chainsaw kills the shell: the script never prints its own error, and a cleanup trap never runs. The gateway listener-set step could poll for 660s under a 6m op. Its cleanup trap also deleted two tenants with no limit on the wait. Each delete now runs under an outer `timeout`, because kubectl's own `--timeout` does not cover discovery or a stalled request. A tenant delete gets 300s, the same 5m that `.chainsaw.yaml` gives a delete op. The parent is deleted only after the child is gone, because deleting a parent while its child is still uninstalling wedges the parent's cleanup. A failed tenant delete fails the step with kubectl's own error, so a leaked tenant shows up here and not in a later suite. The op is now 24m, which covers the polls plus those bounds. The redis-tls CA-projection step has 30s read caps that add up to well over its 5m op. Raising the op would not hold, because several reads run in loops. So all reads now share a 240s step budget. No read starts after the budget runs out, and a read that is still running gets cut off when it does. A read that hits its bound, or finds the budget already spent, says which read it was. Two exec checks in the same step blamed the 15s redis-cli cap for any failure, including a refused exec or a missing container. They now print the exit status and what the exec printed. The step also runs under bash now, for pipefail. The sandbox's `/bin/sh` is dash, which rejects `set -o pipefail`. Chainsaw 0.2.15, the version the sandbox image installs, supports a `shell:` field on script ops. Without pipefail, a failed Secret read took base64's zero exit status and was reported as a wrong source name. The private-key check read the projection a second time, so a failed second read made it pass. It now checks the value the step already fetched. I added `hack/chainsaw-op-budgets.bats` to guard all of this. It holds the gateway op above its polls plus its delete bounds, keeps the child-before-parent order, holds the redis budget under its op with every kubectl call going through the clamped helper, and runs the redis step's script against a stubbed kubectl to check each failure message and the clamp. On main, 11 of its 12 tests fail. The one that passes is the healthy-stub control. Closes #4114 Closes #4250 Closes #4246 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches two Chainsaw suites and adds one unit test file under `hack/`. It moves or renames nothing a downstream repository reads. ### Release note ```release-note test(e2e): the gateway listener-set and redis-tls projection steps now fail on their own errors instead of being killed by the op timeout, and redis-tls reports the real cause of a failed read or exec. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added automated checks to verify gateway operations have sufficient time to complete and cleanup deletions are bounded and correctly ordered. * Expanded gateway scenarios to cover cleanup failures: if child-tenant deletion fails or times out, the parent is left in place; deletion failures are reported. * Updated Redis TLS checks with bounded read times and a shared execution limit. Failure reports include command output and status, and checks cover private-key material in the decoded projection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
1ca603a7b2 |
test(e2e): wait for the S3 gateway to accept a key before using it (#4694)
## What this PR does The bucket e2e suite fails now and then because the S3 gateway rejects a key that COSI already reports as granted. The error is `The access key ID you provided does not exist in our records.` from `mc alias set`. In job 109130131391 the BucketAccess assert finished at 21:23:43.57 and the first `mc alias set` failed at 21:23:44.82. In job 109396174094 cap1 and cap2 were accepted, and cap3 was rejected two seconds into the step. Granted access is not the same as a usable key. In seaweedfs-cosi-driver v0.3.1, `DriverGrantBucketAccess` writes the new key into `/etc/iam/identity.json` on the filer and returns. The S3 gateway in SeaweedFS 4.45 reloads that file later, from a filer metadata subscription (`weed/s3api/auth_credentials_subscribe.go`). In our chart the driver writes through `seaweedfs-filer` and the two gateways follow `seaweedfs-filer-client`, so the change may have to pass through the other filer replica first. The suite now waits until the gateway accepts the key before it writes. Every alias in both write steps goes through `await_key`, which uses the same port-forward as the writes and so asks the same gateway pod. It waits only while the answer is the InvalidAccessKeyId text, for at most 60s. Any other error fails at once, and so does an attempt cut off at its 15s cap. When the window runs out, the step fails with a message that names the BucketAccess. One probe costs at most 77s, so both steps go from a 5m to an 8m op timeout. The inner budgets now sum to 409s and 441s, and the budget comments say so. I checked the probe locally against a fake `mc`: ```text case=ok rc=0 attempts=1 elapsed=0s Added `a` successfully. case=lag rc=0 attempts=3 elapsed=5s Added `a` successfully. case=other rc=1 attempts=1 elapsed=0s FAIL: mc alias set for BucketAccess bucket-test-admin failed (exit 1) with an error other than the key not being loaded yet case=never rc=1 attempts=31 elapsed=60s FAIL: S3 gateway still rejects the key of BucketAccess bucket-test-admin after 60s of probing (the gateway never loaded it, or the driver never wrote it) ``` A hanging `mc` fails on its first attempt after 15s with exit 124. `chainsaw lint test` from v0.2.15 accepts the file. The read-only upload check gets fixed along the way. Before, an unloaded `ro-user` key also made the upload fail, and that failure passed as the expected denial. One thing I left as it is: `await_key` is copied into both steps, because chainsaw steps share no shell, and moving it to `_lib/` would run every suite on each edit. Closes #4580 ### Screenshots Not applicable, no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note test(e2e): the bucket suite waits for the S3 gateway to accept a new key before writing with it, instead of trusting BucketAccess readiness. ``` |
||
|
IvanHunters
|
858ca4d0a3 |
test(e2e): cover monitoring OIDC modes with a Chainsaw suite (#4678)
## What this PR does Monitoring OIDC gets live-cluster e2e coverage again. It had none since the two pre-Chainsaw BATS suites were removed without a port. This adds a Chainsaw suite in `hack/e2e-chainsaw/monitoring/` that ports what they asserted, plus the in-place switch from an inline config to a `secretRef` that nothing covered. The suite takes one Monitoring release in `tenant-test` through three modes in a row: System, CustomConfig with an inline config, and CustomConfig with a `secretRef`. It creates that release itself and never reads the tenant-root monitoring instance, so it keeps working if #4023 or #4479 turns root monitoring off. What each phase checks: - System mode: cozystack-api admits `spec.oidc` and both HelmReleases go Ready. The Grafana CR points `auth.generic_oauth` at the cozy realm, has no `role_attribute_path`, and has `skip_org_role_sync` and `oauth_allow_insecure_email_lookup` forced. The KeycloakClient is confidential and references the generated Secret, which holds 32 alphanumeric characters. The audience scope is bound to this release's clientId. There is no chart-owned `KeycloakRealmGroup`. The users-Job carries the users and the mode, and `grafana-deployment` reads the client secret from the Secret. - Switch to inline: the Grafana CR takes the tenant's map, keeps the forced keys and drops every System-only key. The users-Job reruns with the new list. The KeycloakClient, the audience scope and the client Secret are pruned, and the Deployment stops referencing that Secret. - Switch to `secretRef`: the inline `auth.generic_oauth` block is gone from the Grafana CR. The operator Secret is mounted at `/etc/grafana/oidc` on both the CR and the Deployment. This checks the mount only and does not prove Grafana loads the file, see #4679. - After the install and after each switch, neither release may show a remediation cycle. That means no `failed` or `uninstalled` snapshot, and the latest revision must be exactly 1, then 2, then 3. The revision check is there because helm-controller truncates history after the previous deployed snapshot. An upgrade that failed, rolled back and then passed on retry leaves no failed entry, only extra revisions. I did not assert `GF_PATHS_CUSTOM_INI`. I could not find anything in Grafana that reads it (#4679), and pinning it would make possibly dead wiring part of the contract. The coverage split with the render tests is this. `packages/system/monitoring/tests/oidc_test.yaml` keeps the fresh-install render of each mode, including the CustomConfig negatives (no Keycloak objects, no client Secret). This suite covers what a render cannot see: admission, the Deployment the Grafana operator builds, and what a switch between modes prunes or leaves behind. The Keycloak objects are checked as written, so creating the client in Keycloak is not asserted. Removing it is, because the operator's finalizer holds the KeycloakClient until the client is gone from Keycloak. Before the first apply the suite also waits until the tenant's `cozystack-values` carries `oidc-enabled: "true"`. The outer release copies `_cluster` into the inner release once, so a Monitoring applied right after the tenant is created can render from stale values and fail the System-mode guard for the full 15m. That is what the first CI run on this PR hit. The Keycloak CRDs are required, not probed for. The e2e install always enables platform OIDC, so a missing CRD is a broken install. The removed suites skipped in that case, and their probe could not tell an absent API group from a present one. The selector picks the suite up by name from `cozystack.monitoring-application` and `cozystack.monitoring`. A change to `system/monitoring`, `extra/monitoring`, `monitoring-rd` or `grafana-operator` now runs only this suite instead of the full run. The tenant-root Grafana bring-up stays covered by the install BATS while it exists. One selector unit test relied on `cozystack.monitoring` reaching no suite, so it now renames that source in its copy of the graph. The second commit keeps Keycloak changes on every suite that drives a Keycloak client. Before this PR a change to `system/keycloak` or `system/keycloak-operator` escalated to the full run. With the monitoring suite it would select monitoring alone, because `monitoring-application` is the only source that depends on `keycloak-operator`, and `kubernetes-latest` with its OIDC lifecycle would drop out. `src_to_suites()` now maps both to `kubernetes-latest monitoring`. The mapping has to be explicit because `kubernetes-application` has no `dependsOn` on `keycloak-operator` (#4680). A `postgres-operator` change now also reaches `kubernetes-latest`, through Keycloak's database. The selector comment and the e2e doc no longer use `system/postgres-operator` as the example of a path with a source that reaches no suite. Cost: one monitoring stack bring-up, two in-place upgrades and teardown. I expect 10 to 20 minutes on every full run. The selector change makes the CI run on this PR a full run, so the suite's own timing is in the chainsaw report. A red run can spend up to 16m on each of four waits, since each one sits a minute above the 15m timeout of the release it waits on. The suite cannot run locally, so I checked it this way: - `chainsaw lint test` at v0.2.15, the sandbox version, passes. - I ran the suite's own steps against a local envtest apiserver. Chainsaw applies the real manifests, and a stand-in for helm and the operators renders the chart from the stored Monitoring spec. With the chart intact all nine steps pass. Each of 30 single breakages of the chart or of the simulated controllers turns the expected step red. - `hack/select-e2e_test.bats` and `hack/select-install_test.bats` pass. The new Keycloak selection case fails without the mapping and when `kubernetes-latest` is dropped from it. A browser login through Keycloak is out of scope, same as in the removed suites. Left for later: - KeycloakClient and scope `status` after the operator creates them in Keycloak is not checked. - `hack/e2e-chainsaw/README.md` lists suites by hand. I added `monitoring`, but `backup`, `kafka-metadata` and `kubernetes-proxmox` were already missing. - The `operator diff selects all dependent app suites` selector test could also check `monitoring`. - The comment on the Keycloak mapping says both suites wait for the finalizer to remove the client. The monitoring suite does wait. `kubernetes-latest` only reads once after its upgrade. - The ready checks compare `observedGeneration` with `metadata.generation` through `contains()`. Both are fields of the same object, not a number literal, so the v0.2.15 literal gotcha should not apply, but no live run has passed this suite yet. - The inline fixture carries no `enabled: "true"`, so that phase checks a disabled OAuth block. The asserts do not depend on it. Closes #4051 Closes #3785 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added end-to-end coverage for Monitoring installations using system OIDC, inline custom configuration, and Secret-based configuration. The tests verify readiness, rendered resources, configuration cleanup, and upgrade history. * Updated test selection so Keycloak-related changes also run the Monitoring suite. * **Documentation** * Added Monitoring to the documented Chainsaw suite list and clarified how changed paths determine test coverage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
6e4c9c4b6a |
fix(ci): harden the PR labeler's title parsing and map recurring scopes (#4666)
## What this PR does Fixes three problems in `.github/workflows/pr-labeler.yaml`. Each one either killed the job or put a PR in the wrong area, and none of them showed up anywhere. - Both maps are now looked up with `Object.hasOwn`. A title typed or scoped `constructor`, `toString`, `hasOwnProperty` or `__proto__` used to resolve to an `Object.prototype` member and crash the step on `l.startsWith`, which left the PR with no labels at all. - Every leading `[Backport ...]` prefix is now stripped, not only the first one. With two prefixes, the second was read as a bracket scope, so the real scope and the `kind/*` label were both lost. The `area/release` label hid the problem by keeping `area/uncategorized` from firing. - Scopes that recur three or more times over the full PR history now map to an area. So does each area's own name (`networking`, `storage`, `database`, `virtualization`, `testing`, `dependencies`). Scope-to-area decisions: - `area/database`: `database`, `etcd-operator`, `opensearch`, `opensearch-operator` - `area/networking`: `networking`, `kubeovn`, `multus`, `kilo`, `hetzner-robotlb`, `ouroboros` (it rewrites the CoreDNS Corefile to fix hairpin NAT, so it is networking, not platform) - `area/virtualization`: `virtualization`, `vm`, `kubevirt-operator`, `kubevirt-cdi`, `kubevirt-instancetypes` - `area/platform`: `operator`, `cozystack-operator`, `controller`, `cozystack-controller`, `cozystack-basics`, `flux-shard-operator`, `migrations`, `scheduler`, `cozystack-scheduler`, `reloader`, `bootbox`, and `lineage`. I put `lineage` here on purpose: no existing label covers it more precisely, and it is too narrow for an area of its own. - `area/storage`: `storage`, `piraeus-operator`, `backupstrategy-controller` - `area/release`: `changelog`, `changelogs` - `area/ci`: `workflows`, `labeler` - `area/dependencies`: `dependencies`, `renovate` - `area/api`: `registry` (the cozystack-api storage layer) - `area/dashboard`: `dashboard-controller` - `area/monitoring`: `monitoring-agents` - `area/keycloak`: `keycloak-configure` - `area/testing`: `testing` - New `area/dx`, added to `.github/labels.yml`: `dx`, `hack`, `cozypkg`. These are developer tooling (hack/ scripts, package Makefiles, cozypkg). `area/build` covers image builds and `area/testing` covers tests, and neither describes this work. Left unmapped on purpose: - `http-cache`: the application was removed from the tree. - `openbao`: already mapped on main. - `apps`, `system`, `docs`: catch-all scopes that don't point to one area. - `fix`, `bugfix`: malformed titles. - `security`, `rbac`, `governance`, `codeowners`, `maintenance`, `images`, `readme`, `telemetry`, `cluster-autoscaler`: no single area fits any of them. I did not add spelling normalisation. `kubeovn` got its own row, since it is the only alternate spelling that recurs. A new contract test, `hack/pr-labeler-contract.bats`, pulls the workflow's own `script:` block out of the file and runs it under node with stubbed `context`, `core` and `github`, so it tests the shipped code, not a copy. On main it is red on the prototype titles (`THROW: l.startsWith is not a function`), on the double backport (only `area/release`), on the newly mapped scopes and on the area-name scopes. It also checks that every map target exists in `labels.yml` and that no key is written twice. The suite fails rather than skips when node is missing. The older node-based suites skip, so this is the first one that needs node on the unit-test runner. Some things stay out of this PR. The scope table in `docs/agents/contributing.md` is a summary. It has no row for `area/dx` and does not mention that an area's own name works as a scope. Separately, the singular `migration` still maps to `area/virtualization` as it did before, while the new `migrations` maps to `area/platform`. A backport PR whose scope is unmapped still never gets `area/uncategorized`, even with a single prefix. The `area/release` added by the strip satisfies the fallback check. This behaviour is the same on main and is not changed here. The test that checks map targets against `labels.yml` passes by itself if the map literal cannot be extracted. The duplicate-key and area-name tests go red in that case, so the suite as a whole still fails. Closes #4125, closes #4124, closes #4122 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(ci): the PR labeler no longer crashes on titles scoped like Object.prototype members, handles nested backport prefixes, and maps more recurring scopes, including a new area/dx. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added the `area/dx` label for issues and pull requests related to developer tooling. * **Improvements** * Pull requests can now be categorized using a broader set of scope aliases. * Repeated leading backport prefixes are recognized and route the pull request to the release area. * Label mapping now avoids treating inherited property names as valid mappings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
02fbe76418 |
fix(ci): close three blind spots in the e2e test selectors (#4658)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The e2e selectors miss Chainsaw suites that have no `chainsaw-test.yaml` or `.yml` directly in their directory. Chainsaw v0.2.15 also runs a directory that holds only numbered step files such as `01-install.yaml`, and its discovery walks every subdirectory of the path the runner hands it. A suite in either layout ran only when nothing was selected. No narrowed selection named it, the full-suite list left it out, and `select-install.sh --validate` never asked for its mapping. Now `hack/select-e2e.sh` and `hack/select-install.sh` list a top-level directory as a suite when Chainsaw would load a test from it or from anywhere below it. No suite in the current tree uses either layout, so today's suite list stays the same. A file placed directly in `examples/backups/` (a shared `common.sh`, for example) selected nothing, because it fell through to the inert `examples/` rule. It now escalates to the full suite, same as `examples/backups/_lib/`. `README.md` there stays inert through the `*.md` rule. The `test-chainsaw` comment in `packages/core/testing/Makefile` said Chainsaw discovers every `chainsaw-test.yaml`, which was too narrow. I fixed it, and the matching sentences in `docs/agents/e2e-testing.md` too. Every behaviour change has a bats test that fails against the scripts on main. Known gaps I left for a follow-up, none of them a wrong answer today: - The `full_suite_list` helper in `hack/select-e2e_test.bats` repeats the selector's `find`/`sed`, so a bug in that shared pattern would pass every full-suite assertion. The step-file and nested tests compare against literal lists, which covers the new layouts. - The selectors are a little wider than Chainsaw: a broken `chainsaw-test.y*ml` symlink, or one with no Test in it, still lists its suite. That errs toward running more. - A few test comments describe what used to happen rather than why the rule exists. Closes #4630 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(ci): the e2e test selectors now see Chainsaw suites made of numbered step files or nested below a suite directory, and run the full suite for an edit to a file placed directly in examples/backups/. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * E2E suite discovery recognizes supported manifest and numbered-step files at any depth and maps them to the top-level suite. * Changes to files directly under `examples/backups/` trigger full-suite selection. * Suite selection reports an error if discovery fails or finds no suites. * **Documentation** * Clarified supported suite layouts, disabled suite-file handling, discovery and escalation rules, and per-path coverage guidance. * **Tests** * Added coverage for nested suites, numbered steps, unmapped suite layouts, and backup-path selection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
9485f9118b |
test(tests): make negated BATS assertions able to fail (#3790)
## What this PR does
A hundred and twelve assertions across twelve BATS suites begin with
`!`. A hundred and seven of them pass no matter what they find. POSIX
exempts a command whose return value is inverted with `!` from `set -e`,
so the failing status is dropped where it is produced, and
`hack/cozytest.sh` closes each test function with `return 0`, so it
never resurfaces as the test's own status either. The other five are
followed by `|| { ... exit 1; }`, which reads the inverted status and
does work; they are rewritten too, because telling the two apart needs
the tail of every statement parsed, and a guard that guesses wrong there
is green on a real one.
Switching to the `bats` binary does not fix the hundred and seven. Its
ERR trap carries the identical exemption: `bash -c 'set -eE; trap "echo
TRAP" ERR; ! true; echo REACHED'` prints `REACHED`, runs no trap and
exits zero. The only negated assertion `bats` reports is one standing
last in the test body, where the inverted status happens to be what the
function returns. Two of the hundred and seven stood last; the rest did
not.
What was lost is specific rather than diffuse. `!` is how you spell
"this must not appear", and an absence is what no other assertion in a
test covers. The affected suites assert that a migration ran nothing
destructive, that no image was copied, that a flag was not passed, that
a backup cleanup never flushed the master, that an object does not
exist. On that half of their contract they held for any input.
Each site is now written as `if cmd; then echo "FAIL: ..."; false; fi`,
the form already used in the files where this had been noticed one at a
time, and each carries a message naming what must not have happened.
`hack/bats-no-negated-assert.bats` scans every `hack/**/*.bats` and
refuses the old form, so a new one cannot be added silently. Inverted
conditions (`if !`, `while !`, `until !`) are untouched, because `set
-e` is not what decides a condition. `[ ! -f x ]` is untouched for a
different reason and remains an ordinary failing assertion: the `!`
there is an operator of `test`, not negation of a command, so nothing
exempts its status.
The guard pins its own premise instead of describing it, on both sides.
One test asserts the runner still passes a bare negated assertion; a
second, that neither `set -e` nor the ERR trap sees an inverted status;
a third, that the prescribed `if cmd; then ...; false; fi` does fail,
under both runners, and *at* the form rather than somewhere before it,
since a run that died early would satisfy a bare status check and pin
nothing.
Every rewrite here rests on the third test. If `false` inside a `then`
body ever stopped propagating, the first test would still pass, because
a negated assertion would still be vacuous, while every converted
absence went silently green. A ban that prescribes a replacement has to
test the replacement too.
Every claim the guard makes is pinned by removing the thing it claims.
The recursion is exercised *through* the audit function against a
fixture in a subdirectory, not through a second copy of its `find`
expression: a copy keeps passing while the function loses its recursion,
and an audit that returns empty reads as a clean tree. Recursion matters
because `BATS_UNIT_FILES` is a non-recursive `wildcard`, so `make
unit-tests` never runs a suite placed in a subdirectory.
`hack/e2e-apps/` held such suites until they moved to Chainsaw and is
gone now, which is why the test pins against a fixture and not a real
directory.
The rule's surface was checked in the other direction too. The shape
appears in no `hack/**/*.sh` script as an assertion: the one hit,
`hack/testdata/install-wait/call-shapes.sh`, is fixture text that
`hack/run-kubernetes-install-wait_test.bats` parses, not a command
anything runs. It appears in no Chainsaw `script:` step either, so
`.bats` is the whole surface rather than the part that was convenient to
guard.
The scan is lexical, so its header names the statement boundaries it
owns rather than implying it understands shell: `;` `&` `&&` `||` `{`
`(`, the `)` closing a case pattern, and the words `then` `do` `else`,
each measured against the runner before being added. A keyword may
legally be preceded only by whitespace or by `;`, since every other
separator in front of `then`, `do` or `else` is a shell syntax error,
checked with `sh -n` rather than recalled. The scan therefore admits
exactly those two, and that part of the class is closed rather than
patched one spelling at a time. All three keywords are vacuous after
either spelling, so missing one is a silent miss, the single direction
this scan must not err in, given that its whole design trades noise for
safety. For the same reason it takes `!(cmd)` with no space before the
subshell, which is vacuous on dash and bash alike, and reads through a
`time` or `time -p` prefix, which is vacuous under `bats` and under
cozytest wherever `/bin/sh` is bash.
Five over-counts remain and are deliberate, each named in the header: `(
! cmd )` does propagate its inverted status and is reported anyway; so
would a `!` opening a statement after a redirection `&`; `x=$(true || !
false)` is reported although the `||` reads the inverted status; a
condition continued by a trailing `&&` instead of a backslash is
reported, because the strip only recognises a condition whose end it can
see; and `time ! cmd` is reported although on a dash `/bin/sh`, which
has no `time` keyword, it runs an external `time` that fails. Each costs
one rewritten line, where the opposite error costs a silent pass. They
are enumerated rather than engineered away on purpose, because a removed
error leaves no trace while a listed one tells the next reader what they
are looking at. The scan also errs *quiet* in at least two places, named
here rather than in the header. A file cut off midway through a line
continuation loses its last buffered statement, but that input is not a
runnable test, since `sh -n` rejects it with `unexpected end of file`.
And `time -- ! cmd`, which bash accepts, is vacuous under `bats` and not
peeled by the scan; nothing in the tree uses it, and I stopped adding
spellings there rather than chase every prefix bash allows. The test
naming the shapes claims only what its fixture holds. A lexical scan
over shell cannot promise it has enumerated the language, and a name
that promised it would be false whether or not a gap existed today.
The branch name is narrower than the subject and I am leaving it as it
is rather than churn it. This is not a defect in `hack/cozytest.sh` that
moving to the `bats` binary would fix. It is a POSIX rule both runners
inherit, and moving to `bats` would close two of the hundred and seven
sites.
One suite went red for real once its assertions could fail.
`seaweedfs-guard-parity` forbids four mutable discriminators by grepping
each chart, and both charts carry a template comment block arguing why
those signals were rejected, so the grep matched the argument and would
have reported a chart for documenting the rule it follows. The charts
are correct; the assertion was not. It now strips `{{/* ... */}}` before
matching. Two details there are load-bearing and neither is obvious. The
strip removes the comment *span*, not the line that opens it: dropping
the line is shorter and silently loses a discriminator sharing a line
with a comment, which is a chart classifying on a forbidden signal and a
test reporting that it does not. And the strip reports when the file
ended inside a comment, and the checks refuse to read its output when it
did, because every `{{/*` is read as an opener, so one inside a quoted
string (these charts carry long `fail (printf ...)` messages) would
swallow the rest of the file and leave the checks reading nothing.
One commit widens six comments that explained this trap as applying to a
"negated pipeline". It is not about pipelines, and the narrower wording
invites the conclusion that the other shapes are safe. Comment text
only; each of those files already uses the working form.
The last commit makes the one absence check in
`hack/e2e-install-cozystack.bats` that already used the `if` form tell a
missing object from a failed read. It asserted that the version
configmap was gone on `kubectl get`'s exit status alone, and that status
is 1 both for a NotFound and for an api-server the test cannot reach.
### Verification
Every unit suite this branch touches was run through `hack/cozytest.sh`
on the current base and is green. `migration-seaweedfs-db-adopt` passes
15 of 15 on macOS only when its temp directory sits under `$HOME`. Its
container cannot write into a bind mount under `/var/folders`, and
`/usr/bin/mktemp -d` ignores `TMPDIR`, so the run needs GNU `mktemp`
first on `PATH` as well. Measured on Linux before the rebase, it passed
15 of 15 as is.
The six sites that reached main after this branch was first written, in
`redis-backup-strategy` and `proxmox-e2e-fixture_test`, were each shown
to be able to fail: with the condition inverted, the suite goes red at
that assertion and prints its own `FAIL:` message. The nine sites in
`migration-seaweedfs-db-adopt` carry a two-sided check as well: with the
pattern swapped for one the same test proves is present, the rewritten
form fails the suite and the original `!` form passes it.
The guard itself is green under cozytest with bash and with dash as
`/bin/sh`, under the `bats` binary, and with mawk, gawk and BSD awk.
Each branch of its separator class and of the `time` and `!(cmd)`
handling was removed in turn, and each removal turns the shapes test
red.
`e2e-install-cozystack.bats` is an e2e suite and runs only in the E2E
workflow, so its one changed absence check is not exercised locally.
### Screenshots
Not a UI change.
### Downstream repositories
Walked the trigger map against the diff file by file. The change touches
eighteen existing BATS files, adds one, and adds a convention to
`docs/agents/e2e-testing.md`. It does not touch
`hack/e2e-prepare-cluster.bats`, which is the node contract
`ansible-cozystack` and `talm` restate; it moves and renames nothing
under `hack/` and changes no make target's behaviour, which is what
`ccp` gates on; and it touches no package, schema, enum, default, CRD,
label, annotation or metric.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
NONE
```
Closes #3565
|
||
|
IvanHunters
|
94abf6940b |
feat(kubernetes): let kubevirt-csi detach without the HotplugVolumes gate (#4451)
## What this PR does This lets kubevirt-csi detach volumes without the deprecated KubeVirt `HotplugVolumes` gate. The default stays as it is, so nothing changes at runtime yet. Part of #4032. The driver got a switch upstream for exactly this, and I bumped to it. kubevirt/csi-driver#382 added `--enable-vmi-hotplug-fallback`, merged as cb4770e4e475. With it set to false, the driver no longer calls the VMI-level `removevolume` on a VMI owned by a VM. virt-api rejects that call once `HotplugVolumes` is off. Our wrapper now passes the flag to upstream and no longer has its own hot-unplug path. `unpublishHotplugVolume` was a backport of kubevirt/csi-driver#184, and the previous pin already contained that change. Upstream detaches from the VMI when the VM is gone, then waits for the VMI status to drop the volume before it returns. So the wrapper's re-check never found the volume attached. Adding the ownership check there would have been dead code. Closes #4450 The chart leaves the flag at its default of true and does not render it. The driver image in the chart is stamped only at release preparation, so between releases main runs the previous release's binary, which does not know the flag. Go's `flag` package exits on an unknown flag. Rendering `=true` would crash-loop the CSI controller for no effect. The right value depends on the KubeVirt CR, not on the tenant, so the comment next to `HotplugVolumes` in `kubevirt-cr.yaml` points at the controller Deployment. The follow-up PR drops the gate and renders the flag as false. Nothing changes on the node side. Main already pins f5de10d39556, and cb4770e4e475 is that commit plus kubevirt/csi-driver#382. A Go unit test runs an unpublish through the wrapper during a teardown, with the flag on, off with a VM-owned VMI, and off with an orphaned VMI. It checks that the flag reaches upstream and that only upstream makes the VMI-level call. Upstream's own tests cover the ownership rule. A second test holds that an NFS volume only releases the node from its CiliumNetworkPolicy and never reaches the upstream hot-unplug. A helm-unittest pins that the chart renders no such argument. I haven't built the image or run e2e. Volume attach and detach on a tenant cluster in e2e is the real check. ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(kubernetes): kubevirt-csi supports --enable-vmi-hotplug-fallback, so it can detach volumes without the KubeVirt HotplugVolumes feature gate. The flag stays true for now and behaviour is unchanged. ``` |
||
|
Aleksei Sviridkin
|
2df578bffd |
test(e2e): read the opensearch release history through the pinned name
The opensearch suite wrote the history jsonpath out as a literal, so the contract test that runs HELMRELEASE_HISTORY_JSONPATH against the upstream Flux type never saw the expression this suite hands kubectl. A field rename upstream would have reached it only as an empty read. The suite now takes the shared expression, and the contract test reads every Chainsaw suite that calls the guard, so a suite copying the literal, or assigning the shared name after sourcing the library, fails in the unit lane instead of going unchecked. The read is matched inside the script step that calls the guard, because every step runs in a shell of its own and a read in another step never reaches the call. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
091b8549c2 |
build(images): run the tests of the packages the patches touch (#4326)
## What this PR does Twelve first-party images that are built from an upstream tree with patches on top now run `go test` on the packages those patches touch. The step sits in the builder stage, right after the patches are applied. Before this, ten of them ran `go build` only, and `go build` never compiles a `_test.go`, so a test carried inside a patch was proven once by hand and never again. The other two, opensearch-operator and kube-ovn, ran a `go test` line written by hand, which a patch reaching a new directory would slip past; the loop replaces both lines. The package list is read from the patches themselves with `git apply --numstat`, so a new patch is covered without an edit to the Dockerfile. Each package is tested from its own directory, which finds nested modules: in objectstorage, `internal/` is a module of its own and the root module cannot name `./internal/runtime`. An empty list runs nothing. The list is read into a variable before the loop, because a `for` loop drops the exit status of a command substitution, and a patch that git cannot read would otherwise give an empty list and a green step. A few images need more than the plain loop: - opensearch-operator skips `controllers/`. A patch touches it, but its suite starts an envtest control plane from etcd and kube-apiserver binaries the builder does not have. The loop runs with `-race` and replaces the hand-picked `go test -race` line, which also covered `pkg/reconcilers/util`, a package no patch touches. - kube-ovn drops its `go test -run '^TestVMIMigrationSchedulingRetry$'` line. The test it named is copied into `pkg/controller`, which a patch touches, so the loop runs it along with the rest of that package. - redis-operator also runs `go vet -tags integration` in each touched directory. Two of the files its patches edit are upstream integration tests behind that tag, and `go test` compiles none of them. - cluster-autoscaler and kubevirt-cloud-provider export `GOOS` and `GOARCH` for the target in the whole builder stage, so their tests unset both. A local `make image` builds amd64 and arm64 by default, and a test binary for the other architecture runs on the build host only under binfmt emulation. CI is not affected either way: the jobs that build package images each pin one `PLATFORM` that matches the runner's architecture. The first run of the step on cluster-autoscaler found the carried `fix-downscale.diff` out of step with the upstream tests it leaves in place. The patch removes the `maxSize` refusal from `SetSize`, and one case of `TestNodeGroupIncreaseSizeErrors` still expected that refusal. The patch now drops that case and carries `TestNodeGroupDeleteNodesAboveMaxSize` instead. That test covers the downscale the patch is named for: `DeleteNodes` lowers a group through `SetSize`, so with the refusal in place a group standing above its max size could not shed a node. Against the unpatched `SetSize` the new test fails. The image's behaviour does not change, and scale-up stays bounded by the core scale-up orchestrator, which skips a node group already at its max size. CI builds all twelve images, and the step reports `ok` or `no test files` for every package it tests. Locally, the redis-operator, opensearch-operator and kube-ovn builder stages pass. The cluster-autoscaler package was checked from the `cluster-autoscaler-1.32.0` tag tarball, because the Dockerfile's full clone of kubernetes/autoscaler does not survive my network. As negative controls, an undefined call added to a redis integration test fails `go vet -tags integration` while `go test` still passes, and the new autoscaler test fails without the patch. Left out on purpose, and the Dockerfile says so: `multus`. Its only touched package, `pkg/multus`, creates network namespaces in its tests, which a build container cannot do (`mount --make-rshared /var/run/netns: operation not permitted`). `cilium` (C, no compile step) and `piraeus-server` (Java) are out of scope. Closes #4318 ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff: Dockerfiles under `packages/*/images/` and one patch file. No values key, schema, make target, `hack/` file or documented workflow changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` |
||
|
Aleksei Sviridkin
|
f32e0693c1 |
ci(dashboard): run the console's eslint in the UI test job
The console defines a lint script next to typecheck and test, but no workflow ran it, so new findings landed unseen next to the old ones. The backlog is cleared down to three react-hooks/exhaustive-deps warnings. eslint exits zero on warnings alone, so the lint script caps them at that count: the step gates from its first run, and a new warning fails it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4d3796e41d |
refactor(dashboard): mark the deliberate setState calls inside effects
Each of these effects re-syncs local state to something that changes outside the component. The rule's suggested rewrite derives the state during render instead, which changes when the new value is committed and what the first render shows, so each site carries a scoped disable with its reason rather than a rewrite. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
be9b148e6c |
refactor(dashboard): move non-component exports out of component files
Vite's fast refresh can only hot-swap a module that exports nothing but components; a file that also exports a hook, a helper or a registry object forces a full page reload on every edit to it. Hooks, context objects and the RJSF registry now live in their own modules, and the components stay where they were. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f1281e6176 |
refactor(dashboard): drop the unused ad binding in OverviewTab
The tab never reads it; the prop stays in the interface because the detail page passes the same props to every tab. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
8298c43e05 |
refactor(dashboard): type command palette instances as K8sResource
client.list was called without a type argument, so its items came back as unknown and were cast to any to read metadata.name. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
05ff14dbc1 |
refactor(dashboard): type the RJSF templates and source fields instead of any
RJSF's generic defaults are any, and the templates restated them, so every schema read went untyped. The templates now default to unknown and FormContextType, and the source widgets read option schemas through a small local interface. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
def917a5d5 |
refactor(dashboard): type the backup form pages instead of any
The console's own rules forbid any, and eslint flags every one of these. The create pages only read a handful of reference fields, so a local interface per page covers them; the rest of the spec still passes through from RJSF untouched. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
534f8f1db9 |
docs(backups): stop presenting admin-only demo steps as tenant work
The kafka and nats walkthroughs put steps 03, 04, 06, 07 and cleanup under the tenant persona, and the scenario docs and helper comments had the tenant create the backup Secret. The scripts read, create and delete Secrets, wait on HelmReleases, COSI objects and operator resources, exec into a CLI Pod, read Pod logs, and delete the Backup and the cluster-scoped BackupClass and strategy. The tenant roles grant none of that, so a tenant following the docs hit Forbidden at the first read. The tables now say admin and name the part a tenant admin can do on its own, and the scenario docs say the same. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
874e1e44b7 |
fix(backups): run the kafka-metadata demo CLI Pod as restricted
The kafka-metadata walkthrough started its long-lived CLI Pod with a bare kubectl run, so the Pod carried no securityContext and a namespace enforcing the restricted Pod Security level would reject it. The kafka topic-data walkthrough already layers the restricted context onto its CLI Pod; the metadata demo now applies the same JSON patch. A running CLI Pod left by an earlier run without that context is replaced rather than reused, since a Pod's securityContext cannot change in place. The new bats cases read the overrides kubectl run receives and fail unless both the Pod-level and the container-level fields are present and the patch is sent as --override-type=json, and check that an owned Pod without the restricted context is recreated. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
cc731e3105 |
test(e2e): keep redis-tls failures legible under the op ceiling
The CA-projection step caps each kubectl at 30s, but more capped reads run than its 5m op can cover, so a degraded apiserver could spend the ceiling and leave nothing but "signal: killed". The reads now share a step budget held under the op, and a read that hits its bound or finds the budget spent says which read it was. Two exec checks reported the 15s redis-cli cap for any failure, sending a reader after a slow redis when the exec was refused or the container was missing. They now report the exit status and what the exec printed. The step also runs under bash with pipefail: under dash a failed Secret read took base64's zero status and surfaced as a wrong source name. The private-key check reads the projection already fetched, since a second read that failed made it pass. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a057d738d1 |
test(e2e): give the gateway listener-set op room for its own waits
The step's polls can wait 660s under a 6m op, and its cleanup trap deletes two tenants with no bound on the wait. When the op ceiling fires first, Chainsaw kills the shell: the poll that was running never reports, the trap never runs, and both tenants stay behind for every later suite. The deletes keep waiting, child before parent, since a parent deleted while its child is still uninstalling wedges its own cleanup. Each delete runs under an outer timeout with a kill grace, because kubectl's --timeout bounds only the wait after the request and not discovery or a stalled request. The parent is deleted only once the child is gone. A failed tenant delete fails the step with kubectl's error, its exit status and the tenant left behind, rather than passing with the leak. The op is sized above the polls plus those bounds, and a unit guard holds it to that sum and the cleanup to that order. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b1085782c7 |
test(e2e): fail the isp-hosted no-Cilium check on any cilium.io CRD
The check looked for one CRD by name, so a cluster carrying only part of Cilium's CRDs passed as a cluster without Cilium, and a failed lookup read as an absent CRD and passed too. It now lists the CRD groups once, fails if that list cannot be read, and fails on any CRD in cilium.io. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
bd504e0b4e |
ci(e2e): run the isp-hosted install daily against the nightly
The lane installs the latest GHCR nightly on a kind cluster once a day, after the nightly has published. It takes no pull request trigger: a run would install the nightly's images, not the pull request's, and report on code the pull request did not change. The unit test holds the lane's wiring in place, since a lane that quietly installed another variant would test something other than its name says. The sandbox headroom check reads any workflow touching packages/core/testing as one that boots the QEMU guests; this lane starts only the sandbox container, so it is exempted by name, after the job match, and is sized like the others if it ever starts calling the guest targets. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
3e36b35795 |
test(e2e): add an install suite for the isp-hosted variant on kind
isp-hosted shipped unable to converge and nothing noticed, because no lane installs it: every e2e lane runs isp-full on Talos, where Cozystack provides the CNI and storage itself. This suite installs it the way it is meant to run, through the hosted installer variant on a cluster that brings its own CNI, kube-proxy and default StorageClass, and passes only when every HelmRelease is Ready and the variant really left out the data plane it delegates to the host. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
IvanHunters
|
309875ab62 |
test(controller): seed the DataVolume informer for controller-runtime 0.25 (#4729)
## What this PR does `main` does not build `internal/controller` tests, so `Unit & controller tests` is red on every open PR. #4380 added DataVolume watch tests that set `FakeInformer.Synced` and lock `SyncedLock`. #4473 then moved to controller-runtime v0.25.1, where `FakeInformer` keeps its sync state in a private channel and exposes only `Synced()` and `HasSynced()`. Each PR was green on its own base. The tests now seed the DataVolume informer into `FakeInformers.InformersByGVK` and call `Synced()` when they want it synced. Seeding is needed because `FakeInformers` builds every informer it creates itself already synced, so there is no way to get an unsynced one from it otherwise. The unsynced cases still assert that no reader is installed, so they would fail if the informer came up synced. `go vet ./...` is clean and `go test ./internal/controller/` passes. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved automated checks for workload monitoring across different data synchronization states. These changes strengthen validation of watch behavior without changing functionality visible to users. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
2eeaa59055 |
test(controller): seed the DataVolume informer for controller-runtime 0.25
controller-runtime 0.25 made FakeInformer's sync state a private channel, so the DataVolume watch tests no longer compile: they toggled the exported Synced field and its lock. FakeInformers also builds every informer it creates already synced, so an unsynced informer has to be seeded into InformersByGVK and marked synced with Synced() later. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
89a1fe52d9 |
build(multus): say why the image build runs no tests of the patched package
The other patched Go images now test the packages their patches touch in the builder stage. The one package the multus patch touches, pkg/multus, creates network namespaces in its tests, which a build container is not allowed to do, so that image keeps building without a test step. The Dockerfile records the reason, so the absence reads as a decision and not as an omission. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f51be95430 |
build(objectstorage-controller): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Each package is tested from its own directory rather than as a ./... path of the root module, because internal/ is a module of its own and the root module cannot name ./internal/runtime. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b2a5f12e73 |
build(kubernetes): run the tests of the packages the patches touch
The cluster-autoscaler and kubevirt-cloud-provider image builds ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Both builder stages export GOOS and GOARCH for the target, so the tests unset them: a local make image builds amd64 and arm64 by default, and a test binary for the other architecture runs on the build host only under binfmt emulation. The first run of that step on cluster-autoscaler found the carried downscale patch out of step with the upstream tests it leaves in place: SetSize no longer refuses a size above maxSize, while one case of TestNodeGroupIncreaseSizeErrors still expected that refusal. The patch now drops that case, and carries a test for the downscale it is named for instead. DeleteNodes lowers a group one node at a time through SetSize, so with the refusal in place a group standing above its max size could not shed a node at all. The new test deletes a node from such a group and expects the replica count to drop; against the unpatched SetSize it fails with the refusal the patch removes. Scale-up past maxSize stays bounded by the core scale-up orchestrator, which skips a node group already at MaxSize. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
070d235d1c |
build(kube-ovn): run the tests of the packages the patches touch
The image build ran one test by name, the one shipped next to the migration patch, and go build never compiles a _test.go, so the rest of the tests in the packages the patches change never ran. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. It replaces the hand-picked go test line: the shipped test lives in pkg/controller, which a patch touches, so the loop runs it with the rest of that package. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
59c3174b44 |
build(kamaji): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
0b00b69267 |
build(capi-providers-cpprovider): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1ffc8580a4 |
build(keycloak-operator): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e2757ba87b |
build(redis-operator): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Two of the files the patches edit are upstream integration tests behind the integration build tag, which go test compiles none of, so each directory is also vetted with that tag. Doing it per derived directory rather than for the one directory those tests live in today keeps a later patch to a tagged test elsewhere covered. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
0a606815ab |
build(kilo): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a3a7d66bdf |
build(linstor): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a73b91e232 |
build(bucket): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a _test.go, so nothing ran the tests of the code the patches change. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
293a780938 |
build(opensearch-operator): run the tests of the packages the patches touch
The image build ran go test on two package trees listed by hand, and go build never compiles a _test.go. A patch that reaches a directory nobody listed brings tests that are proven once, by hand, when the patch is cut, and nothing notices them failing after the next version bump. Run go test in the builder stage on the packages the patches touch. The list is read from the patches themselves, so a new patch is covered without an edit to the Dockerfile. It replaces the hand-picked go test line, keeping its -race flag: two lines testing the same packages is one too many. The hand-picked line also covered pkg/reconcilers/util, which no patch touches. controllers/ is left out. The stall-tracing patch touches it, and its suite starts an envtest control plane from etcd and kube-apiserver binaries the builder does not have, so testing it fails the build. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
bf03c84711 |
chore(deps): move to the Kubernetes 1.37 libraries without the apimachinery fork (#4473)
## What this PR does This moves the root module's `k8s.io/*` from v0.35 to v0.37.1 and controller-runtime from v0.23 to v0.25.1, and drops the `k8s.io/apimachinery` fork on the way. Supersedes #3001, #4467 and #4472; those were a stack of the same commits. The four commits are meant to be read one by one, and each builds and passes the tests on its own: 1. **Plain watches stop getting the initial-events-end bookmark.** The apiserver defaults `sendInitialEvents=true` onto any watch with an empty or `0` resourceVersion and leaves bookmarks off, and our aggregated storages took that flag alone as a WatchList request. From `k8s.io/apiserver` v0.36 the watch handler compiled into cozystack-api panics on that bookmark, so `kubectl get -w` on any cozystack-api resource would die after the bump. The marker now needs both flags, the same condition the watch cache uses. There are tests per storage and one through the real handler chain. 2. **The apimachinery fork goes.** Every apps kind shares the `Application` Go type, and stock `Scheme.ConvertToVersion` picks the first kind registered for it. The fork patched that; the apps group now encodes through a serializer that keeps the kind the object carries. A handler-chain test covers create, get, list, update, watch, apply and table for two kinds and fails without it. 3. **MutatingAdmissionPolicy stays out of admission.** From v0.36 the plugin is on by default and refuses every write until it has synced `mutatingadmissionpolicies`, which cozystack-api has no RBAC to list and which clusters before 1.36 do not serve. Without this, every create, update and delete would fail after the bump. It is also re-applied in `Complete`, since `--disable-admission-plugins` replaces the list. 4. **The bump itself**, with the regenerated clients, openapi and strategy CRDs. Commits 2 and 4 are where the review effort goes; 1 and 3 are small. Known changes in behaviour: - Without the fork, the row objects of a Table requested with `includeObject=Object` carry the first registered app kind. kubectl asks for that only with `--sort-by`, and it sorts on fields, not on the kind. Nothing at this layer can fix it, because the handler converts through `APIGroupInfo.Scheme`, a concrete `*runtime.Scheme`. Admission webhooks and ValidatingAdmissionPolicy still see the right kind. - The alpha PodSpec field `workloadRef` became `schedulingGroup` upstream. Once the new strategy CRDs are applied, the apiserver prunes `workloadRef` from stored strategies it serves, and their next write erases it. It only ever worked behind an alpha feature gate. The branch is rebased onto the Renovate rollup (#4447), with `go mod tidy` and codegen rerun on top of it. ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note chore(deps): cozystack-api and the controllers build against the Kubernetes 1.37 libraries without the apimachinery fork. Plain watches no longer receive the WatchList initial-events-end bookmark, and objects in Table rows requested with includeObject=Object may carry another app's kind. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Application API responses now preserve the requested resource kind across operations, including lists and watches. * Backup strategy resources support additional Kubernetes pod settings, including probe protocols, volume ownership options, and scheduling groups. * Backup strategy schemas include updated Kubernetes pod fields and descriptions. * **Bug Fixes** * Initial-event watch bookmarks are sent only when requested; plain watches continue without an ending bookmark. * Mutating Admission Policy is disabled by default, including when admission plugin options are customized. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
9809b22280 |
fix(vm-disk): report DataVolume readiness through WorkloadsReady (#4380)
## What this PR does Nothing on a vm-disk application tells whether CDI ever finished populating the disk. Its WorkloadMonitor has no pods to count, and `Ready` follows the Helm release. So an import or clone that never completes can leave a green application over a disk nobody can use. cozystack/cozystack#2908 is one way to get there: a clone into a StorageClass that cannot place a replica where the source data sits. The expensive part there was the silence, not the stuck clone. The disk state was in the DataVolume phase all along, and nothing read it. With this change the WorkloadMonitor in cozystack-controller reads it. The monitor lists the DataVolumes its selector matches and reports not operational while any of them is in flight, Failed, Unknown or has no phase. An upload counts as in flight until it completes. That surfaces on the application as `WorkloadsReady=False` with reason `DataVolumeNotReady` and a message that names the DataVolume and its phase, for example `DataVolume vm-disk-x is ImportInProgress`. `Ready` still follows the HelmRelease, as for any other kind (`convertHelmReleaseToApplication` in `pkg/registry/apps/application/rest.go`), and for a vm-disk it no longer waits on the DataVolume at all. So the disk state has to be visible outside `Ready`, and this PR puts it where operators look by default: - `kubectl get` on any application kind gets a `WORKLOADS` column next to `READY`. It shows the `WorkloadsReady` status, or `<none>` when the application has no WorkloadMonitor. - The console badge turns amber and reads `DataVolumeNotReady` while `Ready` is True and `WorkloadsReady` is not True for that reason. A failing `Ready` still wins. The console shows only the disk case: other kinds need a separate decision, because a stopped VM legitimately has no pod and reports `WorkloadsReady=False` too. A Helm readiness gate on the DataVolume would be the obvious alternative, and it is not used on purpose. helm-controller runs that wait inside its reconcile, so an importing disk would hold one of the shard's worker slots for the whole install window, and a disk stuck for good would keep taking the slot back. cozystack/cozystack#4145 has the details. The monitor costs no slot, keeps watching after install, and covers existing disks without anyone writing to them. Without any gate, the install would still wait on the DataVolume. helm-controller's default poller falls back to a resource's `Ready` condition for kinds it has no rule for (`checkReadyCondition` in fluxcd/cli-utils `pkg/kstatus/status/status.go`), so a vm-disk install could hold a worker slot for the whole install timeout. The ApplicationDefinition therefore gets a `healthCheckExprs` entry for DataVolume with `current: "true"`, the same idiom postgres-rd uses for its ScaledObject. With it Helm never waits on the DataVolume. The entry is pinned by a helm-unittest in `packages/system/vm-disk-rd`, which gets a `test` target for it. Release settings reach a HelmRelease only when the application is created or updated, so an existing disk gets the entry on its next update. The reconciler that syncs definitions onto existing releases copies chartRef, valuesFrom and labels only. That affects any kind with release settings, so it is filed as cozystack/cozystack#4409 and not fixed here. `WorkloadsReady` covers every disk right away, since the monitor needs no write to the application. CDI copies the DataVolume labels onto the PVC it creates and makes the DataVolume its controller, so the monitor also publishes a `pvc-<name>` Workload for the disk. That Workload reads not operational while its DataVolume is not ready, even though the PVC is already Bound, so `kubectl get workloads` agrees with `WorkloadsReady`. While the DataVolumes cannot be read it keeps the verdict it has, like the monitor. The WorkloadMonitor CRD now requires at least one selector label. An empty selector would match every Pod, PVC, Service and DataVolume in the namespace. No chart renders one. When an application has several monitors that are not operational and only some of them name a cause, `WorkloadsReady` carries the named messages and reason only. A vm-disk has one monitor, so it does not hit this. ### Which phases count as ready Settled phases pass: `Succeeded`, `PendingPopulation`, `WaitForFirstConsumer`, `Paused`. A disk with `bindImmediately: false` on a `WaitForFirstConsumer` class rests in `PendingPopulation` or `WaitForFirstConsumer` until a VM consumes it. A multi-stage import rests in `Paused` until its next checkpoint, and the vm-disk chart renders no checkpoint source. These report ready while holding no data, because that is where such a disk is supposed to sit. `UploadReady` is not one of them. CDI enters it when the upload server pod is running and ready, and leaves it for `Succeeded` only when that pod succeeds (`updateStatusPhase` in `pkg/controller/datavolume/upload-controller.go` at v1.66.1), so the one phase covers a disk nobody has uploaded to and a half-uploaded one. A `source: upload` disk therefore reports `WorkloadsReady=False` from creation until `virtctl image-upload` completes. A phase the controller does not know also reads as ready, so a CDI release that adds or renames a phase cannot turn working disks red. The cost goes the other way: a renamed in-flight phase silently reads as settled. The list wants a re-read whenever the CDI version moves. It matches the named phases of the CDI v1.66.1 API, the version vendored in `packages/system/kubevirt-cdi-operator`. ### Clusters without CDI CDI comes with the IaaS bundle, while cozystack-controller runs everywhere, and CDI can be installed while the controller is running. A watch on a kind whose CRD is absent never syncs, and then the whole controller does not start. So the watch is not set up at startup. A runnable adds the DataVolume watch once the kind is served. It waits for the first reconcile, because only from then on does a new watch start at once, and then checks discovery right away and every 30s after that. It installs the DataVolume reader only after the DataVolume informer has synced within a timeout, because a reconcile context has no deadline and a List on an informer that never syncs would block every WorkloadMonitor reconcile. The watch source is started once: a controller-runtime `Kind` source adds an event handler to the informer and never removes it, so on a timeout the next check waits on the source it already has instead of starting another one. RBAC needs no change: the controller's ClusterRole already has read on everything. When the DataVolumes cannot be read, because the reader is not installed yet or the List fails, the monitor still publishes its replica counts and keeps its last DataVolume verdict, and a List error is returned so the request is retried and recorded as a `DataVolumesUnavailable` Warning event on the monitor. The status of a monitor without DataVolumes is not affected by a failing List. A disk whose verdict is already recorded does not read as ready in the window after a controller restart. A monitor with no recorded verdict yet, which is every monitor right after the upgrade that brings this change and any disk created right after a restart, reads ready until the DataVolume watch has synced. Because the watch starts on the first reconcile rather than on the next discovery tick, that window is the time to sync the DataVolume informer plus at most one 5s requeue, not a 30s poll interval. The window is not zero, since a monitor reconciled in it is published before its DataVolumes can be read. From then on, a reconcile that runs before the watch has synced is requeued after a few seconds, so the DataVolume events that queue it are not lost. A pass that changes nothing in the monitor status does not write it, so that requeue sends no status writes to the API server. Once the reader is installed, every monitor that holds a DataVolume verdict is queued once more. So a verdict whose DataVolume was deleted while the controller was down is recomputed then, not at the periodic resync. ### Tests The table test in `workloadmonitor_controller_test.go` covers the named phases of CDI v1.66.1, plus Failed, Unknown, an empty phase and a missing status. A second table checks the message and reason for an importing, a failed, a phaseless and a populated disk, and a third test lists two stuck DataVolumes in both orders and expects one message. It also covers a DataVolume outside the selector, the kind not being served, a List error that still publishes the replica counts, and a missing reader that keeps the last verdict. The watch tests check that the watch starts on the first reconcile and not before it. A source that never syncs leaves no reader and is retried on the same source, with the reconciles in between still requeued. A List failure records the Warning event, the sync window records none, and an unchanged pass writes no status. A manager that stops during the wait gets the cancellation back, and a reconcile that runs while the watch syncs is requeued without an error. A table test covers the PVC Workload: importing, populated, a PVC with no DataVolume controller or a controller of another kind or group, and the unread cases. A helm-unittest in `packages/system/cozystack-controller` pins the selector's `minProperties`. `rest_table_test.go` pins the `WORKLOADS` column, `rest_workloads_test.go` the condition message and reason, and `status.test.ts` the console badge, including a halted VM that stays green. In `hack/e2e-chainsaw/vminstance` the existing VMDisk scenarios now wait for the DataVolume to be Ready, then for `Ready` and `WorkloadsReady` on the VMDisk itself. That replaces the wait on the HelmRelease. Those asserts would pass even if the monitor ignored DataVolumes, so a new scenario imports from an `.invalid` URL. It first waits for the importer to have failed, which CDI reports on the DataVolume as `Running=False` with reason `Error`, and then for `Ready=True` with `WorkloadsReady=False` and reason `DataVolumeNotReady` on the VMDisk. Without the first wait the VMDisk assert passed in about a second, on a DataVolume that had no phase yet. The e2e runs on a fresh install only, so nothing here exercises an existing release across an upgrade. ### What it does not do #2908 lists possible fix directions, and none of them is here. The cross-class clone still never completes, it just stops being invisible. Keep the issue open. While the DataVolume watch syncs, every monitor is still requeued. Only the ones whose selector reaches a DataVolume need it, but the controller cannot tell which those are before it can read DataVolumes. Closes #4449 ### Screenshots There is no screenshot: I had no running console to take one from. The badge change is pinned by the vitest cases in `packages/system/dashboard/images/console/apps/console/src/lib/status.test.ts`: a VMDisk whose `WorkloadsReady` has reason `DataVolumeNotReady` goes amber, and a halted VMInstance with the generic reason stays green. ### Downstream repositories I walked the trigger map against the diff. cozystack/terraform-provider-cozystack is affected: its `wait_for_ready` reads only `Ready`, so on a `cozystack_vmdisk` it returns before the disk is populated once this lands. That needs a decision on the provider side, so it is an issue there. The release-time bot regenerates `packages/apps/vm-disk/README.md` into the website's virtualization pages, and `vm-disk` is in the `VMS` list that generator reads, so the website needs no follow-up. The ApplicationDefinition keeps its kind, plural, prefix and path. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: https://github.com/cozystack/terraform-provider-cozystack/issues/59 - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(vm-disk): report whether CDI finished populating a disk in the WorkloadsReady condition, which names the stuck DataVolume and its phase. kubectl get shows WorkloadsReady in a new WORKLOADS column for every application kind, and the console badge turns amber while a disk is not populated, so a disk whose import or clone never completes no longer looks healthy. Ready of a new or updated vm-disk no longer waits for the DataVolume, so wait on WorkloadsReady to know the disk is populated. ``` |
||
|
IvanHunters
|
dad5924b21 |
fix(computeplane): do not grant tenants the cluster admin kubeconfig (#4722)
## What this PR does A tenant user could read the admin kubeconfig of their own ComputePlane cluster, which the module promises never to hand out. The module deploys the `kubernetes` app into the tenant namespace as `computeplane-cluster`. The secrets allowlist kept the Secret out of `tenantsecrets`, but the chart's dashboard Role still gave the tenant's `use` subjects get/list/watch on `computeplane-cluster-admin-kubeconfig`. So a plain `kubectl get secret` worked. The fix drops that one rule for the ComputePlane cluster only. The kubernetes chart now skips it when the internal value `withholdAdminKubeconfig` is set, and the computeplane module sets that value inline in the cluster HelmRelease. The tenant can't reach those values, because the module reads only `nodeGroups` and passes an allowlisted subset of it to the pool releases. The value is kept out of `values.yaml` and the schema, same as `clusterReleaseName` in kubernetes-nodes. A regular Kubernetes app keeps the rule, so the dashboard still offers its kubeconfig. Only the v1.7.0 alphas ship the module. After this lands, the `kubectl auth can-i` check can go back into the e2e suite in #4713. ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(computeplane): tenant users no longer get read access to the ComputePlane cluster's admin kubeconfig Secret through the kubernetes app's dashboard Role. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Security** * Computeplane dashboards no longer grant access to the cluster’s admin kubeconfig Secret. Other dashboard resource permissions remain available. This limits access to sensitive cluster credentials through the dashboard while preserving the service-related permissions needed for dashboard resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
ded2354b65 |
fix(tenant): skip Cilium policies on isp-hosted without Cilium (#4710)
## What this PR does The tenant chart no longer fails to install on a hosted cluster without Cilium. The isp-hosted variants install no CNI of their own, but the chart rendered its Cilium policies unconditionally, so `tenant-root` failed with `no matches for kind "CiliumNetworkPolicy" in version "cilium.io/v2"`. Everything that depends on `cozystack-basics` then never converged. This also unblocks the isp-hosted e2e lane, which fails on exactly this. The guard is keyed on the platform variant. The platform already passes it to every tenant as `_cluster.bundle-name`. It covers every Cilium object the chart renders, including the shared-central tracing egress policies. - On variants where Cozystack installs Cilium itself, nothing changes and a missing CRD still fails the install. - On `isp-hosted` and `isp-hosted-slim`, the policies render only when the host serves both Cilium policy kinds, since a hosted cluster may run Cilium. The render on Cilium variants is byte-identical to `main`. I checked it with `helm template` over all four Cilium variants and an absent `_cluster`, with optional services on and off, with and without the Cilium API, for root, child and grandchild tenants. I didn't use the capability check alone because it fails open. helm-controller upgrades a deployed release only when the chart or the values digest changes, so a change in API discovery never triggers a re-render. That is `VerifyRelease` in `internal/action/verify.go` at v1.5.0, the version Cozystack ships. If the Cilium CRDs were not discoverable at install time on a Cilium cluster, a capability-only guard would leave the tenant without policies until the next chart or values change. An upgrade rendered in that window would delete existing ones. On isp-hosted without Cilium, tenants get no network isolation from Cozystack. Isolating them is up to the host. I didn't fall back to standard `NetworkPolicy`, because the host CNI may not enforce it. The tenant policies also use Cilium features that don't map one to one: the `world` and `kube-apiserver` entities, clusterwide policies, selectors on ancestor namespace labels. A partial translation would look like isolation without really being one. The tenant README now documents this, including that Cilium added to the host later takes effect only on the tenant's next chart or values change. The threat model and the self-assessment list it under non-goals. The new helm-unittest suite fails on `main` for the hosted cases without Cilium. Each guard mutation I tried turns at least one case red: no variant check, no capability check, `and` changed to `or`, `isp-hosted-slim` dropped from the list. I didn't add an explicit "require Cilium" value, since the Cilium variants already behave that way. One thing to keep in mind: the list of hosted variants now lives in the platform bundle and in the tenant helper, so a new hosted variant has to go into both. A platform `_cluster` flag saying whether Cozystack manages the CNI would remove that, but it's a platform change. Closes #3387 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The tenant README change reaches the website through the release bot that regenerates the managed-apps pages from each package README. ### Release note ```release-note fix(tenant): the tenant chart no longer fails to install on the isp-hosted variants when the host has no Cilium. There, the Cilium network policies are created only if the host serves the Cilium policy API; otherwise tenants get no network isolation from Cozystack. Other variants are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Tenant network-isolation policies are rendered for `isp-hosted` and `isp-hosted-slim` only when the required Cilium APIs are available. If Cilium becomes available later, policies are applied after a subsequent chart or values change. Tracing policies also require the Cilium APIs. * **Documentation** * Clarified when Cozystack provides tenant network isolation and that isolation is the host’s responsibility when the required Cilium APIs are unavailable. * **Tests** * Added coverage for policy rendering across bundle types and Cilium API availability. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
11910e0b2c |
fix(kubernetes-nodes): refuse a resources.cpu that cannot be a core count (#4386)
## What this PR does A worker pool's CPU value could land in the VM as something KubeVirt cannot accept. `resources.cpu` is typed as a quantity, but on the kubevirt substrate the template wrote it unchanged into `domain.cpu.cores`, which is `uint32` in KubeVirt and an integer in the KubevirtMachineTemplate CRD. With `resources.cpu: 500m` the chart rendered `cores: 500m`. That string was truthy, so it also counted as explicit sizing and dropped the pool's `instanceType` from the VM, and then the API server refused the template anyway. The render now decodes the value as YAML, the same way `cores: <raw>` is decoded, and stops unless the result is a whole number from 1 to 4294967295. `500m`, `"1.5"` and `"1e10"` now fail at render time with an error naming the field, before the value reaches the VM template or the instancetype decision. `"0"` and `"-2"` were already refused on `main` and still are. Only inputs that could not produce a working pool are refused. Every value that rendered a pool before renders the same bytes now, so the content-hashed template keeps its name and nothing rolls. That includes `2`, `"2.0"`, `"+2"` and `"1e1"`, and also two odd cases I kept on purpose because pools may already carry them. An unquoted `0` is falsy, so the pool renders without `domain.cpu` and is sized by its instancetype, as on `main`. A leading-zero value such as `"010"` is still emitted as written and keeps the reading it gets on `main` today, where YAML treats it as octal. I checked both against a render of `main`, and the tests pin the template names that `main` produces for them. This is where kubernetes-nodes and vm-instance now differ. Since #4405, vm-instance refuses a leading zero and an unquoted `0`, and this chart does not. Stored worker pools can carry these values, because migration 56 copies `resources` verbatim, so refusing them here without a migration would fail those pools' releases on their next upgrade. #4554 tracks the migration and the refusal after it. The proxmox substrate is not checked. It converts the quantity to millicores and rounds up to whole cores, so `500m` is a valid size there and becomes one core. The field description now says what the value means on each substrate. Memory is left alone. It goes to `domain.memory.guest`, which is a `resource.Quantity`, so a quantity string is valid there. The issue suggested coercing to an integer, as vm-instance did before #4405. I didn't, because `int64` turns `500m` into `0` without an error. The new helm-unittest suite covers each input form, both preserved cases, the upper bound, the instancetype case and the proxmox path. Two neighbouring descriptions still say only "CPU available.": `nodeGroups[].resources.cpu` in `packages/extra/computeplane` and in the parent `kubernetes` chart. computeplane passes `resources` straight into this chart, so the new check applies to its pools as well. I left both alone to keep this change to one chart. Closes #4150 ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kubernetes-nodes): on the kubevirt substrate, a worker pool whose resources.cpu cannot be a whole vCPU core count (for example 500m, 1.5 or a value above 4294967295) now fails at render time with a clear error, instead of rendering a VM template that is refused later or dropping the pool's instanceType. Values that rendered a working pool before render unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * KubeVirt CPU settings now accept whole-core values from 1 to 4,294,967,295. Fractional, millicore, negative, and larger values are rejected; an unquoted `0` is treated as unset. * Proxmox rounds fractional and millicore CPU values up to whole cores. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
fb7e780517 |
fix(postgres): reassign dropped roles' objects to an unprivileged role (#4574)
## What this PR does Objects owned by a role the Postgres init-job drops no longer end up owned by the `postgres` superuser. This covers a user removed from `users` and an orphaned managed `<db>_admin` or `<db>_readonly` role. Before dropping such a role the job reassigned everything it owned to `postgres`, and objects a tenant role created should not become superuser-owned. They now go to `cozystack:orphaned`, a NOLOGIN role that is granted nothing. The init-job creates it and resets its attributes on every run. The `:` in the name is outside the charset allowed for user and database names, so values cannot declare it. There is a trade-off. Grants and memberships of the dropped role do not carry over, so a view or function it owned that ran with its owner's privileges can lose access it had through them. To keep such objects working, keep the user in values or move the objects to another owner before removing it. `docs/operations/backup-classes.md` is updated accordingly. Role names read from the catalog now reach SQL as psql variables instead of being spliced in by the shell. The queries are echoed after substitution, so the job log shows which role is dropped. Tested with helm-unittest and by running the rendered init script against PostgreSQL 13 and 18. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(postgres): objects owned by a user removed from `users`, or by an orphaned managed database role, are now reassigned to the unprivileged `cozystack:orphaned` role instead of the `postgres` superuser. A view or function of the removed role that ran with its owner's privileges can lose access it had through that role's grants or memberships. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified that after restore, the first initialization releases undeclared managed databases without deleting their data and removes users not declared in the chart. * Removed users’ objects are reassigned to a restricted, non-login role. Grants and memberships are not preserved; functions use invoker rights, and PostgreSQL 15+ views are set to use invoker security. * Added guidance for finding eligible objects still owned by the PostgreSQL superuser for manual reassignment. * Declare a user in `users` to keep it; the chart will generate a new password. * **Bug Fixes** * Removed users’ objects are reassigned across connectable, non-template databases before their users are dropped. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |