Commit Graph
7777 Commits
Author SHA1 Message Date
Aleksei Sviridkin cbbb69afc3 fix(platform): stop treating a failed ConfigMap read as a fresh cluster
run-migrations.sh decided whether cozystack-version exists by the exit
status of a silenced kubectl get, so a timeout or a dropped connection
counted as "no ConfigMap": the runner stamped the target version and
exited 0, and every pending migration was skipped for good. The hook Job
only renders when the ConfigMap already exists, so in practice an error
was the only way to reach that branch.

--ignore-not-found suppresses NotFound alone; any other error keeps its
non-zero status and now fails the pod, so the Job retries the read.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-07 12:06:18 +03:00
Myasnikov Daniil 34b1533a1a ci(labeler): map the local-storage scope to area/storage
The labeler falls back to area/uncategorized for a scope it does not
know, and re-applies the fallback on every run, so a manually fixed
label does not stay. The local-storage package is a storage component,
and its PR title scope should land on area/storage.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 11:30:46 +03:00
Myasnikov Daniil d61fed46cc feat(local-storage): add opt-in OpenEBS LocalPV hostpath package
LINSTOR needs the DRBD and ZFS kernel modules, so a cluster without
them has no dynamic PV provisioning. OpenEBS LocalPV hostpath needs no
kernel modules and also serves deliberately node-local workloads.

The package is opt-in so it cannot change an existing cluster. Its
class is named cozystack-local-hostpath so it cannot collide with an
existing class name, and it is not the default so PVCs that omit
storageClassName keep their current binding. The vendored RBAC is
patched down to what volume reconciliation calls; helper Pods, Leases
and analytics state stay in the install namespace.

Images stay on docker.io, pinned by index digest.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 11:30:45 +03:00
Aleksei Sviridkin 6addef4d3c fix(kubevirt): route the export proxy to the Service KubeVirt creates (#4637)
## What this PR does

The VM export routes point at a Service that does not exist, and the
tenant network policy would block the Ingress even if it did.

KubeVirt creates the export proxy Service under the name
`virt-exportproxy` (`VirtExportProxyName` in
`pkg/virt-operator/resource/generate/components`, v1.9.0). The Ingress
and the TLSRoute that publish `vm-exportproxy.<host>` name a
`vm-exportproxy` Service, and nothing creates one. The backend was
`virt-exportproxy` when the Ingress was added in 0ddaff938. Then
3780925a6 renamed the hostname to `vm-exportproxy` and the backend along
with it. The hostname stays, only the backend goes back.

Renaming the backend is not enough for the Ingress, because it is served
by the ingress controller in tenant-root and tenant egress is
default-deny. Each allowed destination has its own `allow-to-*` policy.
The CDI upload proxy has `allow-to-cdi-upload-proxy`, the export proxy
had nothing. The new `allow-to-vm-exportproxy` policy renders for the
same tenants and the same source pods as the CDI one. Its destination is
narrower: only the `kubevirt.io=virt-exportproxy` pods in
`cozy-kubevirt`, on TCP 8443, the container port virt-operator gives
them.

The unit tests pin the backend name for the Ingress and the TLSRoute,
and the placement, selector and port of the new policy.

I checked this on a test cluster with KubeVirt v1.9.0 and ingress-nginx.
Without the fix the controller logs `Error obtaining Endpoints for
Service "cozy-kubevirt/vm-exportproxy"`. With the backend renamed and
the tenant allowed egress to virt-exportproxy, a passthrough request to
`vm-exportproxy.<host>` reaches the export proxy, and it answers with
its own serving certificate.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(kubevirt): requests to vm-exportproxy.<host> now reach the KubeVirt export proxy: the Ingress and TLSRoute route to the virt-exportproxy Service KubeVirt creates, and tenants are allowed egress to the export proxy.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Tenant workloads can now connect to the KubeVirt export proxy over TCP
port 8443.
* **Bug Fixes**
* Updated export-proxy Ingress and TLSRoute routing to use the
`virt-exportproxy` backend, keeping export-proxy traffic directed to the
correct service.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 11:19:32 +03:00
Aleksei Sviridkin 8c1a78dc9b feat(operator): default system namespace memory via LimitRange (#3494)
## What this PR does

System components now get a default container memory limit, so their pod
cgroups carry a `memory.max` and drop out of the Talos userspace OOM
handler's victim set. `cozystack-operator` maintains a LimitRange named
`cozystack-system-defaults` in every system namespace it reconciles.

The Talos handler (v1.12+) scores cgroups with `memory_max.hasValue() ?
0.0 : {Besteffort: 1.0, Burstable: 0.5, ...}[class] * memory_current`
and drops everything that scores zero. Only a memory limit sets
`memory.max`, so only a limit takes a pod out of the candidate set. A
request just moves a pod from BestEffort to Burstable, which under the
default `strictCgroupClassOrdering: true` still means killed second.
`PriorityClass` does nothing here, because the handler reads cgroupfs
and never talks to the apiserver. Victim selection is also unrelated to
the trigger, so a tenant workload thrashing against its own limit can
drive node memory pressure while a system component with no limit is
what gets killed.

An audit of a running cluster found around 70 workloads across ~30
packages with at least one container missing a memory limit. Per-chart
resources would not cover that set, and would not keep covering it as
packages are added. So the LimitRange is written by
`reconcileNamespaces`, which already creates these namespaces and
already tells system from tenant. It also covers init containers, which
#3495 can't reach: one of them without a limit leaves the pod cgroup
without `memory.max`, whatever the main containers declare.

Tenant namespaces are skipped. Where the tenant chart's own
`tenant-range-limits` exists (only when the operator configured
`resourceQuotas`), a second LimitRange next to it would leave the result
to the order LimitRanger iterates them. Where it doesn't exist, a tenant
namespace is not the place for the platform to impose a default.

### Why the default is 32Gi

The handler grants immunity for `memory.max` existing at all, whatever
its value. So the ceiling doesn't need to be fitted, and 32Gi is high
enough that nothing plausibly reaches it. A ceiling low enough to bind
would turn a rare pressure-driven kill into a deterministic one for any
component whose real usage nobody measured. This repo already has one
such case, recorded in the `vertical-pod-autoscaler` values, where a
110Mi limit OOM-killed the updater and failed HelmReleases with it.
Scheduling is unaffected, since pods are scheduled against requests and
the paired default request is 32Mi. An operator who wants the ceiling to
bind can lower it, per component in that component's chart or fleet-wide
with the knob, once real usage is measured.

### When the default is withheld

A default below a container's own request makes that pod impossible to
admit, so the operator checks before it applies. It scans Deployments,
StatefulSets, DaemonSets, CronJobs, Jobs and Pods in the namespace, init
containers included, for a container that requests more than the
configured limit and declares no limit of its own. If it finds one, it
withdraws the managed LimitRange and records a hold in the
`operator.cozystack.io/system-memory-state` annotation on the Namespace.

The hold does not clear by itself when the blocker disappears, because
an empty scan looks the same as a pod between deletion and recreation.
Lowering a previously applied limit also creates a hold, even with no
visible blocker, because LimitRanger writes the defaulted limit onto
admitted pods and the scan can no longer tell which containers declare
one. A cluster administrator releases the hold by copying the
`<hold>:<target>` token from the state into the
`operator.cozystack.io/system-memory-ack` annotation. The token is used
once and only for that hold and target. If a blocker is still visible,
the token is rejected and the hold stays. Package reconciles are
scheduled at least every five minutes, so an acknowledgement takes
effect on a quiet cluster too. The reason and the token also appear in
the operator log under `withholding the default container memory limit`.

### Namespace another LimitRange already governs

If a system namespace already has a LimitRange that says anything about
memory, at the Container or the Pod scope, the operator writes nothing
there and withdraws what it wrote before. A second default leaves the
effective value to iteration order. A foreign `max` below this default,
a `min` above the default request or a `maxLimitRequestRatio` rejects
pods outright, which takes out a whole namespace. At the Pod scope the
collision can't be decided in advance, because it depends on how many
containers a future pod has. A LimitRange that bounds only cpu or
storage coexists normally. A same-named LimitRange without the
operator's managed-by label is treated as foreign and left alone.

This matters most in `kube-system`, which is in scope because
`cozystack-scheduler` installs there, and where a distribution's or an
administrator's own LimitRange is likeliest to sit.

### Disabling

Setting `cozystackOperator.systemNamespaceMemoryLimit` to `0` disables
the feature. On the next reconcile the operator removes every LimitRange
it owns cluster-wide, found by the managed-by label. While the feature
is on, it also removes its LimitRange from namespaces no active Package
targets any more. The safety state on the Namespace stays in both cases.

### Docs

`docs/operations/system-memory-limits.md` landed in #3496 ahead of this
code, so main describes a `4Gi` default and installer values that do
nothing yet. The page is corrected to the 32Gi default and now describes
withholding, the acknowledgement, and the foreign LimitRange rule.

### Not in this PR

linstor policy. #3824 extends the controller's existing exemption from
limits to the satellite, and a linstor limit here would contradict it,
so linstor stays with that PR.

The scan reads six workload kinds through `APIReader`, uncached, on
every reconcile of every system namespace. That cost is tracked in
#3977, since the system namespace set isn't known when the cache is
built and the right fix depends on a reconcile-rate measurement nobody
has taken.

### Verification

`go vet` and `go test` pass for `cmd/cozystack-operator` and
`internal/operator`, and `helm unittest` passes on the installer chart
for the knobs unset, set, and the limit at `0`. The reconciler tests
cover withholding on an oversized request, the hold surviving the
blocker's removal, the hold on a lowered limit, acknowledgement and
token rejection, every memory field of a foreign LimitRange at both
scopes, ownership by label, the disabled sweep, and a deleting Package
not creating policy.

The withhold design has not been run on a live cluster yet. An earlier
design of this PR was, and that run does not carry over.

### Screenshots

Not a UI change.

### Downstream repositories

I walked the trigger map against the diff.
`packages/core/installer/values.yaml` gains two keys, but the
`ansible-cozystack` trigger is for a required key or a rename of one the
role passes, and these are new, optional and default to `""`, so the
role is unaffected. Nothing else in the map is reached by the Go
changes, the operator template or the page.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(operator): system namespaces now get a default container memory limit through a LimitRange maintained by cozystack-operator, which keeps system components out of the Talos userspace OOM handler's victim set. Tune it with cozystackOperator.systemNamespaceMemoryLimit (default 32Gi) and cozystackOperator.systemNamespaceMemoryRequest (default 32Mi); set the limit to 0 to disable it. The default is far above any real working set on purpose, because the handler grants immunity for a memory.max existing at all, not for its value. A namespace with a container that requests more than the limit and declares no limit of its own gets no default, and stays without one until a cluster administrator acknowledges the hold recorded on the Namespace (see docs/operations/system-memory-limits.md); lowering a previously applied limit needs the same acknowledgement. A namespace where another LimitRange already sets anything about memory is left alone. A container that declares its own memory limit is never changed.
```
2026-10-07 11:19:26 +03:00
Aleksei Sviridkin 84e8fcd477 fix(dashboard): re-authenticate on 401 instead of failing silently (#3115)
## What this PR does

When the dashboard session expires, the console now sends the user back
to sign-in instead of sitting on a stale page. This re-applies
cozystack/cozystack-ui#34 by @IvanHunters onto the in-tree console,
since the UI was vendored into the monorepo (#2963) and the standalone
repo is being archived. It is part of the cozystack-ui retirement
tracked in #3097.

When the `kc-access` cookie expires, k8s API calls return 401.
`K8sClient` now treats a 401 as an auth boundary and makes one top-level
navigation to `/oauth2/sign_in?rd=<current-url>`. That path is the
sign-in entry both gatekeeper modes serve. With OIDC enabled,
oauth2-proxy goes straight to the provider (`--skip-provider-button`)
and returns the user to `rd`. Without OIDC, token-proxy renders its
token form there, but it does not honour `rd` yet, so the user lands on
`/` after sign-in.

The handler can be replaced through `K8sClientConfig.onUnauthorized` and
does nothing outside the browser. A latch keeps concurrent 401s from
redirecting more than once, and any 2xx from either fetch site re-arms
it, so a handler that re-authenticates in place still sees the next
expiry.

Related: #2788.

### Verification

`client.unauthorized.test.ts` covers the hook on the request and watch
paths, the latch across concurrent 401s, the re-arm after a success, and
the default handler's target and relative `rd`. `pnpm typecheck` and the
console test suite pass.

Original PR: cozystack/cozystack-ui#34, credited via `Co-authored-by`.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(dashboard): on a 401 (expired session) the console now sends the user to sign in again instead of silently showing a stale page. With OIDC enabled the user returns to the page they were on; without OIDC they land on the start page after entering a token.
```
2026-10-07 11:14:37 +03:00
Aleksei Sviridkin 3e078b79a9 ci(e2e): refuse a new fork image before the build starts (#4767)
## What this PR does

A fork PR that adds a first-party image cannot pass the fork e2e lane,
and today it finds that out only after the whole build. `e2e-fork.yaml`
pushes the fork's image archives only under names the base tree already
exports, so a new image is refused at the push with `image '<name>' is
not in the base-tree allowlist`. Nothing the contributor does on the PR
changes that.

This PR keeps the push allowlist as strict as it is and moves the
failure to the start of the run. On fork PRs, `plan` in
`pull-requests.yaml` now runs `hack/fork-new-images.sh`. The script
derives the image names of the PR tree and of its base parent with the
same `make -n` derivation the push step uses. If the PR tree exports a
name the base does not, it fails before any build, names the images and
says what has to happen: a maintainer lands the image on the base branch
first, or the change runs from a branch in this repository.

The check runs from the fork's own tree, so a fork can remove it. That
is fine, because it only saves an honest contributor a wasted build. The
push allowlist in `e2e-fork.yaml` is still the boundary. That workflow
cannot call the script, since it runs from the default branch over an
untrusted checkout. So `hack/fork-new-images.bats` pins the two
derivations to each other, and also covers the script's behaviour and
the fork-only gating of the step.

I replayed the script against the merge commit of #4291 and its base
parent. It fails with `cluster-api-provider-kubevirt`, the same name the
push step refused there. The workflow_run lane itself cannot be
exercised locally.

This does not lift the limitation itself: a fork still cannot get a
green run for a new image. Part of #4329.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
ci(e2e): a fork PR that adds a new first-party image now fails in the plan job, naming the image, instead of after the whole build at the image push.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Fork pull requests that add image names not present in the base branch
are now rejected during planning, with the new names reported before
builds begin.
* **Documentation**
* Updated guidance describes the early check and clarifies how it
relates to the image allowlist used during publishing.
* **Tests**
* Added coverage for new, unchanged, reduced, and empty image sets,
along with checks that the pull request workflow applies the check only
to fork submissions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 11:14:23 +03:00
Myasnikov Daniil cc942fed68 test(tests): run unit suites with Bats and retain POSIX coverage
The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.

Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 11:04:00 +03:00
Yan Bondarenko 8ce3f853af fix(dashboard): drop the resource map ConfigMaps once the map is unreadable
React Query keeps the last successful list when a refetch fails. After the
resource map became forbidden or went away, the hook went on parsing that
old copy, so the ConfigMaps tab stayed with stale names, and a malformed
map kept its old error, instead of the tab going away as it does when the
map is unreadable from the start.

Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
2026-10-07 11:00:18 +03:00
IvanHunters 3cce629535 feat(redis,postgres): certify the per-engine external hostname (#4627)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

Redis and postgres server certificates now carry the per-engine external
name `<release>.<engine>.<apex>` (for example
`redis-foo.redis.example.org`) next to the flat `<release>.<apex>` they
already had. Both names appear under the same condition as before: TLS
on and `external: true`, with a tenant apex present.

A TLS-passthrough listener on the tenant Gateway forwards the handshake
untouched, so the certificate the client checks is the engine's own. The
design in #2815 routes databases by SNI under `*.redis.<apex>` and
`*.postgres.<apex>`, and without the per-engine name in the SAN list a
verifying client would fail the hostname check. Issuing both names now
means a later move between the two hostname schemes reissues no
certificate and keeps clients on the same trust anchor. This is the
certificate step of #2815. Nothing renders a route yet.

The SAN list changes, so an existing external TLS release gets its
server certificate reissued once on upgrade, from the same CA. For redis
the operator then rolls the Redis and Sentinel pods one at a time, the
same way it handles a renewal.

The READMEs say who can reach the new name through a passthrough
listener. For postgres that is only clients using direct TLS negotiation
(server 17+, `sslnegotiation=direct`), because by default a plaintext
SSLRequest comes before the handshake. `redis-cli` needs `--sni`.

MongoDB is not in this PR. PSMDB issues the certificate itself and its
CRD has no field for extra SANs, so adding the mongos name means the
chart taking over the certificate. That needs its own change.

Covered by helm-unittest: the name is present with TLS and external on,
and absent with external off, with TLS off, and in a tenant with no
apex.

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
feat(redis,postgres): server certificates of external redis and postgres releases also carry the `<release>.<engine>.<apex>` name that the tenant Gateway's TLS-passthrough listeners are designed to route by. Existing external TLS releases get their server certificate reissued once on upgrade, from the same CA.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* PostgreSQL TLS certificates now cover both the tenant hostname and the
per-engine `postgres` hostname for external connections.
* Redis TLS certificates now also cover the per-engine `redis` hostname
when external services are enabled.

* **Documentation**
* Clarified connection requirements for per-engine hostnames: PostgreSQL
routing requires direct TLS negotiation, PostgreSQL 17 or later, and a
compatible client; Redis clients must specify SNI.
* Clarified that the hostnames may need to be mapped manually to the
LoadBalancer address.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 10:56:34 +03:00
Aleksei Sviridkin b1c84e9c06 feat(platform, tests): select E2E packages and declare controller dependencies (#4022)
## What this PR does

E2E installs the whole platform on a fresh cluster for every PR,
whatever the PR touches, so install time and flake surface go to
packages the selected suites never use. This PR makes the E2E install
take only what the selected suites need.

`hack/select-install.sh --disabled <suites>` generates
`bundles.disabledPackages` for fresh E2E installs using an ordinary
platform variant. It keeps suite owners, runtime requirements and the
install baseline, then follows their forward `PackageSource`
dependencies.

The baseline follows the unconditional waits in
`hack/e2e-install-cozystack.bats` and stays seeded directly even where
the graph reaches the same packages. It also keeps platform services
that no suite owns but the rest of the platform uses: `multus`,
`cozystack-scheduler`, `linstor-scheduler`, `flux-plunger`, `cozy-proxy`
and `ouroboros` (the Kubernetes suites assert its hairpin NAT). Backup
suites also keep `backupstrategy-controller` and its dependencies, and
the gateway suite keeps `kubeovn-webhook`.

The E2E job now selects the suites before it installs, and passes the
complement to the platform Package as `disabledPackages`. The `full-e2e`
label and an empty selection keep the full install, and the nightly and
tag lanes pass no list.

The platform sources also change: `mongodb-application` now depends on
`mongodb-operator`, and `seaweedfs-application` on
`objectstorage-controller`. These dependencies apply to regular platform
installs and upgrades too.

The new `mongodb-operator` edge also reaches reverse test selection: a
change under `packages/system/mongodb-operator` now selects the
`mongodb` suite instead of falling back to the full suite. Backup APIs
and the gateway webhook remain suite-specific install requirements, so
they add no edges.

The disable list governs fresh installs because existing Packages carry
`helm.sh/resource-policy: keep`. Optional packages still need
`bundles.enabledPackages`. Packages that no suite owns and the baseline
does not keep stay uninstalled on purpose, also when test selection
answers with every suite (monitoring exporters like `goldpinger` and
`kubeovn-plunger`, apps without a suite, provider-specific operators).
Nothing in the run looks at them, so installing them only adds a
readiness wait, and a change to such a package gets no install check
from this gate.

### Measured

One run per variant on a 32 vCPU / 128 GB VM, container lane, the CI
artifacts of this PR, `postgres` suite selected, 55 packages disabled:

- HelmReleases 111 to 77, pods 219 to 197
- pod requests 14.8 CPU / 37.7 GiB to 13.5 CPU / 35.7 GiB
- host memory peak during install 33.4 GB to 30.2 GB
- `install-cozystack` 26m33s in both runs
- `postgres` suite 25m50s to 25m17s, passes on the reduced install

Install time did not change. The removed packages are mostly operators
and apps without instances, and what the install waits on (SeaweedFS,
LINSTOR, CDI, the Cluster API providers, Keycloak) stays in the
baseline. With one run each, differences of a minute or two are noise.

The selection, the install closure and the workflow wiring are covered
by bats tests (`hack/select-install_test.bats`,
`hack/select-e2e_test.bats`, `hack/platform-packages_test.bats`) and by
a helm-unittest suite in the platform chart.

### Screenshots

Not a UI change.

### Downstream repositories

The new dependencies ship in the platform chart. No coordinated changes
are needed in downstream repositories.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(platform, tests): E2E installs only the packages the selected suites need. MongoDB now depends on its operator, and SeaweedFS on the object storage controller.
```
2026-10-07 09:36:24 +03:00
Aleksei Sviridkin 798094e725 feat(release): add backport-audit, a pre-release backport triage tool (#3527)
## What this PR does

This adds `cmd/backport-audit`, a maintainer tool that checks, before a
release or an rc is cut, whether everything labelled for backport
actually landed on the release branch. It prints the URLs of the PRs
that did not and exits non-zero, so the cut can gate on it.
`docs/release.md` and `docs/agents/releasing.md` now send the patch-cut
check through it.

The docs it replaces inventoried backports with `gh pr list --search
"is:merged label:kind/backport"`. That lists every PR ever labelled, not
what reached the branch. Answering the real question by hand means
reading the branch history per PR, and that is how backports go missing
quietly.

```console
$ go build ./cmd/backport-audit
$ ./backport-audit release-1.4

=== release-1.4 === 30 candidate PRs: MISSING=5 dropped=4 backported=21

  MISSING -- labelled for this line, no trace of it here (5):
    https://github.com/cozystack/cozystack/pull/2704
      #2704 fix(info): use root-host for Keycloak OIDC issuer URL in tenant kubeconfig
      label=kind/backport author=myasnikovdaniil merged=2026-05-26 -- no backport PR, nothing on branch
```

A `kind/backport` label does not name a branch. `backport.yaml` resolves
the target when the PR merges, so the same label means `release-1.5` on
a PR merged in June and `release-1.6` on one merged in August. The audit
reproduces that rule instead of trusting the label text. The rule
changed once, on 2026-08-03, when 53a7fc4dc made cutting an rc freeze
the line and 6ff5b98d5 pointed the bot at the branch list. Before that
date a line exists from its first stable release, after it from the
moment its `release-X.Y` branch is pushed. The audit estimates that
moment from the earliest `vX.Y.0-rc.N` tagged commit. This is an
estimate, because the commit timestamp is not the branch-push timestamp.
The README has the full reasoning.

Landing is checked per commit, because a commit is what a backport
loses: the bot's conflict drafts stop at the first commit that does not
apply. Every non-empty commit of the original needs its own evidence on
the branch. That is reachability, an `-x` reference, or a commit with
its subject among the branch commits that are not on main. Some but not
all is PARTIAL. A multi-commit original with a merged backport PR and no
commit evidence is UNVERIFIED. Both fail the gate. A maintainer can
confirm a squashed backport with a comment on the merged backport PR
that is exactly `backport-audit: complete`.

Two more sections start from the backport PRs on the branch instead of
from labels. DUPLICATE lists an original with more than one live
backport PR and fails the gate. UNLABELLED lists backports of originals
that are not labelled for this line. It is informational and does not
change the exit code.

Exit code 2 means the audit could not give an answer. That happens when
a `gh` listing reaches its cap, since `gh` truncates silently, or when a
git read that a verdict depends on fails. The label listings are checked
against the lower of `--limit` and 1000, because GitHub search never
returns more than 1000 results.

### Verification

`cmd/backport-audit/main_test.go` covers candidate selection, line
resolution and the evidence rules. `make unit-tests` runs it through a
new `test-backport-audit` target, since `./cmd/...` is not part of
`go-unit-tests`. `hack/release-freeze-contract.bats` pins the audit's
label set equal to the one `backport.yaml` acts on, in both directions.

### Limits

A MISSING verdict is a prompt to check, not proof of absence. A hand
backport that was squashed, reworded and never linked to its original is
invisible to every evidence layer. A label added after merge is read
against the line that was current at merge time, while the bot targets
the line current when the label is added. Such a backport shows as
MISSING on the old line and under UNLABELLED on the new one. The audit
reports that something is missing, never why. `docs/release.md` now
notes that before #3155 the bot left only a "Backport failed" comment on
a conflicting cherry-pick, which is the first thing to check behind a
MISSING.

`make generate` does not apply. This is a standalone `main` package with
no API types, no CRDs and no package `values.yaml`.

### Screenshots

Not a UI change. Sample CLI output is above.

### Downstream repositories

I walked the trigger map in `docs/agents/contributing.md` against the
diff file by file. The diff adds
`cmd/backport-audit/{main.go,main_test.go,README.md}` and changes
`Makefile`, `hack/release-freeze-contract.bats`, `docs/release.md`,
`docs/agents/releasing.md` and one `.gitignore` line for the root
binary.

Nothing in it reaches a downstream repository. No package under
`packages/apps/` or `packages/extra/` is added, renamed or removed, and
no `values.schema.json`, `values.yaml` default or version enum changes,
so the website generator lists and the terraform provider schemas are
untouched. No `ApplicationDefinition` semantics, `release.prefix`,
output Secret or Service name, namespace, platform variant or bundle
changes. The paths that reach ccp and external-apps-example
(`hack/package.mk`, `hack/common-envs.mk`, `hack/update-crd.sh`) are not
touched. The Makefile change adds a `test-backport-audit` target to
`unit-tests` and changes no existing target. No node prerequisite
changes, so talm and the ansible playbooks need nothing. The website
"developer tooling" trigger covers tools contributors use on packages.
This is maintainer release tooling, and its process doc lives in this
repository only.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(release): add `backport-audit`, a maintainer tool that reports which `kind/backport` / `kind/backport-previous` PRs have not landed on a release branch (missing outright, partially backported, waiting on an open backport PR, or deliberately dropped) and exits non-zero while anything is outstanding, so a patch cut can gate on it
```
2026-10-07 08:59:38 +03:00
Aleksei Sviridkin 5486d4ffcf fix(networking): harden webhook policy assumptions (#4058)
## What this PR does

This is follow-up for
[#3997](https://github.com/cozystack/cozystack/pull/3997). It reads
ingress-nginx admission port from effective subchart values and refuses
empty value, and fixes `kubeProxyReplacement=false` status because
cozystack has no supported Service translator in that state. Gateway
test now requires one real ClusterIP and keeps same ready Pod UID, IP
and port for whole probe, so stale EndpointSlice, replaced Pod or wrong
Service can't report deny. Policy selectors and two-ended deny from
[#3997](https://github.com/cozystack/cozystack/pull/3997) stay
unchanged, and this does not claim root cause for
[#3983](https://github.com/cozystack/cozystack/issues/3983).

### Screenshots

Not applicable.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(networking): use effective ingress admission port, report unsupported Cilium service routing correctly, and reject stale backend evidence in gateway tests
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Admission webhook connectivity checks now validate the configured
Service and its exact backend endpoint throughout probing, improving
failure detection.
- Admission webhook policies now fail closed when the webhook port is
not explicitly configured.

- **Documentation**
- Clarified that supported service routing requires
`kubeProxyReplacement: true`.
- Updated guidance to explain that disabling this setting or webhook
policies does not provide a supported fallback.

- **Tests**
- Added coverage for missing webhook port configuration and strengthened
networking stability checks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 08:37:53 +03:00
IvanHunters ac4ab1c4a6 chore: drop private cluster and host names from the tree (#4461)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

I remove internal cluster names, a real hostname with its IP addresses,
internal mirror hostnames and an internal product name from comments, an
example file, test fixtures and a dead commented-out config block. Where
a stand-in is needed I use neutral test-cluster wording, RFC 2606
domains or RFC 5737 documentation addresses. I also replace two real
Keycloak usernames and a reviewer's name in code comments with
placeholders, or drop them. Rendered output does not change. The test
fixture hostnames change on the input and the expected side together,
and the unit tests of every touched package still pass.

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Updated backup, OIDC, autoscaling, and platform references to describe
test environments without naming specific development clusters.
* Replaced development-specific endpoints in the platform example with
`example.org` and documentation-only IP addresses.
  * Updated OIDC username examples.
* **Tests**
  * Updated fixture hostnames and expected URLs to use `example.org`.
* **Chores**
* Removed commented-out registry mirror entries; the active mirror
configuration is unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 08:05:19 +03:00
Aleksei Sviridkin bc9a594e47 test(clickhouse): add backup API render assertions (#4011)
## What this PR does

This adds unit tests for how the ClickHouse chart wires up the backup
API that the default backup strategy calls. The Pod of the
`cozy-default-altinity` strategy reaches the clickhouse-backup sidecar
on the named port `ch-backup-api`. It authenticates with the credentials
from the `<release>-backup-api-auth` Secret.

The new suite is
`packages/apps/clickhouse/tests/backup_api_contract_test.yaml`. It
checks that:

- the sidecar exposes `ch-backup-api` on 7171/TCP;
- `API_USERNAME` and `API_PASSWORD` come from the `username` and
`password` keys of `<release>-backup-api-auth`;
- that Secret renders with `username: backup` and a non-empty password;
- no auth Secret renders when backups are disabled.

All of this is rendered by the chart, so the unit tests catch a
regression on every pull request. The e2e assertions on the same wiring
run only when test-impact analysis selects the ClickHouse suite. They
stay as they are in `hack/e2e-chainsaw/clickhouse`, because they check
the operator-created Pod and the admitted Secret, and rendering can't do
that.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```
2026-10-07 06:58:44 +03:00
Aleksei Sviridkin a1416655f4 fix(tenant): allow tenant egress to the KubeVirt export proxy
The vm-exportproxy Ingress is served by the ingress controller in
tenant-root, and tenant egress is default-deny. Nothing allowed traffic
to virt-exportproxy in cozy-kubevirt, so the Ingress stayed unreachable
even with the right backend Service. The new policy renders wherever
allow-to-cdi-upload-proxy does, but narrows the destination to the
export proxy pods and their container port.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-07 06:23:28 +03:00
Aleksei Sviridkin c069a191fc fix(monitoring-agents): give kube-state-metrics a sized memory limit (#3846)
`kube-state-metrics` ran BestEffort with no resources at all: the
subchart ships `resources: {}` and our wrapper never set any. So it put
no `memory.max` on its pod cgroup, and the Talos userspace OOM handler
only ever selects cgroups that lack one, which left it permanently in
the victim set. Any tenant workload driving node memory pressure could
evict the source of every kube-state metric we alert on. Background is
in `docs/operations/system-memory-limits.md`.

## The numbers are measured

On a three node cluster with 322 pods and roughly 25.6k series, which is
the smallest we run: 14 day peak working set 307Mi, peak RSS 280Mi, mean
176Mi.

So the request sits above the mean at 192Mi rather than at the peak, and
the limit carries wide headroom at 2Gi. The headroom is deliberate: this
component's memory scales with the number of objects in the cluster, and
the `--metric-labels-allowlist=pods=[*],deployments=[*]` already in this
file widens every pod and deployment series further, so a cluster larger
than the one measured lands well above what was seen here. `2Gi` is 6.7x
that peak, which read as linear in object count is roughly 2100 pods and
170k series, so the limit stays a ceiling rather than a bound well past
the largest cluster this sizing was drawn from.

No CPU limit, following the node DaemonSet sizing in `c9ba7a3cc`, which
set one zero times across nineteen files. CPU is irrelevant to the OOM
ranking, and a ceiling that binds on a scraper turns into scrape
timeouts rather than backpressure.

## This supersedes #2632

#2632 proposes 256Mi, which is below the 307Mi peak measured on the
smallest cluster we have, so it would convert an eviction risk into a
reliable OOM kill. It also adds a `cpu: 200m` limit, which is the half
that buys nothing. The diagnosis there was right and it sat since May;
the numbers are what needed replacing.

## Checks


`packages/system/monitoring-agents/tests/kube_state_metrics_resources_test.yaml`
pins the limit, both requests, and the absence of a CPU limit. Mutation
checked: dropping the limit to 256Mi turns it red. The chart's suites
are 3 files and 6 tests, all green.

### Release note

```release-note
fix(monitoring-agents): kube-state-metrics now carries a memory request and limit sized from a measured peak, so it is no longer a permanent candidate for the node OOM handler
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Configured kube-state-metrics with requests of 50m CPU and 192Mi
memory, plus a 2Gi memory limit. No CPU limit is configured, allowing
CPU usage to scale as needed. These settings help ensure the service has
the memory and CPU resources it needs while keeping memory use bounded.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 06:19:17 +03:00
Myasnikov Daniil 9927c299d4 fix(operator): keep deleting packages out of memory policy creation
A foreground-deleting Package can survive long enough to reconcile
after cleanup retires its default. It must not reinstate that policy
or consume an acknowledgement while its components are being removed.
An active Package sharing the namespace can still manage the default.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:52 +03:00
Myasnikov Daniil 9d5d6d8f45 test(operator): exercise failed memory safety scans
Foreign ownership short-circuited the read-error fixtures before their
interceptors ran. Require the injected failure to occur and preserve
both policy and Namespace state, including a pending acknowledgement.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:52 +03:00
Myasnikov Daniil 5b585d6eda fix(operator): persist holds for unsafe memory defaults
An admitted Pod loses the distinction between explicit and defaulted
limits. Its disappearance cannot prove a lower default is safe for
its replacement. Keep that decision on the Namespace until an
administrator acknowledges the particular target, without waiving
current blockers or foreign policy ownership.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:51 +03:00
Myasnikov Daniil 30ecc41070 fix(operator): close memory policy safety gaps
Preserve foreign same-name objects and keep create-time SSA non-force.

Only ignore failed pods controlled by terminal Jobs.

Make the failure-containment test inject Apply errors for real.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:51 +03:00
Myasnikov Daniil 22a27e75ac test(operator): cover memory cleanup failures
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-10-07 06:06:51 +03:00
Myasnikov Daniil e83b4eb492 fix(operator): align memory policy ownership messages
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-10-07 06:06:50 +03:00
Myasnikov Daniil 0d5b2e35c4 docs(operator): align scan comments with withholding
Replace three stale descriptions of the removed ceiling-raise behavior with
the current withhold and withdraw semantics. Historical explanations of why
the raise design was rejected remain intact.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:50 +03:00
Myasnikov Daniil b3168a3824 test(operator): pin system memory defaults
Keep the parser inputs tied to the constants registered as flag defaults,
but compare the results with literal 32Gi and 32Mi expectations. A shipped
default can no longer move while the defaults case stays green.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:50 +03:00
Myasnikov Daniil 6a521f0c68 fix(operator): sweep memory defaults when disabled
List LimitRanges by the ownership label once per Package reconcile when
the feature is disabled, and remove managed objects even in namespaces
outside the current Package target set. Skip the per-namespace path while
disabled and keep cleanup failures non-fatal to namespace reconciliation.

Update the flag help, installer guidance, and operations documentation to
describe the global cleanup boundary.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:49 +03:00
Myasnikov Daniil 492c9acdd0 fix(operator): ignore failed pods from finished jobs
Record terminal Job UIDs while scanning templates, then ignore only
Failed pods owned by those Jobs. Failures from unfinished controllers and
ownerless pods remain evidence because they may still be recreated.

Also remove the stale constructor comment that still described raising a
namespace ceiling.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:49 +03:00
Myasnikov Daniil 2e3c40b125 fix(operator): ignore non-pod LimitRange scopes
Kubernetes accepts memory keys on PVC-scoped LimitRange items when a
storage bound is also present, but those quantities cannot affect pod
admission. Restrict the foreign-policy guard to Container and Pod items.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:48 +03:00
Myasnikov Daniil f6dcb69b61 refactor(operator): withhold instead of raising, and drop the grace period
The raise-and-grace machinery cannot work: LimitRanger writes the
raised limit onto the very container that justified the raise, so at
that container's next admission it carries a limit, the scan skips
containers that have one, and the raise has erased its own evidence.
The ceiling then drops on schedule and the workload is rejected
permanently. Any such machinery approximates admission logic ahead of
admission, and that is incomplete by construction: a
LimitRange default is one number for a whole namespace and cannot mean
"the configured limit, or this container's request, whichever is
larger". Only per-pod conditional defaulting expresses that, which is a
mutating webhook and not this.

The justification for the machinery had also expired. It was designed
when the default was 4Gi, where withholding was self-perpetuating for
VPA-managed namespaces. At 32Gi nothing in the tree approaches the
ceiling -- the largest static memory request is 2Gi -- and VPA sets
requests and limits together, so those containers carry their own limit
and were never blockers.

So a namespace holding a container that requests more than the
configured limit with no limit of its own is now left with no default at
all, and anything written there earlier is withdrawn. It fails in the
safe direction, which is the whole argument: a namespace with no
memory.max is a namespace behaving as it did in every release before
this, while a default below a request is a namespace whose workload
cannot be admitted. It needs no stored state and no timer, because
withholding is not a decision that has to survive between reconciles --
once the oversized request is gone the default is applied again on the
next pass.

Gone with it: the grace period, the clear-since annotation and its
clamp, the injected clock, containerMemoryDefault, raisedCeiling, and
fourteen tests. The scan stays, because deciding whether to apply or
withhold is what it was always for. The foreign-LimitRange ownership
guard, the disable path and the 32Gi default are untouched.

The residual sharp edge is stated rather than machinery: a workload
whose only evidence is a live Pod, built from a custom resource with no
template this scan can read, can still be missed if the scan runs while
that Pod is absent, and applying the default then rejects its
recreation. At 32Gi that is unreachable in this tree. It becomes
reachable if an operator lowers the knob below such a workload's
request, which the flag help, values.yaml and the operations page now
say plainly.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:48 +03:00
Myasnikov Daniil d3df7c3d4d style(operator): gofmt stray blank lines in the package reconciler
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:47 +03:00
Myasnikov Daniil 588ecbaeeb fix(operator): scan pods every reconcile, and key ownership on the label
The pod half of the scan was skipped once a namespace defaulted container
memory at exactly the configured limit, on the argument that the shape
the scan looks for could no longer be admitted there. That argument
proves such a pod cannot come to exist. It does not prove one cannot be
attempted, and a failed attempt leaves nothing behind: a workload built
from a custom resource whose request grows above the limit after its
namespace settled was rejected permanently, with no artefact anywhere
that could justify raising the ceiling again. The premise was not sound
even for existing pods, since LimitRanger reads LimitRanges through an
informer and the shape is admissible in the window before the plugin
sees the apply. The skip bought one namespaced List out of six and paid
for it with a silent permanent-rejection mode, so it is gone along with
memoryDefaultSettledAt and the two tests that pinned it. One of those
tests carried the counterexample in its own fixture: a settled 4Gi
LimitRange beside an 8Gi pod, asserted invisible.

The coexistence guard identified its own object by name. The name is not
reserved, so an administrator's LimitRange called
cozystack-system-defaults was skipped by the guard and then overwritten
by the forced apply, and because LimitRangeSpec.Limits is an atomic list
that replaced every item in it, taking their max and cpu policy along.
Ownership is now the managed-by label, which is what the delete path has
always used.

That leaves a same-named object with no label ambiguous: ours with the
label stripped, or an administrator's. The two mistakes are not
symmetric, so the ambiguous object is treated as foreign and left
entirely alone rather than adopted. Re-stamping it with the label goes,
and its test is replaced, because the same path reaches an
administrator's object and adopting it means the disable path later
deletes their policy. Declining to delete is no longer silent either.

Also: containerMemoryDefault now takes the last matching item rather
than the first, which is what LimitRanger applies, so a multi-item spec
no longer reports a ceiling the cluster is not using.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:47 +03:00
Myasnikov Daniil c6342bf885 fix(operator): bring the flag help in line with the guard and removal scope
The --help text still carried two claims that values.yaml and the
operations page had already narrowed, and it is the one surface an
operator reads without opening the tree.

Both were wrong in the direction that costs the reader something. "A
namespace where another LimitRange already defaults or caps container
memory is left alone" described the guard before it was widened, so an
administrator holding a container memory min, a maxLimitRequestRatio or
any Pod-scoped bound would read it, expect the LimitRange to be applied
there anyway, and get a namespace with no memory.max for a reason the
help says does not apply. "Removes any previously created one" is the
universal removal claim, and removal is scoped to namespaces an active
Package still targets.

Two comments corrected while here. constrainsMemory said it switched on
the item's own maps "rather than a hand-picked few", which contradicts
the hand-picked list immediately below it; what is actually true is that
a field added upstream has to be added there too, which is the reason
it has its own table test. And memoryDefaultSettledAt claimed the count
of raised namespaces falls to none once the oversized requests are
gone, which stopped being true when the scan started keeping Failed
pods: a carcass Kubernetes never collects holds both the raise and that
namespace's per-reconcile pod List open.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:46 +03:00
Myasnikov Daniil 81c12bc953 fix(operator): withhold on any foreign memory policy, not two fields
The coexistence guard treated a namespace as already-governed only when
another LimitRange carried a container memory default or max, and said
so in its comment. That is not the set of fields that collide with what
this reconciler writes. A container min above the default request
rejects the pod, and so does a maxLimitRequestRatio, since a 32Gi
default against a 32Mi default request is a ratio of 1024:1 and no cap
allows it. So the namespace-wide admission failure the guard exists to
prevent stayed reachable through fields the guard did not read, and
kube-system is where it is likeliest, this feature having deliberately
brought it in scope.

Enumerating the colliding fields does not fix it, because at the Pod
scope the collision is not decidable here. Whether a per-container
default of 32Gi breaches a pod-wide max depends on how many containers
the pod has, and the pods in question do not exist yet: any arithmetic
about them is a guess, and a wrong guess costs a namespace its entire
pod admission. So the rule is ownership rather than arithmetic. If
anything else in the namespace has an opinion about memory, at either
scope and in any field, the namespace is not this reconciler's to
default. Narrow still on the resource, because nothing here writes cpu
or storage and a LimitRange bounding those must keep coexisting. This
is the rule the tenant-namespace exclusion already rests on.

The pod scan no longer skips Failed pods, only Succeeded. A pod that
ran to completion is never recreated from its spec, but an eviction or
a node crash leaves a Failed carcass holding the same oversized request
that its controller will recreate, and that carcass is frequently the
only surviving evidence the namespace needs a raised ceiling, since the
shape the scan looks for cannot be admitted once the namespace has
settled. Dropping it lowered the ceiling under a workload that was
coming back, which is what the grace period exists to prevent.

A grace stamp dated ahead of the clock is clamped to now. Unclamped it
held the ceiling for the skew plus the grace period, silently, so an
hour of skew bought ninety minutes instead of thirty. Clamping never
shortens the wait, it only stops it being extended.

The two flag defaults are named constants the flag registration and its
test both read. The test previously duplicated the literal and still
asserted 4Gi after the default moved to 32Gi, so it misinformed a
reader about the shipped value and pinned nothing.

The disable claim is narrowed to what the code guarantees. "Removes any
previously created one" was universal, but the reconciler only runs for
namespaces an active Package still targets, so a namespace whose
Package was removed keeps its LimitRange. The flag help, values.yaml
and the operations page now say that, and name the managed-by label for
finding leftovers by hand.

The test asserting a Pod-scoped memory max should be ignored is gone,
since under the ownership rule a Pod-scoped memory bound withholds.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:06:45 +03:00
Aleksei Sviridkin 52aa837821 fix(kubernetes): force the kubevirt-csi temp NFS unmount after a timeout (#4584)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

The kubevirt-csi node plugin now unmounts its temporary NFS mount with a
timeout, and retries with `umount -f` when the timeout runs out.

For an NFS publish the plugin mounts the export root into an
`nfs-init-*` temp dir to create `/data`. It unmounted that dir in a
defer with a plain `umount`, which has no deadline. Before this change,
if the NFS server was gone by then, the publish could hang on that
`umount` forever and the temp mount stayed.

The force-unmount helper in `k8s.io/mount-utils` doesn't work for this
case. `UnmountWithForce` retries with `-f` only after its first `umount`
returns. That `umount` forks the `umount.nfs` helper. When the timeout
kills `umount`, the helper stays stuck on the dead server and keeps the
output pipe open, and `CombinedOutput` waits for that pipe, so the `-f`
retry never runs. The plugin now runs `umount` itself with `WaitDelay`
set, so the pipe is closed a second after the kill, and then runs
`umount -f` under its own timeout. A "not mounted" answer counts as
done, same as in mount-utils. `umount2` detaches the mount before it
blocks on tearing down the superblock, so the `-f` may find nothing
left.

Some umounts can't be killed at all, for example one stuck in
uninterruptible sleep. So the whole cleanup also waits at most 30s, well
inside the 2m deadline kubelet sets for NodePublishVolume (`csiTimeout`
in `pkg/volume/csi/csi_plugin.go`). Once the budget runs out the publish
goes on, and the unmount keeps running in the background and logs its
result when it ends. The temp dir is removed by that background unmount
once it succeeds, so an unmount that finishes late still cleans up after
itself. A mounter other than the exec-based one keeps its plain
`Unmount`, under the same budget.

The temp mount now uses `nosharecache`. A forced unmount runs
`nfs_umount_begin`, which kills every RPC in flight on the superblock,
and NFS shares a superblock between mounts of the same export with the
same options. Without this option, `-f` on the temp mount could fail I/O
on the pod mount this same publish creates, and on other pods on the
node that mount the same volume. The cost is a separate cache: pods that
already have `/data` mounted may not see what the migration moves in
until their attribute cache expires. The migration is a one-time
backfill, so I'm fine with that.

The leak only lives until the container restarts, because `/tmp` in the
node plugin container has no hostPath and the mounts go away with the
container's mount namespace. So this change stops leaked mounts from
piling up inside one container. For the same reason a sweep of
`nfs-init-*` dirs at startup would never find anything, so I did not add
one.

Two cases stay as they were. An `umount` that fails fast, for example
with `target is busy`, gets no force retry, is logged, and the dir stays
because `rmdir` refuses a mount point. And since the temp mount is a
hard mount, a server that disappears before the unmount still hangs the
mount and migration calls earlier in this path. That second case is
#4581.

The tests run the real exec-based mounter against a fake `umount` on
`PATH`. The fake covers a plain unmount that succeeds, one that fails
fast and gets no `-f`, and one that hangs while an orphaned child holds
its output pipe. The last one has to end in `umount -f` inside the
budget. A "not mounted" from `-f` counts as done, and any other `-f`
failure is reported and keeps the dir. Fake mounters cover the
background path: giving up on a stalled unmount, removing the dir after
a late unmount, and logging the late result. Other tests check
`nosharecache` on the temp mount and the temp dir after a failed mount.
I haven't reproduced this against a real NFS server that went away, so
it is not checked whether `umount -f` in the almalinux image gets as far
as `umount2(MNT_FORCE)` there. The tests are in a new file, so this PR
does not overlap with #4562.

Closes #4561

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

No UI change.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(kubernetes): the kubevirt-csi node plugin no longer hangs an NFS volume publish forever when unmounting its temporary mount after the NFS server went away; the unmount now times out and is retried with force
```




<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Temporary NFS mounts use isolated caching to reduce interference with
pod mounts.
* Cleanup waits up to 30 seconds for unmounting. If the limit is
reached, the publish operation returns an error while unmounting
continues in the background.
* Temporary directories are removed after a successful unmount or a
failed mount, and retained if unmounting fails.
* When a standard unmount times out, cleanup retries with a forced
unmount when available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-07 06:03:37 +03:00
Myasnikov Daniil bcfa50d7e9 test(e2e): install only the packages the selected suites need
The E2E job installs the whole platform on a fresh cluster before it
knows which suites will run, so every PR pays install time and flake
surface for packages its suites never use. Select the suites first and
pass the complement computed by select-install.sh to the platform
Package as disabledPackages.

The full-e2e label and an empty selection keep the full install, and
the nightly and tag lanes pass no list, so they are unchanged.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:53 +03:00
Myasnikov Daniil 974aaa0bac test(hack): regenerate the selective install fixture
Two optional Cluster API providers, Proxmox and in-cluster IPAM, were
added to the platform sources after the fixture was generated. The
drift test compares the fixture with the current disable list, so it
failed until they were added.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:53 +03:00
Myasnikov Daniil f46a9797b8 fix(hack): keep platform services in selective installs
Dropping every package that no suite owns also drops services the rest
of the platform relies on: multus for secondary networks, the cozystack
and LINSTOR schedulers, flux-plunger, cozy-proxy and ouroboros, whose
hairpin NAT the Kubernetes suites assert. Nothing waits for them, so no
PackageSource edge reaches them, and adding one would also change
reverse test selection.

The remaining packages stay uninstalled on purpose, also when test
selection answers with every suite: nothing in the run looks at them,
so installing them only adds a readiness wait.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:52 +03:00
Myasnikov Daniil f863967ca5 fix(hack): retain the webhook used by gateway tests
The gateway namespace-churn regression requires a registered and ready
pod admission webhook. Its availability cannot be tested when selective
installation removes the package that registers it.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:52 +03:00
Myasnikov Daniil 59555c895e fix(hack): retain controllers and APIs for selective installs
Tenant HelmReleases need a shard reconciler, and the root SeaweedFS
release creates COSI resources even when no backup suite is selected.
Without those controllers, a fresh reduced install cannot become ready.

The Kubernetes suites create worker pools through KubernetesNodes, and
MongoDB renders an operator-owned CR. Selecting only the cluster API or
application chart drops resources those suites require.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:52 +03:00
Myasnikov Daniil 064231df47 fix(hack): support BWK awk in install graph validation
BWK awk rejects literal newlines in command-line assignments. Every
non-empty selection and graph validation therefore aborts before it can
check dependencies when this awk implementation is selected.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:51 +03:00
Myasnikov Daniil 361893e6f4 fix(hack): retain backup APIs in selective installs
Backup roundtrip suites create Bucket and backup resources that the
application graph does not require. A graph-only closure therefore
omits the APIs those suites need on a fresh install.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:51 +03:00
Myasnikov Daniil 7fda27385e fix(hack): validate selective install graphs before output
Dangling references and cycles make the inventory unsuitable for a
fresh install even when the selected suite does not reach them.
Accepting that inventory would hide a broken platform graph.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:50 +03:00
Myasnikov Daniil 3766579e89 test(hack): tighten selective install assertions
PackageSource names contain dots, so regular-expression matching can
accept a different source. Selecting a suite also does not prove its
install closure retains every API that suite uses.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:50 +03:00
Myasnikov Daniil 610578be16 test(hack): verify selective install contract
A missing suite owner or a discarded discovery failure can leave a
reduced install without the APIs its tests exercise. Graph-only checks
cannot establish that the chart preserves the requested package set.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:50 +03:00
Myasnikov Daniil 4b13893911 feat(hack): generate installable selective package sets
A reduced E2E install needs the install harness prerequisites and every
owner of a selected suite, including requirements absent from the
application dependency graph.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 06:01:44 +03:00
Myasnikov Daniil b6c33926dc fix(dashboard): re-arm the 401 latch once the session recovers
unauthorizedHandled was set once per client and never cleared, so it
collapsed every later 401 too, not just the concurrent ones it was meant
to. Harmless for the shipped default handler, which navigates away and
takes the client with it, but the onUnauthorized hook exists precisely so
a caller can re-auth in place, and such a handler would fire exactly once
for the client's lifetime and then go silent.

Clear the latch on any 2xx from either fetch site: a success means the
session is alive again, so the next 401 is a new episode. The storm guard
still holds, since concurrent 401s have no success between them.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 05:58:15 +03:00
Myasnikov Daniil 7e5f9ab152 fix(dashboard): re-auth via /oauth2/sign_in so token-proxy mode works
The dashboard gatekeeper runs one of two mutually exclusive proxies,
chosen by the cluster's oidc-enabled flag (templates/gatekeeper.yaml).
/oauth2/start exists only in oauth2-proxy. The token-proxy branch
registers only sign_in, sign_out and userinfo under the /oauth2
prefix, so there /oauth2/start falls through to the catch-all
reverse-proxy handler, which does not re-verify the token. With a
present but expired cookie it proxies the request to the console
upstream, the SPA reloads, gets a 401 again and loops.

/oauth2/sign_in is the one entry point both modes implement, so the
default 401 handler goes there. oauth2-proxy's sign_in hands straight
off to OAuthStart because the gatekeeper passes --skip-provider-button,
and it resolves rd through the same app-director, so the OIDC path
behaves as it does through /oauth2/start. token-proxy serves its token
form there. It does not honour rd yet, so after sign-in the user lands
on / instead of looping.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 05:58:14 +03:00
Myasnikov Daniil badbec63fe test(dashboard): cover the 401 -> oauth2-proxy redirect path
The 401 handling is what brings an expired session back to sign-in,
and a regression in it stays invisible until a cookie expires. The
test drives K8sClient through the vi.stubGlobal("fetch", ...) harness
the sibling k8s-client tests use. It checks that the hook fires once
per 401 from both the request and the watch fetch sites, once across
concurrent 401s, never on other responses, and that the default
handler passes the current location as a relative rd, which
oauth2-proxy's open-redirect check accepts.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 05:58:14 +03:00
Myasnikov DaniilandIvanHunters 8bf78d3bfe fix(dashboard): redirect to oauth2-proxy on 401 instead of failing silently
When the kc-access cookie expires, k8s API calls return 401 and the
SPA stays on a stale page until the user clears the cache by hand.
K8sClient now treats 401 as an auth boundary and makes one top-level
navigation into the oauth2-proxy sign-in flow, passing the current
location as rd so the user comes back to the same page. The handler
can be replaced through K8sClientConfig.onUnauthorized and does
nothing outside the browser. The unauthorizedHandled latch keeps
concurrent 401s from redirecting more than once.

Re-applied from cozystack/cozystack-ui#34 onto the in-tree console
after the UI was vendored (#2963), since the source repo is being
archived. Related: #2788.

Co-authored-by: IvanHunters <49371933+IvanHunters@users.noreply.github.com>
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 05:58:13 +03:00
Myasnikov Daniil 3d901ca9b4 test(clickhouse): add backup API render assertions
The port the backup strategy dials, the two secretKeyRefs the
clickhouse-backup sidecar authenticates with, and the auth Secret that
backs them are all rendered by this chart. A helm-unittest suite
catches a regression in them on every pull request, while the e2e
assertions on the same wiring run only when test-impact analysis
selects the ClickHouse suite. The e2e assertions stay, since they
check the operator-created Pod and the admitted Secret.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 05:52:28 +03:00