Commit Graph
7777 Commits
Author SHA1 Message Date
Andrei Kvapil 57b44343f9 docs(governance): document the Reviewer role and align the maintainer roster
GitHub write access and the documented roster describe two different
projects: ten people hold write or admin on this repository without
appearing in MAINTAINERS.md, and three people listed there hold only
read. CNCF Incubation requires code ownership in GitHub to match
documented governance roles, and there is no in-between state for it.

Add the three contributors whose work is project-wide -- lexfrei,
myasnikovdaniil and IvanHunters -- as maintainers, and introduce a
Reviewers table for people who own a specific area with write access but
not the project-wide responsibilities of a maintainer. The Reviewer tier
already existed in CONTRIBUTOR_LADDER.md and had never been used; the
area-to-owner mapping stays in .github/CODEOWNERS, which this table
mirrors.

Also name the maintainers who triage security reports, so the Required
'contact information' item points at people rather than at a list.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-27 21:12:37 +02:00
Myasnikov Daniil fea40d6314 fix(ci): only overlay current-main images on main-based PRs
The PR finalize job pulls cozystack-packages:main and repoints every package
the PR did not rebuild at current-main images. On a main-based PR that is the
whole point: it stops e2e from testing last-release images for everything
outside the build matrix. On a release-line PR it is wrong — the committed refs
there are not stale, they are that line`s released digests, and the charts are
written against exactly those.

So a release-line PR was installing main`s binaries onto its own charts, and the
mismatch grows with every commit main gains. #3437 is the demonstration: a
one-line change on release-1.6 that deactivates an app failed install
deterministically, twice, with

  SchemaError(...core/v1alpha1.Option.spec): unknown model in reference:
  "...core~1v1alpha1.OptionSpec"

from main`s cozystack-controller serving an aggregated OpenAPI that branch`s
charts cannot validate against. Nothing in the PR was broken; the lane was.
Left alone this makes every 1.6 backport look red, which is when release-branch
PRs are busiest.

Both overlay steps are now gated on `github.base_ref == main`, so a release-line
PR keeps its committed digests — the behaviour that predates the overlay.
Retargeting the overlay at a per-line artifact would be better but is not
possible today: build-main.yaml publishes only cozystack-packages:main, and the
registry carries no release-* equivalent (verified against the packages repo`s
tag list: `main` plus per-PR tags, nothing else).

hack/overlay-main-images_test.bats pins the wiring per step, so adding a third
overlay step without the guard fails the suite. Mutation-checked by removing one
guard. 12/12 green; actionlint clean.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-27 21:41:57 +05:00
Andrei Kvapil a5a0449c9b test(hack): re-point the shape-2 image-ref exemplar at grafana-dashboards
The promote-retag and image-refs enumeration tests used the monitoring
grafana.tag as their canonical example of a reference living outside a
values.yaml. That file is gone now that the Grafana rebuild is dropped,
so use the grafana-dashboards tag — the same storage shape, still in
the tree — as the exemplar.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-27 13:08:50 +02:00
IvanHunters eeee87f0b1 Merge branch 'main' into feat/kubernetes-nodes-phase2b
Resolve the migrations pin conflict in packages/core/platform/values.yaml:
keep targetVersion 55 (this branch owns migration 54) and take the newer
platform-migrations image v1.6.0 from main. The in-tree image pin
intentionally lags targetVersion until the release retag, and
run-migrations.sh fails loud if a targeted migration is missing from the
image, so the lag is safe by design.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-27 12:35:15 +03:00
IvanHunters cbb2692db0 fix(platform,kubernetes-nodes): fail worker-pool adoption and unpin reads closed (#3450)
## What this PR does

Stacked on #3315. The worker-pool adoption path treats `kubectl` reads
as fail-open: `kubectl` exits 1 for NotFound and for transient server
failures (timeout, 429, etcd hiccup) alike, and three code paths
classified every non-zero read as "object absent". Each sits exactly
where absence and failure must not be confused, because "absent" there
means "do not pin" or "do not unpin":

- **Migration 54, `adopt_one`/`pin_keep`**: a transient error while
reading a worker object skipped the keep pin and the ownership rewrite,
the migration still completed and stamped version 55 (so it never
re-runs), and the parent's control-plane-only upgrade would then prune
the un-pinned live MachineDeployment — deleting every running worker VM
in the pool. The `cur=$(... || true)` owner read additionally bypassed
the foreign-owner refusal guard.
- **Migration 54, KubevirtMachineTemplate listings** (adopt loop and
`pin_pool_objects`): masked twice (`2>/dev/null` plus `|| true`), so a
failed list adopted or pinned zero KMTs and the parent upgrade would
prune a preserved older-revision KMT still referenced by an in-flight
MachineSet — permanently, since the child chart only re-emits KMTs
already annotated with its own release name.
- **Pre-delete unpin hook, existence check**: `kubectl get ...
>/dev/null 2>&1 || return 0` four lines below the comment declaring
"Failures are NOT swallowed". A timeout during uninstall read as absent,
the Job reported success, and Helm skipped the still-keep-annotated
MachineDeployment — leaking the adopted pool's worker VMs into the
tenant namespace, running and unmanaged, after the pool was deleted.

The fix applies one contract everywhere: absence is detected with
`--ignore-not-found` — a missing object is success with empty output —
so a non-zero read exit can only be a real failure and aborts (the
migration exits 1 before the version stamp so `run-migrations.sh`
retries on the next upgrade; the hook Job fails via `set -e` and is
retried by `backoffLimit` with logs retained). Two properties fall out
of choosing `--ignore-not-found` over classifying stderr text. stderr is
never captured into data: kubectl prints warnings on *successful* reads
(API deprecation headers — deterministic for a deprecated GVK — and
client-side throttling notices), and a warning glued onto a
`2>&1`-captured annotation value would trip the foreign-owner refusal or
fake a pinned state. And absence is never inferred from message text: a
transient conversion-webhook outage fails a read with `... service
"capi-webhook-service" not found`, which a `*"not found"*` match would
misclassify as absence, silently re-opening the prune hazard.

Mechanically: `adopt_one` reads `-o json --ignore-not-found` and
extracts the owner annotation with jq (already a dependency), keeping
absence distinct from present-but-unannotated; `pin_keep` reads `-o
name`; the hook's `unpin()` folds absence and "present but unannotated"
into one empty-output no-op, which is exactly its semantics, and the
whole error-classification branch disappears. `list_kmts` is the one
read that still needs stderr — for the two deterministic missing-CRD
messages (migration-47 precedent) — so it captures stderr to a temp
file, never into the data variable; KMT listings capture into a variable
first because `for x in $(...)` discards the substitution's exit status
even under `set -e`, leaving `|| true` to cover only the
empty-grep-match case. The one deliberately tolerant read — the child-HR
existence check — is annotated as such: its fallback is an idempotent
apply that itself fails closed. `adopt_one` also stamps
`meta.helm.sh/release-namespace` alongside the release name, closing the
corner where an object with out-of-band-stripped Helm annotations would
fail the child's ownership validation.

Every kubectl invocation in migration 54 and the hook is now either
fail-closed or carries a comment stating why fail-open is safe at that
call site. Fail-closed reads cut the other way for names that cannot
exist: kubectl rejects a name containing `/` client-side and
deterministically, so a pin-path read for one would deadlock the
migration on every retry. `pin_pool_objects` therefore skips statically
when the composed `kubernetes-<cluster>-<group>` is not a valid DNS-1123
subdomain — such a name provably never had an object applied under it,
so there is nothing to pin and no read to misclassify; keys that are
label-invalid but still valid object names (dotted) keep the pin
behavior. The PR also re-homes three control-plane-chart comments that
still described the talos-reconcile Job as living in that chart (it runs
per-pool in kubernetes-nodes releases since the split, and a pool-less
cluster keeps DNS SANs only — a best-effort extra nothing depends on).

Tests, each red against the code it guards (verified by running the
pre-fix scripts under the same injections):

- The migration-54 fake kubectl models absence the way real kubectl
behaves — `--ignore-not-found` present: success + empty; absent: `Error
from server (NotFound)` + exit 1 — and gains
`FAKE_GET_FAIL`/`FAKE_GET_FAIL_MSG` for transient-error injection and
`FAKE_WARN_STDERR` for success-path stderr warnings. Six new bats tests
pin the contract: an absent object still completes and stamps; a
transient error on the adopt path, the KMT listing, or the pin path
aborts before the version stamp; a deprecation warning does not break
adoption or fake a foreign owner; and a failure whose message contains
"not found" still aborts without stamping.
- New `hack/kubernetes-nodes-unpin-hook.bats` tests the hook's embedded
script behaviorally — renders the chart, extracts the Job's args,
executes them against a fake kubectl — since error classification cannot
be asserted from the manifest. Seven cases: absent objects no-op and
succeed (native pool preserved), a transient named-get error fails the
Job (the original script printed "unpin done" and exited 0 under the
same injection), a "not found"-worded failure fails the Job, a failed
KMT listing fails the Job, a success-path warning does not fake a pinned
state (the `2>&1` variant issued three spurious annotate calls),
keep-annotated objects are unpinned with the 6-hex hash anchor
respected, and an existing-but-unpinned object is untouched.

Verified locally: both migration-54 bats suites (14 + 3 tests), the hook
suite (7 tests), and `helm unittest` for both charts (kubernetes-nodes
32 tests + 12 snapshots, kubernetes 117 tests) all pass. The slash-key
deadlock was reproduced against the pre-fix script (exit 1 at the first
pin read, no stamp) before the fix turned it into a warn-and-skip.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

Walked the trigger map against the diff: a platform migration script, a
hook script inside an existing chart template, and unit-test additions
under `hack/`. No package added or renamed, no values/schema/API/README
change, no `hack/*.mk` or make-target change, no node-prerequisite
change — no trigger matches.

### Release note

```release-note
fix(platform): the worker-pool adoption migration and the kubernetes-nodes pre-delete unpin hook fail closed on transient apiserver errors instead of misreading them as absent objects, which could otherwise let a platform upgrade prune a live worker pool or let a pool deletion leak its VMs
```
2026-07-27 12:24:43 +03:00
Myasnikov Daniil 1eded27900 test(release): scope the trim pin to the parse step, fix two stale references
Three review nits, all verified against the tree, plus a tighter scope than the
first attempt at them.

The behaviour suite asserted "promote-rc does not trim the rc tag" against the
whole file. promote-rc.yaml has no .trim() today, so it passed, but any unrelated
.trim() added anywhere in the workflow would have failed a test about tag policy.
Scoping it to the parse JOB was not enough either: that job is ~180 lines, most
of them the e2e gate's github-script, whose string handling has nothing to do
with tag policy. Both greps are now scoped to the parse STEP, and run through a
code_lines filter copied from the contract suite so a YAML comment mentioning
.trim() cannot fail them.

Mutation-checked in both directions: a .trim() injected into the parse step fails
the test, one injected into the e2e-gate step of the same job does not.

- The Step 2 checklist entry still said "must be main" while Step 2 itself now
  lists three valid starting points (main, detached at the release tag, detached
  at the rc tag being promoted). The checklist is what an agent ticks off, so it
  was the half that mattered; it now points at Step 2 rather than re-encoding the
  count, so a fourth configuration cannot desync it. The heading said "Checking
  current branch", which two of the three states do not have.
- hack/testdata/needs-semantics.yaml named release-changelog-contract.bats as its
  consumer; the only consumer is release-changelog-behaviour.bats.

50/50 tests green across the three suites (11 + 24 + 15).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-27 14:03:49 +05:00
Myasnikov Daniil a2667c58ca fix(ci): grant rc-e2e the checks:write ceiling e2e-tag.yaml declares
A caller job's permissions are the ceiling for every job in the workflow it
calls, and GitHub validates that ceiling when it CREATES the run — before the
calling job's `if:` is evaluated, let alone before a step runs. e2e-tag.yaml's
e2e job declares `checks: write` for its workflow_dispatch-only breakpoint, so
granting only `contents: read` here made the run fail validation with
"requesting 'checks: write', but is only allowed 'checks: none'".

A startup failure takes the whole run with it, and an `if:` cannot gate it, so
this was not limited to rc cuts: EVERY tag push — rc and stable, including the
one finalize pushes to publish a release — would have produced no build, no
draft release and no staging branch.

The comment claimed the opposite: that because workflow_call can never reach the
breakpoint branch, the grant was unnecessary. Runtime reachability has no bearing
on a static check. e2e-tag.yaml now states the contract for its callers, next to
the permission block that was being read as "contents: read is all this needs".

hack/promote-gate-contract.bats pins both halves of the pair, and separately pins
the `E2E <tag> (full suite)` job name the gate correlates evidence by. Neither is
reachable from any PR lane — tags.yaml runs only on tag pushes — so a regression
here would otherwise surface at the next release. Mutation-checked: dropping the
grant fails the first pin, renaming the e2e job fails the second.

changelog-rc.yaml's reusable-workflow call now pins `contents: read` explicitly,
matching promote-rc.yaml's identical call. The outcome is the same either way,
but the docs describe an omitted block as the repository default rather than the
workflow's own top-level block, and this is the file class where that ambiguity
just cost a release lane.

Reported by CodeRabbit on #3449; blast radius and the missing pins found by an
adversarial re-check.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-27 14:03:49 +05:00
Myasnikov Daniil ae39475789 fix(release): prove a stable tag is absent before copying over it
Review notes from #3435, and a correction to my first attempt at them.

manifest_digest gated on skopeo`s exit status alone, so a zero exit with no
bytes hashed the empty string to sha256:e3b0c442... — a digest that looks real
and belongs to no manifest. The first fix simply required non-empty bytes, which
was worse than it looked: the caller turns an empty result into "not published,
safe to copy", so a registry answering 200 with an empty body went from an
accidental refusal (the bogus digest mismatched, so the write-once check
refused) to an actual copy over whatever the tag really held. Verified against
real skopeo 1.23: a 200 with Content-Length 0 does exit 0 with no stdout.

The helper is now explicitly three-state. A digest, or empty output for a tag the
registry PROVED absent, or non-zero for anything indeterminate — which under
set -e aborts the promotion before it writes. Absence has to be reported as
"manifest unknown"; 429, 5xx, auth and network failures all exit non-zero with
no bytes too, and reading those as "unpublished" is the same fail-open one level
down. finalize retags ~42 refs with no retry wrapper, so a single rate-limit was
enough to reach that path. skopeo`s stderr is captured instead of discarded, so
the diagnostics now name the real failure.

Tests: the write-once refusal branch the review found untested (an existing
stable tag at a different digest must be refused before any write), the empty-
body case (refuses, writes nothing, and the empty-input hash never appears in
the diagnostics), and a 429 that must not read as an absent tag. The mock now
emits realistic registry errors, since a bare non-zero exit is no longer proof
of absence. Both new guards are mutation-checked: accepting any non-zero exit as
absence fails the 429 test, dropping the empty-bytes check fails the empty-body
test.

11/11 tests green, hack/image-pin-consistency.bats still 2/2, sh -n clean.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-27 13:56:45 +05:00
Myasnikov Daniil e02adad38f fix(seaweedfs): read the chart name first-match, fail the fake on unmodelled calls
Review findings from #3436, plus what an adversarial re-check of the first
attempt at them turned up.

The chart-name extraction was a greedy sed, i.e. LAST match. Helm marshals
"config" (the release`s values) after "chart", so a values subtree that spells
chart.metadata.name shadowed the real chart name: the release then read as
non-SeaweedFS, the tenant silently vanished from the report, and the script
exited 0 -- the precise false clean this PR exists to prevent, reachable
without any corruption at all. It now takes the FIRST match. The path stays
adjacent on purpose: a looser "any name after metadata" matches
chart.templates[].name, which Helm emits immediately after metadata on every
healthy release, so the review`s suggested relaxation would have returned a
template path for every tenant. Over-strictness fails loudly and recoverably;
over-looseness deletes data quietly.

Newlines are now folded before matching, so a pretty-printed payload parses at
all rather than aborting, and whitespace around the punctuation is tolerated.
first_deployed got the same treatment -- it had the identical greedy, single-
line-only shape, and a spaced payload silently dropped the PV-vintage row from
the report.

The FATAL message names the path it read and the shape it expected, so a future
Helm format change is diagnosable instead of looking like real corruption.

Tests: three new payload shapes (spaced, pretty multi-line, and a values decoy)
each byte-compared against the same golden as the compact payload, so the shape
must make no difference to the report. Mutation-checked -- restoring last-match
extraction fails the decoy test on its own, restoring the single-line matcher
fails the pretty test.

The test fake answered any unmodelled kubectl invocation with exit 0 and empty
stdout, the fail-open shape this script was rewritten to reject, and enough to
let a newly added query pass the goldens unnoticed. Unmodelled calls now exit 97
naming the invocation. Verified inert first by instrumenting the fake: 88
invocations across the suite, none unmodelled. Also added the missing test for
the by-name PVC GET fatal path, whose FAIL mode existed with nothing behind it.

The runbook now tells the operator to read the exit code: a non-zero exit means
the table is incomplete and no step may be taken on it. That contract lived only
in the script header, while the runbook is what the operator follows.

29/29 tests green; sh -n clean.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-27 13:52:59 +05:00
Myasnikov Daniil ee838261b1 test(release): pin the promote gate with POSIX grep, not ripgrep
The rc-E2E gate contract used ripgrep for every pin. The unit-test runner
has no ripgrep, so `code_lines()` hit "rg: not found", the `|| true` on it
swallowed the 127, and every pin degraded to "0 matches" — the suite failed
on the first assertion with no hint that the tool, not the workflow, was
missing. No other hack/*.bats uses ripgrep.

Swap all pins to POSIX grep and make the filter fail closed: grep exits 1
when nothing is selected (legitimate for an empty block) and 2 on an actual
error, so only the latter propagates instead of silently yielding no lines.

Verified 13/13 green both normally and with `rg`/`fd` shimmed out of PATH to
reproduce the runner, plus the eleven suites the aborted CI run never reached
(all green, zero shim hits).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-27 12:02:53 +05:00
Timofei Larkin 616d96f6fc docs(kubernetes-nodes): align version description with skew guard
The version field's description said the pool "must match the parent
cluster's version", but assertParentVersion deliberately enforces a
different contract: a pool may lag the parent -- the normal state
during a rolling upgrade, parent Kubernetes CR first, then each pool --
and only a pool AHEAD of the parent fails the render (a symmetric
equality requirement would flip every pool to render-failure the
moment the parent is bumped). A user following the description would
believe the intended upgrade sequence leaves their pools unsupported.

Reword the @param to state the actual contract and regenerate with
cozyvalues-gen v1.6.0 (the CI-pinned version; 1.3.0 silently drops the
@immutable rules) so schema, README, Go types and the RD all follow.
Description-only: no defaults, enums or validations change, pinned by
the generate-drift CI check and the existing parent_version_skew_test
lag case the words now agree with.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-27 10:00:41 +03:00
Timofei Larkin 66c96b4329 fix(dashboard): stop requesting offline_access so logout can revoke sessions (#3447)
## What this PR does

The dashboard and linstor-gui auth-proxies (oauth2-proxy) requested the
`offline_access` scope, so Keycloak issued an **offline** refresh token.
oauth2-proxy has no server-side session store: it keeps the refresh
token in
the `kc-access` cookie and silently refreshes it (`--cookie-refresh`).
An
offline token is decoupled from the SSO session by design, so logout
(`end_session`) does not revoke it. As a result a copied `kc-access`
cookie
stayed usable for the whole offline-session lifetime (Keycloak default
~30
days) even after the user logged out — logout did not invalidate it.

This drops `offline_access` from the proxy `--scope` in both packages,
so
Keycloak issues an **online** refresh token bound to the SSO session,
which
logout revokes. The `offline_access` optional client scope is also
removed for
cleanliness (with a note on why).

Scope of the fix:

- `packages/system/dashboard` — proxy `--scope` + optional client scope
- `packages/system/linstor-gui` — proxy `--scope` + optional client
scope + ingress auth unittest

The public `kubernetes` OIDC clients (kubectl `oidc-login`) are
intentionally
left untouched: `offline_access` is legitimate there and those clients
do not
sit behind oauth2-proxy.

Notes:

- The keycloak-operator only *adds* client scopes, so removing the
optional
  scope does not strip the realm-default `offline_access` from an
already-created client. The proxy `--scope` change is what actually
prevents
  issuance.
- This prevents *new* offline tokens. Offline sessions already issued to
existing users remain valid until the realm's offline-session idle
timeout
(~30d) or explicit revocation via the Keycloak Admin REST API / Account
  Console.
- After this change dashboard session lifetime is governed by the online
SSO
session (`client.session.idle.timeout` / `client.session.max.lifespan`).

### Screenshots

Not applicable — no UI change (auth-proxy OIDC scope only).

### Downstream repositories

Walked the trigger map against the diff: the change touches only helm
templates
and a unittest under `packages/system/dashboard` and
`packages/system/linstor-gui`.
No downstream repository restates the auth-proxy OIDC scope.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(dashboard): stop requesting the offline_access scope in the dashboard and linstor-gui auth-proxies so that logout revokes the session; a copied kc-access cookie is no longer refreshable after logout. Offline sessions already issued to existing users must be revoked manually via Keycloak.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Security Improvements**
* Removed requests for offline access and persistent refresh tokens from
OIDC authentication.
* Sessions now rely on standard scopes, helping ensure logout
invalidates authentication as expected.

* **Tests**
* Updated authentication checks to verify the revised OIDC scope
configuration.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-25 08:17:13 +04:00
Timofei Larkin af2b40be9f docs(kubernetes): re-home talos-reconcile comments after split
Three template comments in the control-plane chart still described the
talos-reconcile Job as living here, but the worker split moved it to
the kubernetes-nodes chart (one run per pool release):

- talos-pki.yaml pointed at "the routing note in
  talos-reconcile-job.yaml", a file this chart no longer contains.
- cluster.yaml said "the talos-reconcile main-phase Job re-patches
  certSANs" as if this chart still runs it; it now also states the
  consequence the kubernetes-nodes side already documents -- the
  ClusterIP SAN arrives only through a pool and a pool-less cluster
  keeps DNS SANs only, a best-effort extra that nothing depends on.
- cilium.yaml credited the /etc/hosts seeding to the same Job without
  naming its new home; it also now notes why the entry is always
  present wherever Cilium runs (workers only exist through pools).

Comments only ({{/* */}} blocks, stripped at render time); no manifest
output changes, pinned by the chart's unittest suite staying green.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-25 02:40:10 +03:00
Timofei Larkin 730d006ce8 fix(platform): skip pinning names no object can have
The pin path composed "kubernetes-<cluster>-<group>" and read it back,
but a group key that cannot form a valid object name (a slash, a space,
an underscore...) provably never had objects applied under it -- the
old chart's apply would have been rejected by the apiserver. Worse,
kubectl rejects a name containing '/' client-side, deterministically,
so pin_keep's fail-closed read classified it as a real failure and the
migration exited 1 on every retry: a permanent fleet-wide pre-upgrade
deadlock over a pool that cannot exist.

Lean on the invariant instead of the read: pin_pool_objects statically
checks the composed name against the DNS-1123 subdomain shape and skips
with a note when it cannot name an object -- nothing to pin, no kubectl
call to misread. Keys that are label-invalid but still valid object
names (e.g. dotted) may have live objects and keep today's pin
behavior; the pin-path transient-error test now uses a dotted key so it
still exercises a genuine read.

The fake kubectl reproduces real kubectl's client-side slash rejection,
and a new bats case pins the skip: a "pool/a" key must warn, pin
nothing, apply nothing, and still reach the version stamp. Red against
the previous script, which deadlocked at the first pin read.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-25 02:40:10 +03:00
Timofei Larkin a54d5ca4a5 fix(kubernetes-nodes): unpin absence via --ignore-not-found
Same two holes as the migration's read classification, in the unpin
hook. Capturing stderr into $pol (2>&1) contaminated the success path:
kubectl warns on successful reads, so an unannotated object read as
"pinned" and drew a spurious annotate call -- one per named object.
And the *"not found"* pattern classified a transient conversion-webhook
failure as absence, letting the Job report success while the
keep-annotated objects leaked past the uninstall.

--ignore-not-found makes the whole classification unnecessary here:
absence folds into the same empty-output no-op branch as "present but
unannotated", which is exactly the semantics unpin() wants, and any
non-zero exit is a real failure that set -e turns into a failed Job.
The case/esac disappears; stderr stays on the Job log where it is
visible instead of parsed. This also avoids tempfile juggling under the
container's readOnlyRootFilesystem.

The bats suite gains a previously missing failure-path case and
regression tests for both holes, red against the previous commit: a
failed KMT listing must fail the Job (FAKE_LIST_FAIL), a read failure
whose message contains "not found" must fail the Job, and a
success-path stderr warning must not fake a pinned state
(FAKE_WARN_STDERR). Absence is now modeled the way real kubectl
behaves under --ignore-not-found: success with empty output.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-25 02:07:43 +03:00
Timofei Larkin bdadb3ff04 fix(platform): detect absence via --ignore-not-found
The stderr-based classification this branch introduced had two holes of
its own. Capturing stderr into the data variable (2>&1) contaminates
the SUCCESS path: kubectl prints warnings on successful reads (API
deprecation headers -- deterministic for a deprecated GVK, and CAPI
v1beta1 is one -- plus client-side throttling notices), so the warning
glued onto the owner annotation would trip adopt_one's foreign-owner
refusal on every tenant and permanently deadlock the pre-upgrade hook.
And the *"not found"* pattern over-matches: a transient
conversion-webhook outage fails a read with '... service
"capi-webhook-service" not found', which the classifier would treat as
absence -- skipping the keep pin and re-opening the exact fail-open
prune hazard this branch exists to close.

Detect absence with --ignore-not-found instead: a missing object is
SUCCESS with empty output, so a non-zero exit can only be a real
failure and aborts before the version stamp, with no stderr parsing at
all. adopt_one reads -o json and extracts the owner annotation with jq
(already a dependency), keeping absence distinct from present-but-
unannotated; pin_keep reads -o name. list_kmts still needs stderr for
the one legitimate classification -- the two deterministic missing-CRD
messages -- so it captures stderr to a temp file, never into data, and
no longer accepts NotFound-shaped text for a list.

The fake kubectl honors --ignore-not-found the way real kubectl does,
gains FAKE_WARN_STDERR (success-path stderr warnings) and
FAKE_GET_FAIL_MSG (transient error with arbitrary text). Two new bats
tests, each red against the previous commit: a deprecation warning must
not break adoption or fake a foreign owner, and a failure whose message
contains "not found" must still abort without stamping.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-25 02:07:28 +03:00
Timofei Larkin 1ca947e2fb fix(kubernetes-nodes): fail unpin reads closed
The pre-delete unpin hook's existence check swallowed every kubectl get
failure (`... >/dev/null 2>&1 || return 0`), four lines below the
comment declaring "Failures are NOT swallowed". kubectl exits 1 for
NotFound and for a transient apiserver error alike, so a timeout during
uninstall read as "object absent": the unpin was skipped, the Job
succeeded, and Helm then skipped the still-keep-annotated
MachineDeployment -- leaking the adopted pool's worker VMs into the
tenant namespace, running and unmanaged, after the pool was deleted.
The KMT-listing path was already hardened for exactly this class; the
existence check was the remaining instance.

Classify by stderr: NotFound (native pool, or already gone) stays a
no-op; any other failure fails the Job, bounded by backoffLimit and
activeDeadlineSeconds, with logs retained. The existence check and the
annotation read fold into a single request.

The embedded script's behavior cannot be asserted from the manifest, so
a new bats suite renders the chart, extracts the Job's script, and
executes it against a fake kubectl: NotFound no-ops and succeeds, a
transient named-get error fails the Job (red against the previous
script, which printed "unpin done" and exited 0), keep-annotated
objects are unpinned with the 6-hex hash anchor respected, and an
unpinned existing object is left untouched.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-25 01:45:44 +03:00
Timofei Larkin f8258c8859 fix(platform): fail migration 54 reads closed
kubectl exits 1 for NotFound and for transient server failures (timeout,
429, etcd hiccup) alike, and migration 54 treated every non-zero read as
"object absent". A transient error while reading a worker object would
skip the keep pin and the ownership rewrite, let the migration complete
and stamp version 55 (so it never re-runs), and the parent's
control-plane-only upgrade would then prune the un-pinned live
MachineDeployment -- deleting every running worker VM in the pool. The
doubly-masked KubevirtMachineTemplate listings (2>/dev/null plus
`|| true`) had the same shape: a failed list adopted or pinned zero
KMTs, permanently stranding a preserved older-revision KMT still
referenced by an in-flight MachineSet, since the child chart only
re-emits KMTs already annotated with its own release name. The
`cur=$(... || true)` owner read additionally bypassed the foreign-owner
refusal guard.

Classify reads by stderr, the migration-47 precedent: NotFound means
genuinely absent and is skipped; anything else exits 1 so
run-migrations.sh leaves CURRENT_VERSION at 54 and retries on the next
upgrade. adopt_one and pin_keep fold the existence check into the
annotation read (one request instead of two); KMT listings go through a
capture-first list_kmts helper because `for x in $(...)` discards the
substitution's exit status even under set -e, leaving `|| true` to
cover only the empty-grep-match case, mirroring the unpin hook. The
child-HR existence check stays deliberately tolerant: its fallback is
an idempotent apply that itself fails closed. adopt_one now also stamps
meta.helm.sh/release-namespace so an object whose Helm annotations were
stripped out-of-band cannot fail the child's ownership validation.

The fake kubectl now reports absent objects the way the real apiserver
does (NotFound on stderr, exit 1) and gains FAKE_GET_FAIL to inject
transient errors; four new bats tests pin the classification: NotFound
still completes and stamps, while a transient error on the adopt path,
the KMT listing, or the pin path aborts before the version stamp. Each
red against the previous script.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-25 01:45:32 +03:00
Andrei Kvapil 1c7eb22918 feat(tenant): allow egress to the grafana-dashboards plugin server
Tenant egress is default-deny plus explicit allow-to-* policies, so the
Grafana pods deployed per tenant could not reach the in-cluster plugin
server in cozy-grafana-operator. Mirror the allow-to-keycloak pattern
and cover the policy with a test.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-24 16:08:51 +02:00
Andrei Kvapil 705e39d3d7 feat(monitoring): install Grafana plugins from the in-cluster mirror
Point GF_INSTALL_PLUGINS at the grafana-dashboards static file server
instead of the public Grafana catalog, so plugin installation needs no
internet egress at pod startup and keeps working on air-gapped installs.
Route the upstream Grafana image reference through cozy-lib.image so
mirrored-registry installs keep working, and add regression tests for
the image reference and the plugin install path.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-24 16:08:51 +02:00
Andrei Kvapil 606ffd5b1b build(grafana-operator): mirror Grafana catalog plugins into the dashboards image
Download the pinned Grafana plugin zips (VictoriaLogs datasource + panels)
from the official catalog at image build time and serve them next to the
dashboards. Grafana pods can then install plugins from this in-cluster
server instead of reaching the public catalog at startup, which keeps
air-gapped and egress-restricted installs working.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-24 16:08:51 +02:00
Andrei Kvapil 9c63df7545 build(monitoring): stop building and publishing the Grafana rebuild
Remove the custom Grafana image build now that the deployment runs on the
upstream grafana/grafana image with catalog plugins. Drops the Dockerfile,
the pinned image tag, the image targets in both monitoring Makefiles, and
the build step from the root Makefile, so ghcr.io/cozystack/cozystack/grafana
is no longer produced or published.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-24 16:01:46 +02:00
Andrei Kvapil 44ff0fb286 feat(monitoring): run Grafana on the upstream image with catalog plugins
Point the Grafana deployment at the upstream grafana/grafana image and
install the VictoriaLogs datasource and panel plugins at startup via
GF_INSTALL_PLUGINS, pulled from the official Grafana plugin catalog
(https://grafana.com/grafana/plugins) instead of being baked into a
custom rebuilt image.

The VictoriaLogs datasource is now published and signed in the catalog
(victoriametrics-logs-datasource), so allow_loading_unsigned_plugins and
the custom /var/lib/grafana-plugins path are no longer needed; plugins
land in the writable default GF_PATHS_PLUGINS. Plugin versions are pinned
to releases compatible with the shipped Grafana version.

Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-24 16:00:30 +02:00
Timofei Larkin 436743eb34 feat(workloadmonitor): bucket metrics endpoint override (#3443)
## What this PR does

The cozystack-controller meters S3 bucket sizes for billing by querying
SeaweedFS `SeaweedFS_s3_bucket_(size|physical_size)_bytes` metrics from
a VictoriaMetrics vmselect it locates via the
`namespace.cozystack.io/monitoring` label on the tenant namespace. That
discovery hardcodes the assumption that the monitoring stack scraping
SeaweedFS runs in the same cluster. Some deployments host SeaweedFS —
API and metrics both — in a separate cluster, so bucket sizes must be
fetched from a remote Prometheus-compatible endpoint that discovery can
never produce.

### Configuration surface

A single controller flag, `--seaweedfs-metrics-endpoint`, exposed as a
chart value:

```yaml
cozystackController:
  seaweedfsMetricsEndpoint: "https://vm.example.com/path/to/prometheus"
```

The value is an arbitrary base URL of a Prometheus-compatible query API
— scheme, host, optional port, and an optional path prefix — and the
controller appends `/api/v1/query` to it, normalizing slashes so a
trailing slash or a path prefix never produces a doubled or dropped
slash. Basic auth can be embedded as URL userinfo
(`https://user:pass@vm.example.com/...`); net/http turns it into an
`Authorization` header on every request. TLS uses the system trust
roots; a custom CA or bearer-token support would be a follow-up flag if
a deployment needs it. The flag is validated and normalized at startup
(absolute URL, http/https, no query string or fragment), so a malformed
value fails the rollout instead of failing queries at runtime.

Cluster-wide scope is deliberate: where SeaweedFS and its metrics live
is a property of the deployment, not of any tenant, and tenant
monitoring stacks keep working and keep being discovered via the
`namespace.cozystack.io/monitoring` label for everything else — the flag
only redirects the bucket-size queries. Being a deployment argument, it
is settable exclusively by platform administrators, so no new writable
surface is exposed to tenants. Rejected alternatives: a WorkloadMonitor
spec field (bucket monitors are rendered by the tenant-facing bucket
chart, so the field would have to travel through tenant-writable values
— an SSRF primitive — and it would be near-permanent CRD surface for a
deployment-level fact) and per-namespace annotations (per-tenant
granularity that nothing needs, plus a second admin-facing configuration
mechanism to document and validate).

### Backwards compatibility

Strictly opt-in: with the flag unset — the default — resolution walks
the exact same label-discovery path and produces the same vmselect URL.
No existing deployment changes behaviour.

### Billing safety: failures no longer collapse to zero

This also fixes a latent billing bug in the existing path: any metrics
query failure (endpoint down, non-200, malformed response) produced an
empty result map, and the reconciler overwrote
`Workload.status.resources` with it — silently wiping the recorded
bucket sizes, which billing reads as an empty bucket. Now:

- `queryAllBucketMetrics` returns errors instead of swallowing them into
an empty map, so "the bucket is empty" and "the endpoint could not be
queried" are distinguishable.
- On failure — and per bucket, when a query succeeds but reports no
series for a bucket that previously had a size — the reconciler retains
the last known `s3-storage-bytes` and `s3-physical-storage-bytes` values
instead of dropping them.
- The failure is surfaced three ways: a `BucketMetricsUnavailable`
warning event on the WorkloadMonitor, an error-level log line, and an
error returned from `Reconcile`, which lands in controller-runtime's
reconcile error metrics for alerting.

Unit tests cover URL validation and path joining, flag-over-discovery
precedence, userinfo-based basic auth, and both retention scenarios
(query failure and missing series).

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the diff
(controller Go code plus the cozystack-controller chart value and arg
plumbing): none of the listed triggers match — no package added or
renamed, no CRD, `ApplicationDefinition`, or platform values change, no
`hack/` or layout change.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
feat(workloadmonitor): a new --seaweedfs-metrics-endpoint controller flag (chart value cozystackController.seaweedfsMetricsEndpoint) points bucket size metering at an explicit Prometheus-compatible endpoint for deployments where SeaweedFS and its metrics live in a separate cluster; metrics query failures now retain the last known bucket sizes and surface an event and error instead of silently zeroing billing data
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added an optional SeaweedFS metrics endpoint configuration.
  * Metrics endpoint URLs are validated and normalized before use.
* Added support for endpoint overrides when discovering bucket metrics.
* **Bug Fixes**
* Preserved previously recorded bucket sizes when metrics are
unavailable or incomplete.
* Metrics failures now trigger retries and visible warning events
without blocking other reconciliation.
  * Improved handling of invalid or unsuccessful metrics responses.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-24 17:25:26 +04:00
Myasnikov Daniil 717c8860f6 docs(release): make the release playbook coherent with the promote flow
Final coherence pass over docs/release.md (and the releasing.md
pointer) after several incremental patches to the release-cycle
rework left the whole-document narrative uneven.

- Rewrite the regular and patch release sequences so a maintainer
  reading top-to-bottom performs the actual flow: cut rc -> watch
  mandatory rc E2E -> [optional Generate RC Changelog button] ->
  promote dispatch (E2E gate; changelog generated or picked up;
  website docs PR opened and parked) -> review the promote PR (no
  E2E by default; E2E/changelog/website-docs lines) -> merge ->
  finalize -> merge the parked website PR, then announce.
- Purge the last old-model sentence: "vX.Y.0 is tagged from the
  last RC or a later commit in main" -> promoted from the rc by
  digest, tag cut write-once at the promote PR merge commit.
- State the alpha/beta contract where prereleases are discussed:
  built and published, button-only E2E (no mandatory rc-e2e), and
  no promotion path (promote-rc accepts vX.Y.Z-rc.N alone).
- Fix the Phase 4 heading (was "chore(release): cut vX.Y.Z"; the
  PR is "chore(release): promote ... -> ...") and add the missing
  website-docs line to the reviewer checklist; add a
  keep-parked-until-publish item to the pre-release checklist.
- Add cut-prerelease.yaml, changelog-rc.yaml, and
  changelog-generate.yaml to "See also" so the workflow inventory
  is complete and consistent with the "seven workflows + reusable
  changelog-generate" count.
- releasing.md: replace the stale "typically involves ... creating
  git tags / building artifacts" list (invalidated by the
  promote-not-rebuild model) with the real high-level sequence.

Every workflow/job/step name cited was verified against the YAML on
this branch. hack/release-changelog-contract.bats stays green (24/24);
no test pins docs/release.md strings, so no test change was needed.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 18:16:26 +05:00
Myasnikov Daniil 01e1e7188e ci(release): stop finalize persisting credentials so the tag push triggers tags.yaml
pull-requests-release.yaml's finalize job checked out the repo with the default
persist-credentials, leaving GITHUB_TOKEN embedded as http.extraheader. Every
later tag/branch push re-injects the CI app token via `git remote set-url`, but a
persisted extraheader silently wins over the URL credential — so those pushes
authenticated as GITHUB_TOKEN. A GITHUB_TOKEN-authenticated push creates no
workflow run (GitHub anti-recursion), which is exactly why v1.6.0's stable tag
never triggered tags.yaml: the generate-changelog and update-website-docs
backstops never ran.

Set persist-credentials: false on the finalize checkout, so the set-url app token
actually authenticates the pushes — the same handling promote-rc.yaml already
uses. Audited every git/gh step in the job: both tag steps set-url the app token
before pushing, and Ensure-maintenance-branch / Publish-draft-release / the
retag+chart steps use the app token via github-script or a separate registry
login — none depends on the persisted GITHUB_TOKEN extraheader.

Pin it in hack/promote-gate-contract.bats.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 18:16:26 +05:00
Myasnikov Daniil 936190d4b6 ci(release): open the website docs PR at promote time
The cozystack/website 'update managed apps reference' PR used to be opened only
at tag time by tags.yaml::update-website-docs — after finalize pushed the stable
tag, i.e. after the release was already published (too late to review) — and it
fetched README content from that stable tag (BRANCH=RELEASE_TAG), which finalize
only creates post-merge.

Add a promote-time website-docs job to promote-rc.yaml (needs [parse, promote])
that opens the same update-docs-vX.Y.Z PR from the release-X.Y.Z staging branch
via the website Makefile's FETCH_REF variable, so the docs are reviewable
alongside the promote PR while the stable tag is still unborn. It refreshes the
trunk pins from the staging ref before release-next so a new minor inherits fresh
Talos/cozystack pins (the v1.6 talos-pin incident class), then runs update-all
with FETCH_REF pinned to the staging branch.

The job is non-blocking: a FETCH_REF-support guard fails loudly (warn + exit 1)
when cozystack/website main predates the FETCH_REF Makefile variable — otherwise
make would silently ignore the unknown FETCH_REF override, fall back to the
not-yet-created stable tag, and update_apps.sh would warn-and-continue into STUB
docs. Any failure surfaces as a website-docs warning line in the promote PR body
(plumbed through open-pr via needs.website-docs.result) and never blocks
promotion; the tag-time job stays as the backstop and no-ops when promote already
did the work.

MERGE-TIMING CONTRACT: the website PR must NOT be merged before finalize
publishes the release — doing so flips the site's latest-version pointer to a
version whose docs and openapi are not on GitHub yet, 404ing every version-pin.
The website PR body and the promote PR body both say so in bold.

Pin the contract in hack/promote-gate-contract.bats (needs shape, FETCH_REF
invocation, the guard, non-persisted checkout credentials, the DO-NOT-MERGE body
wording, the promote-PR status line, and the tags.yaml backstop comment) and
relax the release-changelog-contract needs pin to tolerate the appended
website-docs dependency.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 18:16:26 +05:00
Myasnikov Daniil ff7ddd3c9a ci(release): gate promotion on a green rc e2e and drop the automatic full-e2e label
Promotion now assumes checks passed at rc time instead of re-running
them. At dispatch, before any staging happens, promote-rc verifies the
rc tag has a green full e2e via either evidence path: the rc's
tags.yaml push run containing a successful 'rc-e2e / E2E <tag> (full
suite)' workflow_call job, or a successful manual e2e-tag.yaml dispatch
correlated by its exact tag-bearing job name (immutable rc digests make
past evidence valid). No evidence fails the dispatch with both paths
and the remediation named. A skip_e2e_gate boolean input bypasses the
gate loudly for emergencies; the promote PR body always carries the
verification state (verified or bypassed), and 'verified' can only
originate in the gate's green branch.

The promote PR keeps the release label but no longer gets full-e2e
automatically: its tree is the rc's already-validated digests with only
tag strings rewritten, so pull-requests.yaml now runs e2e on the
release path only when a maintainer opts in by adding full-e2e. The
regular-PR path and the ordinary full-e2e forcing are untouched, and
the new labeled trigger makes a post-open label actually start the run.

A new promote-gate contract suite pins the gate's DAG position, the
override default, the label set, the body lines, and the label-gate
condition with comment-proof executable-line counts.

Assisted-By: GPT-5 <noreply@openai.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 18:16:26 +05:00
Myasnikov Daniil 2dfbeea776 ci(release): dispatch changelog generation at rc time
Extract the promote-time changelog generation into a reusable workflow
(changelog-generate.yaml, workflow_call) and add a dispatchable wrapper
(changelog-rc.yaml) so a maintainer can generate the release changelog
the moment an rc goes green, while there is still time to review it.

The reusable core keeps promote-rc's observable behaviour: detached
rc-tag checkout, read-only app token for the AI step, the
validate-changelog.sh gate, non-blocking AI failure, and the same
changelog-vX.Y.Z artifact. .release-tooling is overlaid from
github.ref_name, which under workflow_call is the caller's dispatch ref
(and coincides with the reusable workflow's own ref for a local uses:),
so "current tooling against an old rc tag" is preserved. Secrets are
declared explicitly rather than inherited, so a caller that forgets one
fails at parse time.

The rc tag is parsed by a shared hack/parse-rc-tag.sh that REJECTS
whitespace outright (a stripped 'v1. 6.0-rc.2' would silently become a
different tag). promote-rc parses it ONCE in a cheap `parse` job that
both `promote` and `changelog` depend on — so they still run in parallel
with each other while consuming one validated tag, and a whitespace tag
fails the whole promotion rather than letting promote succeed while the
changelog job rejects the same input and the PR opens empty. The
reusable core mirrors the reject-whitespace policy inline because its
parse must run before any checkout.

changelog-rc.yaml verifies the rc staging branch release-X.Y.Z-rc.N
exists UP FRONT (in parse, which generate depends on), so a missing
branch fails in seconds rather than after a 30-minute AI run, then
commits the result there as docs/changelogs/vX.Y.Z.md idempotently
(compare-before-push).

promote-rc's changelog job is a thin uses: call. Source selection —
validate-then-fall-through across the rc tag tree, the rc staging branch
(the rc-time pickup), then AI generation, discarding any invalid source
with a loud warning so it never suppresses a valid lower-priority one —
lives in hack/select-changelog-source.sh, executed after the tooling
overlay. open-pr carries the picked-up file onto release-X.Y.Z and
finalize publishes it.

Contract and behaviour bats are repointed and extended: the shared
parser, the source-selection script, and promote's parse policy are all
exercised by execution on adversarial inputs; the remaining grep pins
match executable lines with exact counts so a comment or dead block
cannot satisfy them. docs/release.md lists the new workflow and the
rc-time entry point. actionlint and zizmor (default and pedantic) are
clean.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 18:16:16 +05:00
Myasnikov Daniil 025564d961 ci(release): make full e2e mandatory for freshly cut rc tags
Wire the published-tag e2e lane into tags.yaml as a leaf rc-e2e job:
needs prepare-release, fires only for -rc. tags (alpha/beta stay
button-only via e2e-tag.yaml dispatch), and calls the reusable lane
with the pushed tag. The staging branch and published rc release both
exist before the job starts because needs waits for the whole
prepare-release job (Publish rc release + Create release branch).

rc-e2e deliberately also runs when prepare-release short-circuits via
release_exists: that flag means skip-rebuild, not skip-validation, and
revalidating a published rc is idempotent. The caller grants only
contents: read — the lane's checks: write is needed solely by its
dispatch-only breakpoint, unreachable under workflow_call.

Also correct the cut-prerelease handoff comment, which claimed
tags.yaml already ran e2e (it never did).

Assisted-By: GPT-5 <noreply@openai.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 18:16:08 +05:00
IvanHunters c15b4bab6f test(kubernetes-nodes): assert unpin hook egress destination
The CiliumNetworkPolicy test verified only that the policy selects the
unpin pod, not spec.egress, so an egress regression that left the
uninstall hook unable to reach the apiserver would still pass. Assert the
kube-apiserver egress entity.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-24 15:33:44 +03:00
IvanHunters 040ad1d20d fix(kubernetes-nodes): fail unpin hook on KMT discovery error, do not leak
The pre-delete unpin hook discovered this pool's hash-suffixed
KubevirtMachineTemplates through `kubectl get ... 2>/dev/null | grep ...
|| true`, which turned an API or RBAC failure into an empty result: the
Job then reported success while the keep-annotated templates stayed
pinned and leaked on uninstall (Helm skips keep-annotated resources).
That contradicts the unpin() contract right above it, where annotate
failures are deliberately propagated.

Split the list out of the pipe so its failure fails the Job (set -e,
bounded by backoffLimit/activeDeadlineSeconds, logs retained), while a
successful-but-empty listing still no-ops the loop.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-24 15:33:36 +03:00
IvanHunters 928df3c2d3 fix(platform): harden migration 54 against non-object and whitespace nodeGroups
The RFC-1123 key guard added in the previous round closed hostile map
keys, but migration 54 still trusted the shape of nodeGroups and of each
group value, and word-split keys on whitespace. Because it runs as the
platform pre-upgrade hook, either gap can deadlock the upgrade for every
tenant off one hand-mangled parent HelmRelease (kubectl edit bypasses the
aggregated API schema, the same vector the key guard addresses).

Shape: a non-object nodeGroups (array/string/number) crashes jq under
set -e (keys[] on an array yields indices, then .[$idx] fails); a
non-object group value aborts the value-mapping jq on `$grp | has(...)`.
Normalize both: skip the cluster for a non-object nodeGroups map, and
route a non-object group value through the same warn + pin + skip path as
an invalid name. {} / null still fall through to the implicit md0 default.

Whitespace: an unquoted `for group in $(... keys[])` word-splits (and
globs) a key like 'my pool' into fragments that each pass the RFC-1123
guard, fabricating garbage child releases. Iterate keys newline-safely so
the intact key reaches the guard. Fail-closed exit semantics are
preserved through the while pipeline.

Covered by three new bats fixtures in migration-54-adopt-path.bats, each
red against the current script and green with this change.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-24 15:33:26 +03:00
Myasnikov Daniil 5aec70b3ae ci(release): add published-tag e2e validation lane
The release bits currently get their only e2e on the promote PR, which
couples validation to promotion and leaves alpha/beta tags with no e2e
path at all. This lane decouples them: a workflow_call + dispatch
workflow that validates any PUBLISHED release tag — the digest-pinned
tree (staging branch for prereleases, the tag itself for stable), the
release's nocloud disk asset, and the full Chainsaw suite, mirroring
the proven release-path e2e from pull-requests.yaml (no TIA, no PR
context). Draft releases fail fast with a clear message: checks in the
new release model happen at rc time, not at promote time.

A dispatch-only debug input exposes the same SSH breakpoint as the PR
lane; workflow_call runs can never trip it. Artifacts are tag-suffixed.

Wiring it as the mandatory post-rc-cut validation lands separately in
tags.yaml; for alpha/beta/stable this workflow is the manual button.

Assisted-By: GPT-5 <noreply@openai.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 16:24:01 +05:00
IvanHunters 51c34c236e fix(linstor-gui): stop requesting offline_access so logout can revoke sessions
Same offline-token-survives-logout issue as the dashboard: the linstor-gui
auth-proxy requested offline_access, so a copied kc-access cookie stayed valid
for the whole offline-session lifetime after logout.

Drop offline_access from the proxy --scope and the optional client scope, and
update the ingress auth unittest accordingly.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-24 14:19:24 +03:00
IvanHunters 99936a92f8 fix(dashboard): stop requesting offline_access so logout can revoke sessions
The dashboard auth-proxy (oauth2-proxy) requested the offline_access scope,
so Keycloak issued an offline refresh token. oauth2-proxy stores that token
in the kc-access cookie (no server-side session store) and silently refreshes
it, and an offline token is decoupled from the SSO session: logout
(end_session) does not revoke it. A copied cookie therefore stayed usable for
the whole offline-session lifetime even after the user logged out.

Drop offline_access from the proxy --scope so Keycloak issues an online
refresh token bound to the SSO session, which logout revokes. Also remove the
offline_access optional client scope for cleanliness.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-24 14:19:18 +03:00
Aleksei Sviridkin a80f13405c fix(cozy-lib): converge the CA trust-anchor helper on the canonical name (#3408)
## What this PR does

Ships `cozy-lib.tls.caCertSecret`, a render-time helper that emits a
key-free trust-anchor Secret carrying only `ca.crt`. It converges the
object name on `<release>.tenant-ca`, requires the whole value to be
certificate blocks so a stray private-key header or trailing bytes are
rejected, and coerces numeric scalars before the guard runs so a numeric
value cannot slip past.

No chart calls it yet. It is the render-time producer half of the
trust-anchor contract; the controller half is #3407, now merged. The
doc-comment on the helper shows the one-line call a chart uses, and says
outright there is no caller yet, so nobody reads it as wired when it is
not.

Split out from #3299. Comments trimmed on review.

```release-note
fix(cozy-lib): the CA trust-anchor helper emits the canonical `<release>.tenant-ca` name and rejects non-certificate input.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Updated generated TLS CA Secret naming to use the
`<release>.tenant-ca` convention.
- Added/ensured a tenant CA label is set alongside the existing tenant
resource label.
  - Continued support for custom labels (caller labels still applied).

- **Bug Fixes**
- Strengthened CA certificate PEM validation to be fail-closed: requires
non-empty, complete `BEGIN/END CERTIFICATE` blocks only.
- Rejects private-key material and other malformed or improperly
structured PEM input, including unexpected extra content.

- **Tests**
- Updated TLS CA certificate fixtures and assertions to cover additional
negative cases and refined error expectations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-24 13:59:23 +03:00
Myasnikov Daniil 6fc9708514 docs(release): align validator description with validate-changelog.sh checks
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:38:59 +05:00
Myasnikov Daniil dd308644c8 ci(release): execute the release-changelog logic in tests, not just grep it
The existing suite asserts on workflow YAML. That catches a rename or a
deleted `if:` and nothing else: it cannot tell whether the shell inside a
`run:` block does what the step name claims, and it stayed green while the
logic underneath was wrong — twice, in the two preceding rounds.

Add hack/release-changelog-behaviour.bats, which runs things for real.

The GitHub Actions semantics the whole non-blocking design rests on are
now verified by executing them. hack/testdata/needs-semantics.yaml mirrors
the promote / changelog / open-pr shapes with a fatal (non-AI) step in the
changelog job, and `act` runs it. Result: open-pr DOES run when a job it
needs has failed, `needs.changelog.result` is `failure`, and — the part
that could not be read off the YAML — a failed job's outputs come back
EMPTY, while a successful job's propagate. That empty-output behaviour is
precisely why open-pr re-establishes changelog presence from disk instead
of trusting needs.changelog.outputs. Deleting the `if:` from the fixture
makes open-pr get skipped, so the gate is demonstrably load-bearing.

The git logic moves into hack/changelog-preserve.sh and is exercised
against real bare remotes: a branch that exists, one that does not, one
with no changelog, one with an empty changelog, and the full
preserve/checkout -B/restore/force-push cycle promote actually performs.
The last of these directly reproduces the regression the preserve step
exists to prevent. An empty changelog is deliberately NOT preserved — it
would suppress generation and then publish as blank notes.

finalize's release-body selection runs under node across all three inputs
(good, whitespace-only, absent) and the resulting decision table is
compared as a whole.

Replace the shell overlay with a sparse second actions/checkout of the
dispatch ref at .release-tooling. The overlay had a bootstrap problem I
introduced: the script that fetches missing tooling was itself missing
from the rc tree, so it could never have worked for exactly the trees that
need it. actions/checkout comes from the runner, not the repo, so it has
no such circularity. .release-tooling is gitignored because promote runs
`git add -A`, and the contract suite pins that.

All 21 tests mutation-checked. Breaking changelog-preserve.sh, making it
return an empty file, removing the fixture's `if:`, dropping the tooling
checkout, and renaming its path each fail the suite and pass on restore.

Not covered, and stated in the suite header rather than implied: the AI
generation step, the real GitHub API calls, and whether the sparse
checkout resolves on GitHub's runners.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:34:17 +05:00
Myasnikov Daniil 34e45295fe ci(release): fix second-round review findings
Two more review passes (Claude + Codex) on the previous commit. Codex
found the one that would have broken the very next promotion.

The agent reads its own instructions from the working tree, which is the
rc tag. Every rc cut before this change carries the old §2 of
docs/agents/changelog.md — two valid configurations, "stop" on anything
else — and a detached rc-tag checkout is exactly that "anything else". So
promoting v1.6.0-rc.4 would have presented a compliant agent with
instructions telling it to refuse, silently, producing no changelog on
every promotion of an existing rc. Verified against the tag. The same
applies to hack/validate-changelog.sh, absent from those trees entirely,
which would have failed the Verify step with "command not found" and, in
open-pr, taken the promote PR down with it. Both are now overlaid from the
dispatch ref, best-effort, in both jobs.

The structural validation added last round rejected every PATCH
changelog. It asserted a `# Cozystack vX.Y.Z` header and a 20-line floor;
patch releases use `# vX.Y.Z (date)` and v1.5.1 is a complete 19-line
release. Good AI output would have been discarded and the promote PR
mislabelled "generation failed". Measured across the tree: the header form
holds for 23 of 115 changelogs, so it was never an invariant.

The predicates now live in hack/validate-changelog.sh, tested directly
against real changelogs rather than through YAML greps — the reason the
defect shipped is that the suite pinned finalize's guard but nothing on
the promote-side validator, and stayed green when the validator was
mutated away. It accepts both header conventions, has no line floor, and
checks the version in the header, the release-link comment and the compare
link, so a changelog generated for the wrong version is caught. All 16
current-era changelogs pass; whitespace, empty, truncated and
wrong-version inputs are rejected.

Also fixed:

- A fatal artifact download in open-pr would abort the job before the PR
  step — the same wedge as a skipped job, one step lower. Download is now
  best-effort and the commit and PR body key off what is on disk, not an
  upstream job output that says nothing about whether the transfer worked.
- A pre-existing changelog (hand-written, or preserved across the
  force-push) reached the PR without ever being validated, was advertised
  with ✅ and published verbatim. It now goes through the same validator,
  as does the tags.yaml backstop.
- `gh pr view <branch>` matches closed PRs, so re-dispatching after
  closing an abandoned promote PR edited the dead PR and opened nothing.
  Branch on state explicitly.
- `gh pr edit --body` unconditionally overwrote the PR body, destroying
  the upstream breaking-change notes docs/release.md instructs reviewers
  to paste there. Refresh only while the body is still the generated one.
- open-pr consumed raw dispatch input while its siblings used the
  validated parse; the changelog job's bash parse did not trim, so an
  input with a stray space passed promote and failed there. Both fixed.
- The prompt and §2 said "trailing release-link comment"; it is the first
  thing in every changelog.

Tests grow to 16, each mutation-checked: dropping the overlay, making the
download fatal, reverting the existing-changelog check to `-s`, and
removing the backstop validator each fail the suite.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:34:17 +05:00
Myasnikov Daniil f49d54a689 ci(release): fix review findings; keep website docs at tag time
Dual-model review (Claude + Codex) of the previous two commits found seven
substantive defects, two of them destructive. Both models independently
reached three of them; the split is noted per item.

Descope the website half (Claude, verified by reproduction). The docs job
was moved to promote time, but the website Makefile pins the fetch ref
with `override BRANCH := $(RELEASE_TAG)`, which defeats the command-line
BRANCH assignment. `make -p` resolves BRANCH to v1.6.0 — a tag that
cannot exist at promote time, since promote's own validate step hard-fails
if it does. update_apps.sh logs the failed fetch and exits 0 AFTER
truncating each destination file, so every app doc would become a stub,
`changed=true` would open the PR, and the new auto-merge would land it
unreviewed. Not fixable from this repo. update-website-docs returns to
tags.yaml where RELEASE_TAG resolves to a real tag, and the finalize
auto-merge is dropped. Opening it earlier needs a cozystack/website change
first and is left as a follow-up. This also removes the unordered-sibling
race Codex found, where the release PR could merge before the docs job
opened its PR, leaving finalize nothing to merge and no retry.

Fix the non-blocking guarantee (both models). Only the AI step carried
continue-on-error; the changelog JOB did not, so a flaky `npm i -g
@github/copilot` failed it and `needs` skipped open-pr entirely — staging
branch pushed, release drafted, no promote PR. Gate open-pr on promote
alone.

Generate from a checkout the contract permits (both models). The job
checked out the rc staging branch, which docs/agents/changelog.md §2
rejects — a compliant agent stops and writes nothing, so the normal path
produced no changelog at all. Check out the rc tag detached and document
that configuration in §2. The checkout ref now comes from the validated
parse output rather than raw dispatch input.

Port, don't regenerate, in the backstop (both models). For a patch release
promoted onto release-X.Y the changelog never reaches main, so the
backstop fired and ran a second AI pass; once its PR merged,
update-releasenotes.yaml overwrote the published body with unreviewed
text. Detect the file at the tag and port it verbatim; the AI runs only
when it is absent from both main and the tag.

Survive re-dispatch (Codex). promote force-pushes the staging branch,
erasing a hand-written changelog — the documented recovery when generation
fails. Preserve an existing one across the rebuild, and stop open-pr
overwriting it with generated output.

Validate structurally, not by size (Codex). A truncated or whitespace-only
AI run passed `[ -s ]`, was committed, advertised as valid in the PR body,
and published verbatim as release notes. Assert the header, the compare
link and a plausible line count; finalize additionally refuses
whitespace-only content rather than blanking the notes.

Add hack/release-changelog-contract.bats — nine tests pinning each link in
the chain, mutation-checked: removing open-pr's if:, breaking finalize's
body read, and restoring paths-ignore each fail the suite.

Update docs/release.md (~10 sites) and docs/agents/changelog.md §2, which
described the old flow throughout.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:34:16 +05:00
Myasnikov Daniil ba67fea7df ci(release): drop paths-ignore from the release finalize trigger
The promote PR now carries docs/changelogs/vX.Y.Z.md, and Prepare stable
branch explicitly tolerates producing no tag-string changes when the
digests are already stable. Together those can yield a docs-only promote
PR, which `paths-ignore: docs/**/*` would filter out — finalize never
runs, so the stable tag is never cut and the release is never published,
with no error raised anywhere.

The `if:` gate (merged + `release` label + release-bot author) is what
actually decides whether this workflow does anything, so let it do the
filtering. The cost is a skipped job on unrelated closed PRs.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:33:10 +05:00
Myasnikov Daniil 75b5192f1b ci(release): produce changelog and docs before the release goes public
Changelog generation and the website-docs PR both lived in tags.yaml,
gated on the stable tag push. Under the promote flow that tag is created
by pull-requests-release.yaml finalize, which also publishes the release
— so both jobs could only ever run once the release was already public.
A reader hitting the release in that window saw an empty body, and the
site lagged until someone merged the docs PR.

Move both to promote time:

- promote-rc.yaml gains a `changelog` job that runs in parallel with
  `promote` and hands the file to a new `open-pr` join job, which commits
  it onto the release-X.Y.Z branch so it rides the promote PR. Generation
  is AI-driven and slow while `promote` spends its wall clock on the Talos
  assets; neither needs the other, so both start at once.
- The website-docs job moves to promote-rc.yaml too, depending only on
  `promote`. Different repo, so it cannot ride the PR — it is opened early
  and finalize merges it after the release publishes.
- finalize reads docs/changelogs/vX.Y.Z.md from the merge commit and uses
  it as the published release body. update-releasenotes.yaml is no longer
  on the critical path: it triggers on the same push and would race this
  job, and if it won it would find no published release, skip, and never
  fire again. It also only watches main, so a patch release promoted onto
  release-X.Y was never synced at all.
- tags.yaml keeps generate-changelog as a backstop only. It self-skips
  when the changelog is already on main, and covers the two cases where
  promotion does not put it there: generation failed (non-blocking), or
  the promote PR targeted a maintenance line.

Generation stays non-blocking, so a flaky AI step cannot wedge a
promotion. When it fails the promote PR body says so explicitly, and the
escape hatch is to commit docs/changelogs/vX.Y.Z.md to the branch — which
also makes a hand-written changelog first-class: if the file is already
in the rc tree, generation is skipped entirely.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:33:10 +05:00
IvanHunters 2415bc81ec test(talos-log-collector): pin listenPort override into Cilium NetworkPolicy port
The Cilium NetworkPolicy suite asserted the ingress port only at the
default 5170, so a regression hard-coding the port would still pass. Add a
listenPort-override case mirroring the DaemonSet host/containerPort test,
pinning that the override flows into spec.ingress[0].toPorts.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-23 18:19:59 +03:00
IvanHunters bf9c5d37bf fix(talos-log-collector): keep image ref out of generated schema
The first-party image-ref guard rejects values.schema.json: the annotated
image struct baked ghcr.io/cozystack/cozystack/vector into the generated
schema default, an unenumerated second copy of the runtime ref that the
promote and mirror tooling never touches. Move the reference into
images/vector.tag (the enumerated .tag storage shape, read via
.Files.Get) so the schema no longer carries it, matching every other
package. Stamp the tag from make image and regenerate schema and README.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-23 17:56:25 +03:00
Timofei Larkin 95e4faf68e feat(workloadmonitor): bucket metrics endpoint override
The controller discovers where to query SeaweedFS bucket size metrics
via the namespace.cozystack.io/monitoring label, which assumes the
monitoring stack that scrapes SeaweedFS runs in the same cluster. Some
deployments host SeaweedFS and its metrics in a separate cluster, so
add a --seaweedfs-metrics-endpoint flag to cozystack-controller (and a
matching cozystackController.seaweedfsMetricsEndpoint chart value): an
explicit base URL of a Prometheus-compatible query API, with scheme,
host, optional port and path prefix; /api/v1/query is appended. Basic
auth may be embedded as URL userinfo. The flag is validated and
normalized at startup, so a malformed URL fails the rollout rather
than the queries. Empty keeps label-based discovery unchanged.

Bucket sizes feed billing, so metrics failures no longer collapse to
"no size": previously any query failure yielded an empty result map
and the reconciler overwrote Workload resources with it, silently
wiping the recorded sizes. The reconciler now retains the last known
s3-storage-bytes and s3-physical-storage-bytes values, emits a
BucketMetricsUnavailable warning event on the WorkloadMonitor, and
returns the error from Reconcile so it lands in the reconcile error
metrics.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2026-07-23 15:35:00 +03:00
Timofei Larkin 2823b76802 feat(gateway): advertise HTTP/2 via ALPN on Gateway API listeners (#3398)
## What this PR does

Browsers negotiate HTTP/2 exclusively through TLS ALPN: if the
terminating listener does not advertise `h2`, every client silently
falls back to HTTP/1.1, with no error surfaced anywhere. Cozystack's
Gateway API path is served by Cilium's embedded Envoy, and Cilium ships
with `enable-gateway-api-alpn=false` — so every Gateway listener
advertised no ALPN protocols and all browser traffic through a Gateway
ran over HTTP/1.1 (observable in browser dev tools as `http/1.1` on
every request). The ingress-nginx path this replaces advertised `h2` out
of the box, so the migration to Gateway API silently downgraded clients:
head-of-line blocking and the ~6-connections-per-origin limit are back
for anything served through a Gateway.

This PR enables ALPN (`h2`, then `http/1.1`) on Gateway API listeners,
default-on, with an opt-out at each management layer:

- `packages/system/cilium/values.yaml` now defaults
`gatewayAPI.enableAlpn: true`, which renders `enable-gateway-api-alpn:
"true"` into the `cilium-config` ConfigMap.
- The platform chart gains `gateway.http2` (default `true`) next to the
existing `gateway.enabled`/`gateway.attachedNamespaces`. The `isp-full`
and `isp-full-generic` bundles thread it as inline component values into
the `cozystack.networking` Package CR
(`components.cilium.values.cilium.gatewayAPI.enableAlpn`); inline
HelmRelease values override the package `valuesFiles`, so an explicit
`gateway.http2: false` actually reaches the dataplane rather than being
shadowed by the new chart default. On `isp-hosted` the hosting-side CNI
owns this setting and the flag is a no-op.
- Tenant Kubernetes clusters (`packages/apps/kubernetes`) get the same
fix: when the `gatewayAPI` addon is enabled, the tenant cilium
HelmRelease now sets `enableAlpn: true`; individual clusters opt out via
`addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false`.

### Why default-on rather than opt-in

HTTP/2 for browsers was the platform's behavior before the Gateway API
migration, so keeping ALPN off would preserve a regression, not a status
quo. Defaulting on is safe because ALPN is a negotiation: clients that
cannot speak HTTP/2 simply keep negotiating `http/1.1`, exactly as
before. Operators who need the old wire behavior can set `gateway.http2:
false` platform-wide.

### Frontend vs backend hop

This change affects the client↔gateway hop only. Gateway↔backend
connections stay HTTP/1.1 unless a Service explicitly opts in per
GEP-1911 by declaring `appProtocol: kubernetes.io/h2c` on its port.
Cilium couples that backend-protocol support to the ALPN flag
(`enable-gateway-api-app-protocol` flips to true together with it), but
it only acts on Services that declare a recognized `appProtocol` value —
and no cozystack-shipped Service does today (only plain `http`/`https`
values exist in-tree, which Cilium ignores for protocol selection). So
upstreams see no change from this PR alone; h2c/gRPC backends become
possible to configure, not configured.

The setting is cluster-wide Cilium agent config, so it covers the root
Gateway and all tenant Gateways at once — Cilium offers no per-Gateway
granularity here. Flipping it re-rolls the cilium DaemonSet on upgrade
(`rollOutCiliumPods: true`), the same disruption profile as any other
cilium config change.

### How it was verified

- New helm-unittest suites:
`packages/core/platform/tests/bundles_gateway_http2_wiring_test.yaml`
asserts the Package CR carries `enableAlpn` true by default and false
with `gateway.http2: false` for both cilium bundles;
`packages/apps/kubernetes/tests/cilium_gateway_alpn_test.yaml` asserts
the tenant HelmRelease default, the addon-off case, and the
`valuesOverride` opt-out. Full suites for both charts pass (93 and 184
tests).
- `helm template` of `packages/system/cilium` confirms the value is
consumed, not just rendered: default produces `enable-gateway-api-alpn:
"true"` (and `enable-gateway-api-app-protocol: "true"`) in
`cilium-config`; the threaded `false` produces `"false"` for both.

### Screenshots

Not a UI change.

### Downstream repositories

The website's hand-written platform values table
(`content/en/docs/next/operations/configuration/platform-package.md`)
needs a row for `gateway.http2` — filed as cozystack/website#625. No
other downstream repository restates anything in this diff (no app
`values.schema.json`, ansible-consumed platform key, or hack/ tooling
changed).

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/pull/625
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
feat(gateway): Gateway API listeners now advertise HTTP/2 via TLS ALPN (h2, then http/1.1) by default, restoring the browser HTTP/2 support lost in the migration from ingress-nginx. Opt out platform-wide with gateway.http2=false, or per tenant Kubernetes cluster via addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false. Backend (gateway-to-upstream) connections are unchanged and stay HTTP/1.1 unless a Service declares appProtocol kubernetes.io/h2c.
```
2026-07-23 16:05:31 +04:00
Aleksei Sviridkin 0ba79c0168 fix(cozystack-basics): gate the hostname VAP policies on the VAP API (#3409)
## What this PR does

Wraps the route, gateway, and ingress hostname
`ValidatingAdmissionPolicy` templates in a
`.Capabilities.APIVersions.Has` gate so they render only where the VAP
API is available. Without it, an operator-generated HelmRelease with
drift detection off renders the policies out at first install on a
cluster missing the API and never adds them back. Each template gains a
sibling test asserting it renders zero documents when the API is absent.
Install-robustness fix, split out from #3299.

```release-note
fix(cozystack-basics): hostname ValidatingAdmissionPolicies render only where the VAP API is available, so a first install on a cluster without it no longer drops them permanently.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Hostname validation policies now render only when the Kubernetes
admission policy API is available, preventing unsupported policy
resources on older clusters.
* **Tests**
* Added/updated unit tests to explicitly simulate both missing and
present admission policy API capabilities, ensuring templates render (or
not) as expected.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-23 14:54:38 +03:00
Aleksei Sviridkin 5cdeba7c41 feat(cozy-lib): align the CA trust-anchor helper with the canonical name
Rename the Secret cozy-lib.tls.caCertSecret renders from "<release>-ca-cert"
to "<release>.tenant-ca", the same canonical, key-free name the
CA-extraction controller publishes, so a chart that holds its CA PEM at
render time and an operator whose CA is minted asynchronously converge on
one object a tenant can name. "<release>-ca-cert" is not a free name —
Percona Server for MongoDB already claims it with a key-bearing Secret.

Harden the write guard: require a complete BEGIN/END CERTIFICATE block,
keep failing closed on any PEM private-key header, and coerce a non-string
caCert value before trimming so an unquoted numeric YAML scalar cannot
panic the template.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-07-23 14:25:38 +03:00
Aleksei Sviridkin af397bb62c chore: ignore root-level Go binaries built from cmd/ (#3412)
## What this PR does

Adds the root-level binary names that `go build ./cmd/<name>` (without
`-o`) drops in the repository root to `.gitignore`, so `git add .`
cannot accidentally commit an ~80 MB executable.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated repository settings to exclude generated Go executables from
version control.
  * Added explanatory comments for the ignored build artifacts.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-23 14:14:54 +03:00